This week, I set up a testing framework for my open source project, go-go-web.
For this I chose pytest, a very popular Python testing framework. I watched this getting started with pytest video by anthonywritescode, which gave a quick intro into setting up pytest.
pytest for Unit Testing
To install pytest, I simply ran this command in my VS Code terminal.
pip install -U pytest
Next I created a sample source code file and unit test file to try out pytest:
# src/sample.py
def square(x: float) -> float:
return x * x
# tests/sample_test.py
from src import sample
def test_square():
assert sample.square(5) == 25
Running pytest:
pytest
# Alternative
python -m pytest
# Run with detailed view
pytest -vv
# Run for one specific file
pytest <test file>.py
# Run showing print statements
pytest -s
The result:
Automate pytest for GitHub
To setup pytest to run on each GitHub code push / pull, see my post on CI Workflows here
Test with temporary files and directories
My app converts a .txt or .md file into .html. To test this core feature, I used pytest's tmpdir fixture which allows for testing using temporary directories and files. Here is a quick example of how it works:
# src/sample.py
def write_to_file(fname):
with open(fname, 'w') as fp:
fp.write('Hello\n')
# tests/sample_test.py
from src import sample
def test_write_to_file(tmpdir):
file = tmpdir.join('output.txt')
sample.write_to_file(str(file))
assert file.read() == 'Hello\n'
Execute start up and clean up code before and after each test
You can also use fixtures to execute start up and clean up code that runs before and after each test function. Here is an example where I implement a fixture to run clean up code which reverts my CWD back to its original value.
import pytest
@pytest.fixture
def cleanup():
# Startup code
old_cwd = os.getcwd() # Save the original CWD
yield
# Cleanup code
os.chdir(old_cwd) # Change CWD back to original
@pytest.mark.usefixtures('cleanup')
def test_some_function():
...
# Change CWD to something else
os.chdir(...)
assert some_expression
Without the cleanup fixture above, if my assert failed there is no way for me to change the CWD back to its original value since pytest will exit from the test_some_function() function.
Test the console output
The pytest capfd fixture allows us to test what is printed out to the console. Here's an example:
# src/sample.py
def hello():
print("hello")
# tests/sample_test.py
from src import sample
def test_hello(capfd):
sample.hello()
out, err = capfd.readouterr()
assert out == "hello\n"
Troubleshooting with pytest
Import error that only happens with pytest
pytest was getting an ImportError when running an import statement in one of my source code, even though that import statement has no issue running when I run the code outside pytest. Here is an example of the issue:
# src/sample.py
import utils as ut # <---- throws exception only with pytest
def hello():
print("hello")
# tests/sample_test.py
from src import sample
def test_hello(capfd):
sample.hello()
out, err = capfd.readouterr()
assert out == "hello\n"
As you can see, the import statement throws an exception when run with pytest, but does not throw any exception when run normally with python src/sample.py
:
After some digging online and trying a bunch of things, I found this solution:
Create a pytest.ini
file in the root folder with the following contents. (src
is the relative path from the root folder to your source code directory and tests
is the path to your test files directory)
[pytest]
pythonpath = src
testpaths = tests
An alternative solution would be to manually run the following code in the terminal before running pytest. This would have to be done every time you open a new terminal.
$env:PYTHONPATH = "<absolute path to your source code folder>"
Using venv
It's also beneficial to install pytest only inside a venv rather than globally just to avoid any potential issues or conflicts. If you get errors with pytest, you may want to try this:
- Exiting and deleting any virtual env you are in
- Uninstalling pytest with
pip uninstall pytest
- Recreating the venv with
python -m venv .venv
- Open a new VS Code terminal to enter the new venv
- Inside the venv, run
pip install pytest
Refactoring with pytest
It is possible to refactor pytest test code by creating a conftest.py
file at the root. In this file, you can declare global variables, helper class and methods, and fixtures that can be accessed by all your test files.
Global variables
# conftest.py
def pytest_configure():
pytest.var1 = 5
pytest.var2 = 25
# tests/sample_test.py
from src import sample
def test_square():
assert sample.square(pytest.var1) == pytest.var2
Helper class
# conftest.py
class Helpers:
@staticmethod
def file_contents(file_path):
with open(str(file_path), "r", encoding="utf-8") as file:
return file.read()
@pytest.fixture
def helpers():
return Helpers
# tests/convert_test.py
from src import convert
def test_main(tmpdir, helpers):
...
c.main(...) # this generates an output file test.html
output_path = tmpdir.join('test.html')
assert helpers.file_contents(output_path) == expected_contents
Check code coverage with coverage.py
Using coverage.py we can also check how much of our source code is covered by our unit tests.
Install: pip install coverage
Run: coverage run -m pytest
Reflection
While writing my test cases I learned a lot about how to use pytest. I got stuck a few times while trying to deal with the pytest import error and trying find ways refactor my code, and I have detailed my solutions in the above sections.
I also learned how to separate my core code into a main() function so that I could test it with pytest. I found a way to manually pass a list of arguments into my main function to mimic running my program on the command line with command line arguments.
For example, this test mimics the command line program call
python src/convert.py -o <tmp_output_dir> <tmp_input_file>
def test_main(tmpdir, helpers):
arguments = c.parse_args([
"-o",
str(tmpdir),
helpers.new_file(tmpdir, "test.txt",
pytest.simple_txt_contents)
])
# Run main function
c.main(arguments)
# Compare actual output file contents to expected
output_path = tmpdir.join('test.html')
assert helpers.file_contents(output_path) ==
pytest.simple_html_from_txt
I also uncovered a bug which I would never have found otherwise. The only reason the bug was found was because the tmp files I created using pytest were on C:\
and my repo was located in F:\
. If I was not using pytest to create tmp files, I may not have thought to test with files located on a different drive.
I learned a lot about pytest and unit testing and I hope this blog post will help someone with their unit testing. I may look into contributing to some unit test related issues in open source repos!
Top comments (0)