The Context: Test Parameterization
At a previous job, we built home gateway middleware, which had to be tested on a plethora of devices. I advised to start using pytest
for its succinctness and extensibility.
Now a typical use case was to upload new firmware to a device, boot it, and run a test suite against it. The device's IP address and the firmware binary were variables.
Thanks to pytest's hook architecture, a test suite could be extended with a number of command line options. Thanks to pytest
fixtures, it is possible to inject information into a test.
If only we could combine those two nice features to retrieve command line options from within your test....
So the idea grew to create a library to declare options, and make them available as test fixtures. I named the library 'fixtopt' because optfixt is so much harder to pronounce :). It's out in the open, because I think it is useful for more than just that one case.
Action: let's code
Let's start with an arbitrarily simple example piece of code we can test - and work our way up to the command line.
def format_message(body, recipient):
return "Hi there, {recipient},\n{body}"
Simple Test Case
def test_a_person_is_greeted_properly():
message = "Dear Lisa, there is a hole in my bucket. Regards, John"
recipient = "Lisa"
assert format_message(message, recipient)\
.startswith("Dear {recipient}")
... of course it's trivial. But let's look at what we can do with the test input variables message
and recipient
.
Extracting Test Fixtures
Pytest lets you register functions as fixtures. When executing a test function, it will take the test function's argument names, and call the test function with the results of the corresponding fixture functions.
@pytest.fixture
def message():
return "Dear Lisa, there is a hole in my bucket. Regards, John"
@pytest.fixture
def recipient():
return "Lisa"
def test_a_person_is_greeted_properly(message, recipient):
assert format_message(message, recipient)\
.startswith("Dear {recipient}")
Command Line Options
Now, we would like these fixtures to become injected by the pytest command line, like so:
pytest . --message "Nothing else to say" --recipient Lisa
And that's where this fixtopt-xtofl
library comes in: add a file conftest.py
in your test directory, and declare the options:
# conftest.py
from fixtopt import Option, register
def pytest_options(parser):
register(globals(), parser, (
Option(
name="message",
default="Dear Lisa, there is a hole in my bucket. Regards, John",
help="the message to send"),
Option(
name="recipient",
default="Olav",
help="the one to adress"),
))
And there you go! pytest
picks it up, adds it to its own help text and you can just use the options as you like!
/path/to/testdir> pytest --help
...
custom options:
--message=MESSAGE the message to send
--recipient=RECIPIENT
the recipient
...
Running the test (with a broken implementation):
/path/to/testdir> pytest . --message "whatever" --recipient "Lisaz"
==== test session starts ====
platform linux -- Python 3.6.9, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /path/to/testdir
collected 1 item
test_messageformat.py F [100%]
==== FAILURES ====
____ test_a_person_is_greeted_properly ____
message = 'whatever', recipient = 'Lisaz'
def test_a_person_is_greeted_properly(message, recipient):
> assert format_message(message, recipient)\
.startswith("Dear {recipient}")
E AssertionError: assert False
E + where False = <built-in method startswith of str object at 0x7f9a80faa670>('Dear {recipient}')
E + where <built-in method startswith of str object at 0x7f9a80faa670> = 'Dear, {recipient},\n{body}'.startswith
E + where 'Dear, {recipient},\n{body}' = format_message('whatever', 'Lisaz')
test_messageformat.py:5: AssertionError
==== short test summary info ====
FAILED test_messageformat.py::test_a_person_is_greeted_properly - AssertionError: assert False
==== 1 failed in 0.07s ====
Agreed, the resulting failure details are a bit painful to the eyes, but this has nothing to do with the command line arguments or the fixtures.
Conclusion
Well... this is all. A small and simple library to add command line parameters to your tests. It's not feature-complete yet, but it gets the job done. Do take a look, try it out, file an issue, add a pull request. It can only get better.
Top comments (0)