Something a little different

Remember when I was job searching and I would have given anything to be employed? Well, now I am employed – which is great – and a side effect of full-time employment is well, it’s full-time. Some days I miss the time I had to pursue my personal projects. I don’t, however, miss worrying about paying rent. I also don’t want to give up on my personal projects and my personal learning, which means I just need to get creative (and a little more disciplined) with the time I do have for projects like The DevRel Digest.

So this issue of The DevRel Digest is actually going to be the tutorial I have been meaning to write about mocking in Python as part of a larger series on software testing strategies. Lest you argue that isn’t a DevRel Digest, don’t worry, I have my justification ready: If you are in Developer Relations, it’s that much more critical to exercise your technical muscles and occasionally refresh your understanding of some of the more basic concepts in code. It’s also helpful to read other instructionals to inspire the tutorials you have to write for work. So, you see? It makes perfect sense to occasionally switch things up with the DevRel Digest. And anyway, this is my personal project, which means I can proceed however I wish.

And with that, let’s talk about mocking in Python testing.

Eat your vegetables: The benefits of writing tests

I don’t eat enough vegetables. I know I should eat my vegetables, but in a world of pizza and cupcakes, they are just so unappealing. And I feel so much better when I do eat the recommended serving of vegetables consistently. There are lots of different ways and opportunities to eat vegetables and I am most successful when I have found ways to make vegetables part of every meal.

It’s the same thing for writing tests. And notice I said the benefit is in actually writing them. AI is definitely something you want in your toolbox, but not every test is a nail, and as you’ve probably already discovered, when it comes to writing code, the machines often get it wrong. So sure, let AI handle the boilerplate code, but don’t trust it to be able to perform the kind of discernment relegated to humans. We are already bad at writing tests, so imagine just how bad the robots trained on our tests are at writing tests.

Besides the quantitative benefits of software testing such as bug prevention, reduction in development costs, and improved performance, the most compelling benefit of writing tests is it makes us better engineers. Writing tests forces us to ask ourselves, “What exactly is the expected behavior of this method or application?” When software becomes “difficult to test,” it is usually a good indicator of code smells and an opportunity to refactor a method or reconsider the entire design of a system.

Documentation: The less obvious benefit of writing tests

I don’t know about you, but when I pull down new code, the first thing I do is try to get the tests to run (and pass!). This confirms that I’ve at least got the foundation correctly configured. Then, after a tour through the main script, I begin to dissect the separate modules by checking out the unit tests for them.

Best practices for unit tests call for long, descriptive function names. These function names not only make verbose test output more readable and quick to assess, they also provide documentation for the function under test. Test input and test output indicate what the function expects and returns; assertions indicate the function’s intended behavior. Assertions on errors and exceptions indicate how the function handles stuff when things go wrong; assertions on side effects and calls indicate how the function fits into the rest of the code. If you’re not going to write your documentation (you never do anyway), at least write your tests.

What is mocking?

In Plato’s allegory of the cave, the Greek philosopher describes a group of people whose reality is exclusively the shadow representations of the real world as projected onto a cave wall. These projections can be really, really close to the actual thing they represent, but they’re never quite the same as the real deal – they are only shadows and they can be easily distorted or destroyed as soon as all the lights go out.

Mocking is the shadow in the cave. Mocks are representations of methods, API calls, behaviors, classes, or infrastructure that stand in for the real things for the duration of your tests. Mocking goes by other names, such as “stubbing” or “dummies,” but whatever you call it, at its core, mocking enables us to make our tests more reliable, repeatable, and resilient by temporarily replacing components that are outside of our control. We trust the makers of those components have also written thorough tests as well.

Mocking is helpful in testing because it enables us to:

• Isolate and control our code and systems under test because we don’t have to rely on outside dependencies
• Save money and performance because we’re not making real calls to APIs or databases
• Ensure security best practices by not signing into real services and not passing along real secrets
• Contain and limit our tests to just our input and output so we can make more precise assertions and trust our tests

Like all things in code, mocking has its limitations and faults. Mocks are additional code we have to maintain or an additional dependency we have to rely on, so they are always more work. Mocks can fall out of sync with reality, rendering our tests irrelevant and meaningless, which is in conflict with unit testing best practices.

And there’s no way to avoid needing mocks in modern software. So what are some best practices we can keep in mind?

Mocking best practices

• Make one precise assertion at a time: A mock will only make a bad test worse. A good test should have just one precise assertion.
• Use a mocking library: While these are usually only available for more established or well-supported APIs and libraries, a mocking library with official maintenance can be worth the tradeoff of an additional dependency.
• Mock only what you need: Wherever possible, use the real thing, and mock only what you control. Too many mocks can be a sign the test is no longer meaningful.
• Verify your mocks for accuracy: Write tests that occasionally compare mocks against the real components they represent; use “recorded” responses whenever possible.
• Keep mocks simple: When you start wondering if your test actually means anything anymore because so much is mocked, you’re probably mocking the wrong thing.
• Refactor your code: When writing tests starts to feel like solving a LeetCode problem, it’s time to revisit your code. This is the best possible outcome of writing tests.

Meet unittest.mock

From the Python documentation: “unittest.mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used.” The most useful features of the mock library are the classes Mock and MagicMock, and the patch() decorator.

The classes: Mock and MagicMock

Mock is a base class that enables you to instantiate an object that you can create attributes and methods for as you access them. A Mock object is like a ball of clay and we can shape it into different things as we need to. MagicMock is a subclass of Mock “with default implementations of most of the magic methods.” (Read the documentation here.) “Magic methods” are the ones that come baked in – methods like mathematical operations, iteration, and dictionary keys.

Want to see it in action? Fire up Python in your terminal and execute the following:

# Import the mock library

from unittest.mock import Mock

# Create a Mock integer object 

mock_number = Mock(1)

# Try to add mock_number with another integer

mock_number + 2

You should get an error like this: TypeError: unsupported operand type(s) for +: 'Mock' and 'int' – Python doesn’t even know what to do with the plus sign.

Now try the same thing with a MagicMock:

# Import the mock library

from unittest.mock import MagicMock

# Create a Mock integer object 

mock_number = MagicMock(1)

# Try to add mock_number with another integer

mock_number + 2

You shouldn’t get an error this time. You should get a MagicMock object returned to you: <MagicMock name='mock.__add__()' id='4392271440'>. The add method was implicitly created with the mock object because it is magical.

If a Mock object is a ball of clay we can shape, a MagicMock object is a ball of clay that comes with a set of moulds we can use to shape it.

The decorator: patch()

From the Python documentation:

“patch() acts as a function decorator, class decorator or a context manager. Inside the body of the function or with statement, the target is patched with a new object. When the function/with statement exits the patch is undone.”

When used, patch() creates a MagicMock object to replace the target it is called with. So if a MagicMock object is a ball of clay with a set of moulds, then patch() tells us where the ball of clay goes in relation to the rest of the code.

Get your hands dirty with mocking in Python

A little bit of context and setup

You can find the code for this tutorial in this repo on the mocking-tutorial branch.

While the repo contains a Flask application, the full web application is not necessary for this tutorial. This tutorial is concerned with the Pug class in pug.py and in particular, the methods for get_pug_facts, build_pug, and check_for_puppy_dinner. The accompanying tests are in tests/unit/test_pug.py.

get_pug_facts()makes a call to an external static API called Dog API. Dog API returns a JSON object with dog breed facts. In the case of our application, it returns facts about pugs.

On the other hand, the method build_pug uses the OpenAI client to generate an image of a pug based on the prompt we send it. This method is also making a call to an external service, however, unlike the API we are using for get_pug_facts(), OpenAPI returns a dynamic response.

The check_for_puppy_dinner method (inspired by Puppy Songs) gets the current time and compares it with the puppy dinner time supplied to an instance of Pug class. If the current time equals puppy dinner time, the function returns a happy message; if the times do not match, the function returns a sad message.

In the accompanying test, there are two versions of the tests for each of these methods. Each version is meant to illustrate a point. In both cases, there is a commented-out set of test data containing an intentional test failure. (I have done my best to indicate all of this in the comments.)

For this tutorial, I am using Visual Studio Code’s Testing interface, however, you are encouraged to use whatever tools you are already familiar with. For help configuring the Python plugin to run your tests in VS Code, check out the documentation.

Run the tests

1. In terminal in the same directory as the tutorial code, run pip install pipenv --user to install pipenv
2. Install the dependencies from Pipfile.lock with pipenv install
3. Now switch over to Visual Studio Code to run the tests. (For even more human readability, it might be easier to configure tests to View as Tree, which you can select by clicking the three dots in the upper right-hand corner of the Testing interface.)
4. Run the test by clicking the Run Test arrow icon for test_get_pug_facts(). The test should pass.
5. Now run the test for test_build_pug(). It should also pass.
6. Lastly, run the tests for test_check_for_puppy_dinner(). They should also pass.

Now that you’ve successfully run the tests, let’s take a look at how mocking is used in them.

Mocking in the test for get_pug_facts(): Autospeccing

In the test for get_pug_facts() we are using the patch() decorator. What this does is tell Python that for the code block below, replace the the Requests object in Pug.py with a MagicMock. We set autospec to True and this ensures you can only “access attributes on the mock that exist on the real class.”

Because the Mock class has the power to create attributes and methods on the fly, it’s easy to accidentally access an attribute that doesn’t actually exist. Moreover, as we refactor and update our code, it is easy for mocks to fall out of sync. Autospeccing ensures mocks remain faithful to the real classes they represent.

Notice also that one of the tests does not assert on a comparison, but on whether or not the mocked request was made with the correct URL with assert_called_with(). I’m sure you’ve seen similar tests that create a mocked response for the mocked request to return and then compares the test results against a set of expected results. This doesn’t actually test anything because the two sets of results we are comparing were created for the test. What this is actually a test of is Mock itself – it is not a test of our code.

mock_requests.get.assert_called_with(PUG_FACTS_URL)

We don’t “own” the Dog API endpoint, so we shouldn’t be mocking it. What we do own is the specific instance of Request that we create in our code, so that’s what we mock, and then we assert that the mock is called with the correct URL. The method under test (get_pug_facts()) does return a dictionary object, so a more accurate test is to assert on a comparison between the keys in the test results and the keys in the expected results.

self.assertEqual(expected_results, list(test_results.keys()))

1. Uncomment the code for test_get_pug_facts_without_autospec and run the test. It should pass.
2. Uncomment the code for test_get_pug_facts_with_autospec_error and run the test. It should raise the error Mock object has no attribute 'get_response'.

This error occurred because of the following line:

mock_requests.get_response.return_value = mocked_pug_facts_response \

get_response is not a method available on the Request class. Autospeccing prevented this test from passing silently. The test for test_get_pug_facts_without_autospec contains the same line and passed – it is a meaningless test and yet its passing creates a false sense of confidence.

Mocking in the test for build_pug(): Using the real thing

In test_build_pug() we mock the images attribute of the OpenAI object we created using the patch() decorator. We could have mocked the response the client receives, but I was pleased to discover that I could import the ImageResponse object out of the OpenAI SDK, so in the test, we use a real instance of ImageResponse with the values we give it.

mock_openai_response = ImagesResponse(created=1234, data[Image(b64_json=None, revised_prompt=None, url="https://ai-generated-pug")])

Mocking in the test for check_for_puppy_dinner(): The peculiars of patching

In order to correctly use the patch() decorator, we must use the fully qualified name of the function we are targeting. This blog post explains it best: “If a class is imported using a from module import ClassA statement, ClassA becomes part of the namespace of the module into which it is imported.”

@patch('pug.datetime', autospec=True)

This can be especially confusing when it comes to libraries like datetime.

1. Run the test for test_check_for_puppy_dinner. It should pass.
2. Run the test for test_check_for_puppy_dinner_wrong_target. It should fail.

Notice that the target class we specified is datetime.datetime. This is not actually patching the instance of datetime we are using the Pug module, therefore mock_datetime.now.return_value = data['time'] does not replace the datetime.now() in the method under test. now() is actually called and correctly returns whatever time it is at the moment check_for_puppy_dinner() is called while running the test.

@patch('datetime.datetime', autospec=True)

In summary

Mocking is an important and powerful tool in our testing toolbox and this tutorial covers just a small fraction of what the mock library that comes with unittest can do. We looked at how to use autospeccing to ensure the accuracy of our mocks, as well as what to consider when writing our assertions on mocks and how to properly patch a target function or class. Though this is just an introduction to mocking, you can probably see how the power of mocks can also be a liability, which is why it’s important to keep best practices in mind.

What are your thoughts on mocking? How will you use mocks the next time you write your tests?

More resources on mocking

• TalkPython: Testing without dependencies, mocking in Python
• Understanding the Python Mock Object Library
• The Art of Mocking in Software Testing
• “Don’t Mock What You Don’t Own” in 5 Minutes
• Python Mocking 101: Fake It Before You Make It

Events and resources and other notable things

• I am obligated to invite you to Dev Day, but even if I didn’t work for Streamlit, I would still invite you because it sounds cool.
• It’s been interesting to see digital video evolve from firewires (Remember those?) to cameras built into our laptops. So how should we be thinking about video in 2024? Check out the State of Video Report to find out.
• Hacker News can be a great place to engage developers, but how do you do it with authenticity? Dan Moore offers his thoughts on this over at The MonkCast.
• Know how we’re always talking about how DevRel needs to get comfortable with sales and marketing? Here’s a DevRel guide to all the jargon you need to know.
• Need a developer journey map template for Figma? DevRel.Agency has got you covered.
• Pydantic is a a popular data validation library for Python and here’s a tutorial on how to use it.
• Take a journey down the rabbit hole of AI hallucinations.