DEV Community

You are mocking it wrong.

Anton Sizikov on December 27, 2017

Well, probably you are not, but let me grumble a little bit anyway. Mockingbird knows how to mock. I've been working with various code bases thr...

Read full post

scottshipp • Dec 30 '17

I usually find a fundamental misunderstanding of mocking. Use of mocks isn't a problem. Poor use of mocks is a problem. Mocks aren't there so you can test. Mocks are there so you can test only what you want to test. They're there so you can isolate the thing being tested from the thing(s) not being tested.

So what do you want to test?

That changes with the context. Unit tests? I want to test a "unit" of code. What is "unit"? That varies. Integration tests? I want to test the integration of two or more things. System test? I want to test the system! End-to-end, I might be testing more than the system.

Back to the important point. It doesn't matter what size test I have, I still need to isolate the thing I'm testing. Mocks (or stubs or dummies or fakes or etc.) are one way to do it.

The key is: any time a mock isn't used to isolate the subject under test it's being used wrong. Every article I've ever seen about mocks gets this wrong. They always show an example of a mock completely isolating away the subject under test (the "doesn't test anything" ClientTests shown above), isolating the wrong things, or isolating in the wrong way.

Check out Gerard Meszaros' xUnit Test Patterns book. It will change your life.

Anton Sizikov • Dec 30 '17

Yup, totally agree. Mocking is a great tool. It's misused in one way or another in almost every codebase I've ever worked with.

Idan Arye • Dec 28 '17

You don't want to make HTTP calls while running your unit tests suite, you don't want to send out emails, neither do I.

Why wouldn't you though? I mean, obviously you wouldn't want to make HTTP calls to a production server and you wouldn't want to send emails to the actual clients - but what's wrong with sending mails to some testing account or to your own dummy mail server like FakeSMTP, and verifying that you receive these mails?

Anton Sizikov • Dec 28 '17

I actually have a set of end-to-end tests where we send emails and verify their content. There is a problem - emails go through the MailChimp and on test environment it may take quite some time.

Anyway, that's not the point of the article :) I see the value in sending the email out as well I see the value in a quick test run. We have to be practical and try to find the balance.

Mocking out the MailChimp client/queue manager/database/http call/ domain boundary is something we have to do if we want to run tests often, and be able to execute the test suite offline (I work on a plane sometimes :) ). I just want people to stop overusing mocks.

Idan Arye • Dec 28 '17

I agree - my philosophy is that when you need to mock you should consider making it an integration test.

My point is that even when you absolutely can't do the same thing the actual production program will do(e.g. - sending emails), you don't always have to do a traditional mock objects. Maybe going through an actual mail service makes the cycle too long, but if you set up your own local "mail service"(with FakeSMTP or something similar) you can send and verify the emails quickly enough, and it should be much easier to do than to use a mock email client - and also be a better test since it tests against a real API.

Another example - instead of mocking your model classes, set up a small database server. If your ORM is opaque enough you may even use an in-process database like SQLite or H2. It should have orders of magnitude less entries than a real production database(or even a testing database!) so it should run fast enough and fit within the memory limits of whatever machine you use for testing.

Anton Sizikov • Dec 28 '17

When SQLite db is a great choice for in-memory test database (I have about 7k tests like that in my current project, love it so much, but that's a different story), replacing the mail service is not always a good solution. When you test your code against the service which is very different from the one you use in production, what do you really test?

That's the same trade-off: do we verify an integration with the test service, or with the mock? If the test service is close enough to the system you have in prod, it is fast enough, then I would say go for it.

Franco Traversaro • Dec 28 '17

Do you REALLY have written a unit test that send mail and assertPopServerHasMail()? :-/

Jilles van Gurp • Dec 29 '17

In my view overly complicated test code (with or without mocks) is a design smell of the underlying system not being very testable. There are a few common patterns here but it always boils down to violations of the SOLID principles. If you need a lot of mocks to test a method, maybe it has too many side effects? Maybe the class has poor cohesiveness and tight coupling? Fix the underlying problems by refactoring until the test becomes the one or two liner it should have been.

In the case of the email service, you'll almost certainly want to add some abstractions in between where you are sending stuff and how you are sending it. What you want to test here (in a unit test) is that "an email got sent".

Another problem is mixing unit and integration tests. Mocks are best used for unit tests exclusively. Integration tests are slow anyway, so you don't save a lot of time by mocking things and you are reducing their limited coverage by making them less realistic. Make the most of your integration tests by maximizing coverage and realism so you find all those issues that happen in production systems before you ship. Use unit test for spotting logic bugs.

Think realistic user scenarios when doing integration tests. Coming back to the email service, you'll want to test that e.g. a user signs up via some API, clicks a link received via the email that was sent, and then successfully activates. That's a scenario and it will only work if everything lines up perfectly. That's why it is called an integration test. Now if you fake everything and grab the activation code by poking around in the database or from some mock that was technically a waste of time. That will never happen in production and the obvious failure scenarios usually revolve around email issues.

A good unit test requires mocks because fundamentally you should not even want to test stuff outside the scope of the unit under test. Only if the only point of the unit is the side effect with a dependency should you begin to consider using stuff like parameter inspection. Integration tests are the opposite: you are testing system behavior. If you change the system, what are you testing really?

If writing good tests is hard, that's your real problem: write testable code and life gets a lot easier when writing tests.

Anton Sizikov • Dec 29 '17 • Edited

Thanks for the comment.

Mocks and SOLID are not synonyms. Not using mocks in tests does not mean the system is badly designed. Lots of mocks is not a symptom of a good design either.

If you were able to replace the dependency with the mock it does not mean that the test became a good one. It just became a test which verifies the interaction with the mock. If your mock behavior does not affect the class logic/results why do you even need this dependency?

Imagine that you have a class with no dependencies. It does some calculations. You have a test which verifies that. I believe that's called a unit test.

Next day you decided to extract the actual calculation into a separate class. You know, like a one-shortcut action in your IDE. And boom, your test is now a bad and slow integration test. Now you have to mock that dependency. It's ok, you will write a new unit test for the new class, rewrite all the existing tests for the old one.

The day after you realized that that wasn't a right decision. Will that be a simple task to bring everything back? (yes, you can revert the commit, but that's not always a good option).

My point is that the excessive use of mocks made your system very resistant to refactoring. It's just way to expensive to constantly improve and modify your code. And that leads to a bad design. And that makes unit tests with mocks very expensive.

Bohdan Stupak • Dec 29 '17 • Edited

Thank you for great read which I mostly agree. But I would argue even further. In this pretty straightforward example, you might discard the notion of IGreeter at all and interact directly with the concrete implementation. IGreeter itself looks like a case of test induced damage. Also, you might want to check out this question and specifically answer of Mark Seeman who mentions using coarse-grained dependency injection.
Regarding the subject itself for sure we're all aware that unit-test are quite fragile but on the contrary, they're fast and allow us to test business logic without relying on some volatile context as DBs, SMTP servers or etc. But for sure integration test allow us to test all moving parts. That's why gods of enterprise (he-he :D) gave us test pyramid

Anton Sizikov • Dec 29 '17

Thanks for the links.

I kind of see the problem with that pyramid. We've been told so many times that integration tests are slow, that we take that for granted.

The Greeter example is made up, but it shows that such test with the production dependency it's at least not slower than the test with mocked dependency. Mocking frameworks do quite some heavy lifting behind the scenes. And the Greeter implementation is dumb and simple. Much simpler than the mocked version.

There is also, a terminology confusion which I encounter very often. When there is a mock involved it's called a unit test, but when there is a runtime dependency injected it's an integration test. I'm trying to question that. I see them both as integration tests. The first one tests the integration with the mocked dependency, another one tests the integration with the actual implementation. When the non-mocked version performs in a similar way (or better), why would we spend time and resources on building and maintaining all the mocks?

Not every dependency makes HTTP calls.
Obviously, when the class depends on a heavy IO operation, you have to isolate it in order to improve the performance of your test suite. At least that's the part we all agree on :)

bschatz • Oct 25 '19

There is nothing wrong in your example.

With the test, you make sure that the greeter is used by the client, no more no less. (Unit tests can't replace integration tests).
But to emphasize this in the test, I usually introduce a constant in the test that doesn't output a text that a real implementation would do, e.g.:


public class ClientTests 
{
    pubic void Test()
    {
        const string TEXT_FROM_GREETER="Some nice formatted text from Greeter");

        var mock = new Mock<IGreeter>();
        mock.Setup(greeter => greeter.Greet("John", "Mr.")).Returns(TEXT_FROM_GREETER);
        var result = new Client(mock.Object).FormatPageHeader("John", "Mr.");
        Assert.AreEquals(result, TEXT_FROM_GREETER); 
    } 
}

Iyed Bennour • May 10 '19 • Edited

I think the mock example you gave for the IGreeter interface is quite misleading as it's not what I consider a mock: it's a full fledged implementation of the interface implemented with help of a mock. Mocks are useful to test how the - System Under Test - behaves WITH RESPECT TO an interface ,NOT an implementation of it. In other words, how your SUT reacts to the behavior of its dependencies. Mocks help you simulate (or mock) your dependencies at the interface level (read as API level): returned values/errors or exception thrown irrespective of the input. The problem(s) in the example you gave is not the use of the mock, it's the code itself that could be easily fixed to be more testable.