To mock, or not to mock, that is the question

TLTR

Mocking is often seen as a default choice when writing tests. Yet it might introduce unnecessary complexety to your system. There are other approaches to manage dependencies in tests.

More

What is mocking?

Mocking — creating objects that simulate the behaviour of real objects.

Here is how mocking looks in C# in a nutshell (JustMock library):

// Instantiate a new mock
var mockContactRepository = Mock.Create<IContactRepository>();

// Set up the mock and it's return value 
Mock.Arrange(() => mockContactRepository.GetContacts())
  .Returns(new List<Contact>
  {
      new Contact { ContactId = 1 }, new Contact { ContactId = 2 }
  });

// Pass mock as a dependency
var contactManager = new ContactManager(mockContactRepository);

Although it sounds very useful it has to be taken with a pinch of salt.

Shortcomings of mocking

1. Runtime instead of compile-time feedback on changes

If we imagine a test that has Setup() but doesn’t have a return value. When we add a return value, mock doesn’t suggest to add a return type. We only find out about it when running the test.
Here is the example of this shortcoming in C# (Moq library)

public class CarsControllerTests
{

    [Fact]
    public async void testCreateCar()
    {
        var repositoryMock = new Mock<ICarRepository>();

        // No return value set
        repositoryMock.Setup(r => r.Create(car));

        var carsController = new CarsController(repositoryMock.Object);
        var car = new Car() { Name = "BMW", Available = true };
        var result = await controller.Create(car);

        // Use return value on result
        Assert.Equal("BMW", result.Name);
    }
}

The test has built sucessfully but the test will not pass. The reason is no return value set. The controller relies on returned value from the repository.

>> dotnet test

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:01.0309627]     Cars.Tests.CarsControllerTests.testCreateCar [FAIL]
  Failed Cars.Tests.CarsControllerTests.testCreateCar [94 ms]
  Error Message:
   System.NullReferenceException : Object reference not set to an instance of an object.
  Stack Trace:
     at Cars.CarsController.Create(Car car) in /Users/kondrashov/Projects/unit-testing-mocking/src/Cars/Controllers/CarController.cs:line 20
   at Cars.Tests.CarsControllerTests.testCreateCar() in /Users/kondrashov/Projects/unit-testing-mocking/test/Cars.Tests/CarsControllerTests.cs:line 20
   at System.Threading.Tasks.Task.<>c.<ThrowAsync>b__128_0(Object state)
Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: < 1 ms - Cars.Tests.dll (net7.0)

We get a null reference exception at return car.Id.ToString() To make the test happy we need to use ReturnsAsync() method on our mock:

// Set return value
repositoryMock.Setup(r => r.Create(car))
    .ReturnsAsync(new Car { Id = "1", Name = "BMW", Available = true });

It seems easy and straightforward to change the mock above. But more compex methods become less trivial. The value that this test delivers becomes less with all the time that we spent for it’s maintance.

We would want to know when something broke at the compile-time, instead of runtime.

2. Tight coupling to contracts

Mocking is coupled with contracts by nature. It makes any refactoring harder, because you have to change all realted mocks.

Below I have about 6 different services in my mocking. Refactoring contracts of below services will result in breaking all these mocks. Multiply the breakages by number of test files where you set up mocks.

3. Short System Under Test

System Under Test (SUT) is shorter when we mock a dependency:

A good alternative to such test is an Integration Test with longer SUT:

4. Long-winded set up in each test

A testing framework usually allows you to group your set up. However it’s not always the case as each test often requires a dedicated set up. Below is an example of how each test requires a code to set up mocking:

[TestClass]
public class MyTestClass
{
    private Mock<IMyInterface> _mock;

    [TestInitialize]
    public void Setup()
    {
        _mock = new Mock<IMyInterface>();
    }

    [TestMethod]
    public void Test1()
    {
        _mock.Setup(m => m.MyMethod()).Returns("Hello");
        var result = _mock.Object.MyMethod();
        Assert.AreEqual("Hello", result);
    }

    [TestMethod]
    public void Test2()
    {
        _mock.Setup(m => m.MyMethod()).Returns("World");
        _mock.Setup(m => m.AnotherMethod()).Returns(42);
        var result1 = _mock.Object.MyMethod();
        var result2 = _mock.Object.AnotherMethod();
        Assert.AreEqual("World", result1);
        Assert.AreEqual(42, result2);
    }

    [TestMethod]
    public void Test3()
    {
        _mock.Setup(m => m.MyMethod()).Returns("Goodbye");
        var result = _mock.Object.MyMethod();
        Assert.AreEqual("Goodbye", result);
    }
}

5. You can’t mock just any method

We mock methods. And they have to be public to be eligible for mocking.
Yet certain frameworks allow to you mock private methods using reflection. This is wrong as it break the incapsulation of your design. Example below in Mockito in Java:

when(spy, method(CodeWaithPrivateMethod.class, "doTheGamble", String.class, int.class))
    .withArguments(anyString(), anyInt())
    .thenReturn(true);

What can I do instead of mocking?

1. Integration tests

Write an Integration Test, not a Unit Test. There is more value in writing an integration test instead of a unit test with mocks. We cover more code by writing less tests.

2. End-to-end tests

Integration test might rely on an external dependency. In this case you can’t rely on that dependency since you don’t control it. Write an end-to-end test where you would hit your system as if you were a customer of this system.

3. Stubs

A stub is a small piece of code that takes the place of another component during testing. The benefit of using a stub is that it returns consistent results, making the test easier to write.
If you can’t write an end-to-end test due to dependecies you can’t control — use Stubs. It simialar to mocking, yet it provides a type checking when you introduce changes to your system. Below is an example of a simple stub for a car repository:

public class FakeCarRepository : ICarRepository
{
    public async Task<Car> Create(Car car)
    {
        // Any logic to accomodate for creating a car
        return new Car();
    }

    public async Task<Car> Get(string id)
    {
        // Any logic to accomodate for getting a car
        return new Car();
    }
}

Another advantage the test becomes cleaner. All setup in extracted into a separate file:

[Fact]
public async void testCreateCar()
{
   var car = new Car() { Name = "BMW", Available = true };

   var controller = new CarsController(new FakeCarRepository());
   var createdCar = await controller.Create(car);

   Assert.NotNull(createdCar);
}

Summary

The best approach depends on the specific needs of the project. Consider all trade-offs between mocking and non-mocking. There are other alternatives to mocking out there which you might enjoy.

Resources

1. Advantages of Integration test over a Unit test
2. Using TestContainers library when writing an Integration test