Mocks in Elixir

Or “Test driving a Phoenix endpoint, part II”

This story was originally posted by me on Medium on 2016-10-32, but moved here since I'm closing my Medium account.

As the subtitle suggests, this is the second part of a blog post series about Phoenix and TDD. However, this post is not Phoenix-specific and the patterns presented here can be used in any Elixir application.

The first part was about separation of concerns and how to make clean and reusable code within Phoenix. It can be found here.

This part is about testing modules that somehow makes external calls that we would like to avoid while testing.

Mocking external calls

Since I started with Elixir, I have wondered how to mock stuff in tests. And when to mock stuff.

I come from Ruby, and I have developed a habit of injecting doubles for all dependencies of a class when unit testing the class.

Something like this:

class InviteUser
  def initialize(mailer: Mailer.new, user_repo: UserRepo.new)
    @mailer = mailer
    @user_repo = user_repo
  end

  attr_reader :mailer
  attr_reader :user_repo

  def invite(email)
    user = user_repo.create(email)
    mailer.send_activation_mail(user)
    user
  end
end

RSpec.describe InviteUser do
  describe "#invite" do
    it "sends an activation email" do
      mailer = instance_double(Mailer, create: User.new)
      user_repo = instance_double(UserRepo, send_activation_mail: true)

      invite_user = InviteUser.new(mailer: mailer, user_repo: user_repo)

      invite_user.invite("me@example.com")

      expect(mailer).to have_received(:send_activation_mail)
    end
  end
end

This is also possible in Elixir. One could just inject a module into a function, but it does not seem to be the preferred way in the Elixir community.

I needed an answer to two questions:

1. When do I use a mock?
2. How do I mock?

There seems to be a very unambiguous trend in the community to the first question: Only use mocks when you would otherwise hit a third party service. The database does not count as a third party service here, since it is clearly possible to use it in tests.

Since we’re building something that can send out emails (see Part I), we would surely like to somehow mock that out. That brings us to the second question: How do we mock?

Different mocking patterns

I have explored the possibilities of mocking in Elixir and boiled it down to four different techniques, which I will briefly go through here.

Spoiler: The last one is my preferred.

Injection with pure Elixir

As explained with the Ruby example above, we could simply inject a module to a function and have a default value for that:

defmodule InviteUser do
  def invite(email, mailer \\ Mailer) do
    user = create(email)
    mailer.send_activation_mail(user)
  end
end

This would work and does not require any hex libs. But it does not feel very Elixirish. One of the things I love about Elixir is the stuff that happens at compile time. This runtime injection takes away that joy.

I also like that everything is very clear in Elixir. You can look at a function and immediately know what it is doing. With runtime injection, anything can happen. We don’t know what kind of mailer is send to the function.

I like this method of mocking in Ruby, but I don’t like it for Elixir.

Injection with a library

I have tried a hex package called pact that offers dependency injection. It is basically the same as the injection with pure Elixir, except it uses a process to keep a store of modules and some nice convenience functions to create mocks of those modules from a test.

But it is still runtime injection, which I don’t think is a good match in Elixir. An example with pact would look something like this:

defmodule InviteUser do
  def invite(email) do
    user = create(email)
    MyApp.Pact.get("mailer").send_activation_mail(user)
  end
end

And you would then be able to mock out the content of MyApp.Pact.get("mailer") at runtime in tests.

This seems like an even more poor solution than the injection with pure Elixir, since it requires a process to be running and more confusion when trying to read what a function actually does.

Mocking with mock

mock is an Elixir library that uses the meck Erlang library for mocking. I have not tried to use it, so what I’m saying here might be a little wrong. Please correct me.

It seems that mock does runtime substitution of module functionality with some dark magic meta programming. It reminds me a lot of the mocking functionality in RSpec from Ruby.

mock allows you to e.g. tell String.reverse/1 to accept a number and return the number times two(?). Here is an example from the readme:

efmodule MyTest do
  use ExUnit.Case, async: false

  import Mock

  test "multiple mocks" do
    with_mocks([
      {HashDict,
       [],
       [get: fn(%{}, "http://example.com") -> "<html></html>" end]},
      {String,
       [],
       [reverse: fn(x) -> 2*x end,
        length: fn(_x) -> :ok end]}
    ]) do
      assert HashDict.get(%{}, "http://example.com") == "<html></html>"
      assert String.reverse(3) == 6
      assert String.length(3) == :ok
    end
  end
end

This is the kind of mocking that José Valim would describe as “mock (verb), not mock (noun)”.

If, for example, we would like to test an API client, we could mock (verb) the underlaying HTTP library to return the expected responses. In my humble opinion, this is just wrong and takes away all the functional part of Elixir.

A better way, José says, is to create a mock (noun) and at compile time choose to use either the real implementation or the mock implementation.

This leads us to the fourth and my preferred way to test external calls in Elixir.

Compile time settings

This section is basically a summary of José’s post.

We define two modules: One which holds the real implementation and one that holds the mock implementation. In the config we define that the test environment uses the mock implementation and all other environments use the real implementation.

We then define a behaviour to make sure both implementations follow the same interface.

That’s it. It’s simple, it’s compile time and it’s pure Elixir.

Why is it good that it is happening at compile time, you might ask. It’s good because the compiled production code will be essentially the same as if we didn’t use a mock in tests.

In the rest of this blog post we will create a mailer mock, TDD style.

Creating the mailer

Before we create a mock, we create a working mailer, which we test in the dev environment via iex. We write no tests for this implementation.

I followed the Phoenix documentation of how to send emails and came up with this mailer:

defmodule MyApp.Mailer do
  use Mailgun.Client, [
    domain: Application.fetch_env!(:my_app, :mailgun_domain),
    key: Application.fetch_env!(:my_app, :mailgun_key)
  ]

  @from "support@myapp.com"

  def send_test_mail(email) do
    {:ok, _} = send_email [
      to: email,
      from: @from,
      subject: "Test!",
      text: "Test from MyApp"
    ]
    :ok
  end
end

We can easily one-time test this from iex -S mix and verify that we indeed get an email in our real mailbox.

Test driving the mailer mock

We are going to keep the mail composing functionaility in MyApp.Mailer and move the Mailgun implementation to MyApp.Mailer.Mailgun. Likewise we will create the mock in MyApp.Mailer.Mock.

But first, we write a test:

defmodule MyApp.MailerTest do
  use ExUnit.Case

  test "it uses mock in tests" do
    MyApp.Mailer.Mock.clear
    MyApp.Mailer.send_test_mail("me@example.com")

    assert MyApp.Mailer.Mock.mails |> length == 1

    [mail] = MyApp.Mailer.Mock.mails

    assert mail.to == "me@example.com"
    assert mail.text == "Test from MyApp"
  end
end

This is mostly a test that the mock works as expected, but it also tests the small amount of mail composing functionality in the send_test_mail function.

We run the test

$ mix test
Compiling 16 files (.ex)

== Compilation error on file lib/my_app/mailer.ex ==
** (ArgumentError) application :my_app is not loaded, or the configuration parameter :mailgun_domain is not set
    (elixir) lib/application.ex:261: Application.fetch_env!/2
    lib/my_app/mailer.ex:3: (module)
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6

And we get a compilation warning? How? The mailer worked fine in development. We would expect the test to fail, since we don’t have a mock yet, not because it can’t compile.

This is because the current Mailer implementation uses Mailgun and fetches some settings from the config at compile time. We haven’t defined the mailgun settings for the test environment, which is why we get a compilation error.

To fix this, we need to refactor the Mailer. Instead of having a Mailgun implementation, it should just delegate the mail sending part (not the mail composing part) to another module specified in the config.

We should then write the Mailgun implementation and the Mock implementaion.

First, let’s rewrite the Mailer, and save the Mailgun parts for later:

defmodule MyApp.Mailer do
  @from "support@myapp.com"
  @mailer_impl Application.fetch_env!(:my_app, :mailer)

  defmodule Behaviour do
    @callback send_mail([key: String.t]) :: :ok
  end

  def send_test_mail(email) do
    :ok = @mailer_impl.send_mail(
      to: email,
      from: @from,
      subject: "Test!",
      text: "Test from MyApp"
    )
  end
end

We keep the composing logic and delegate the sending to a @mailer_impl which we fetch from the config. We also define a behaviour which can be implemented by the different mailer implementations.

Lets write the Mailgun and Mock implementations

defmodule MyApp.Mailer.Mailgun do
  @behaviour MyApp.Mailer.Behaviour

  use Mailgun.Client, [
    domain: Application.fetch_env!(:my_app, :mailgun_domain),
    key: Application.fetch_env!(:my_app, :mailgun_key)
  ]

  def send_mail(mail) do
    {:ok, _} = send_email(mail)
    :ok
  end
end

defmodule MyApp.Mailer.Mock do
  @behaviour MyApp.Mailer.Behaviour

  def start_link do
    Agent.start_link(fn -> [] end, name: __MODULE__)
  end

  def mails do
    Agent.get(__MODULE__, &(&1))
  end

  def clear do
    Agent.update(__MODULE__, fn _mails -> [] end)
  end

  def send_mail(mail) do
    mail = mail |> Enum.into(%{})
    Agent.update(__MODULE__, fn existing_mails -> [mail | existing_mails] end)
    :ok
  end
end

Each implementation just have to implement the send_mail function as defined by the behaviour. The Mailgun mailer is mostly the same as our original Mailer.

The Mock is implemented with an Agent to keep track of which emails are sent.

Before running the tests again, we define the mailer implementation to use in the config.

In config/test.exs we add this:

config :my_app, mailer: MyApp.Mailer.Mock

And in config/config.exs we add this:

config :my_app, mailer: MyApp.Mailer.Mailgun

This sets the Mailgun mailer as the default and overwrites the setting in tests.

Time to run the test again to see it pass:

$ mix test
Compiling 18 files (.ex)

== Compilation error on file lib/my_app/mailer/mailgun.ex ==
** (ArgumentError) application :my_app is not loaded, or the configuration parameter :mailgun_domain is not set
    (elixir) lib/application.ex:261: Application.fetch_env!/2
    lib/my_app/mailer/mailgun.ex:5: (module)
    (stdlib) erl_eval.erl:670: :erl_eval.do_apply/6

What? We still get the same compilation error about missing mailgun settings!

This is because even though we don’t use the Mailgun mailer in tests, it is still being compiled. We should just add some dummy settings in the test config:

config :my_app, [
  mailgun_domain: "notusedintests",
  mailgun_key: "notusedintests"
]

There! Lets run the tests again. Now it should be all green.

$ mix test
.......

  1) test it uses mock in tests (MyApp.MailerTest)
     test/mailer_test.exs:4
     ** (exit) exited in: GenServer.call(MyApp.Mailer.Mock, {:update, #Function<0.6592501/1 in MyApp.Mailer.Mock.clear/0>}, 5000)
         ** (EXIT) no process
     stacktrace:
       (elixir) lib/gen_server.ex:596: GenServer.call/3
       test/mailer_test.exs:5: (test)



Finished in 0.1 seconds
8 tests, 1 failure

Randomized with seed 356689

Ok, now it compiles at least. Looking at the error message, we see that the Mock agent is not started. This is (doh) because we did not start it. We should add this to our test_helper.exs:

MyApp.Mailer.Mock.start_link

Now it should be green!

$ mix test
........

Finished in 0.09 seconds
8 tests, 0 failures

Randomized with seed 776617

Yay! All green. We have succesfully created a mock (noun) for our mailer.

Afterword

It took me quite some time to figure out when and how to use mocks in Elixir. I think the approach described here is really nice and the most Elixirish of the patterns I have seen.

• It is simple
• It is pure Elixir. No libs needed.
• It is unobtrusive compared to the injection patterns
• It treats the mock as any other implementation, which allows for easy substitution of the real implementation.
• One could even have a one implementation for production, one for staging, one for dev and one for test.

I hope somebody saves some time by reading this blog post :)