Chapter 16 Mox rocks
In the last chapter, we built an end-to-end test that broadcasts trade events and verifies orders land in the database. It works - but it’s slow, brittle, and when it fails, we’re left guessing where the problem is.
Here’s the thing about end-to-end tests: they’re great for proving the system works as a whole, but terrible for developer productivity. Every test run requires database setup, process orchestration, and hardcoded sleeps. And when something breaks, good luck figuring out which module caused it.
What we need is fast, focused unit tests. We’ll use the mox library to mock
the dependencies of our Naive.Trader - PubSub, Binance, the Leader, even Logger.
Each mock will pattern match on expected values, giving us assertions for free.
The result? Tests that run in milliseconds, pinpoint failures exactly, and
don’t need a database in sight.
Along the way, we’ll define behaviours for modules that don’t have them,
wire up environment-based configuration, and see why mox’s approach to
mocking is fundamentally better than hand-rolled stubs.
16.1 Objectives
- introduction to mock-based tests
- add the Mox package
- identify the
Naive.Traderdependencies- mock the
Binancemodule - mock the
NaiveLeadermodule - mock the
Phoenix.PubSubmodule - mock the
Loggermodule
- mock the
- implement a test of the
Naive.Tradermodule - define an alias to run unit tests
16.2 Introduction to mock-based tests
Let’s revisit the specific downsides of our end-to-end test from the last chapter:
- we will be unable to run more than one end-to-end test in parallel as they rely on the database’s state
- we need to set up the database before every test run
- we need to start processes in the correct order with the suitable parameters
- we need to wait a (guessed) hardcoded amount of time that it will take to finish the trading(this is extremely bad as it will cause randomly failing tests as people will make the time shorter to speed up tests)
- we wouldn’t be able to quickly pinpoint which part the error originated from as the test spans over a vast amount of the system
- logging was polluting our test output
How could we fix the above issues?
The most common way is to limit the scope of the test.
Instead of testing the whole trading flow, we could focus on testing a single Naive.Trader process.
Focusing on a single trader process would remove the requirement for starting multiple processes before testing, but it would also bring its own challenges.
Let’s look at a concrete example:
When the Naive.Trader process starts, it subscribes to the TRADE_EVENTS:#{symbol} PubSub topic.
It also broadcasts updates of the orders it placed to the ORDERS:#{symbol} PubSub topic.
How could we break the link between the Naive.Trader and the PubSub(or any other module it depends on)?
We could utilize the trick that we used for the Binance module. We could create a module that provides the same functions as the PubSub module.
We know that the trader process calls Phoenix.PubSub.subscribe/2 and Phoenix.PubSub.broadcast/3 functions.
We could implement a module that contains the same functions:
The above module would satisfy the PubSub’s functionality required by the Naive.Trader module, but this solution comes with a couple of drawbacks:
- it doesn’t test the passed values. It just ignores them, which is a missed opportunity to confirm that the PubSub module was called with the expected values
- we can’t define a custom implementation specific to a test. This is possible by extending the implementation with test-related returns(hack!)
Using the mox module would fix both of the problems mentioned above. With the mox module we can define ad-hoc function implementation per test:
# inside test file
test ...
Test.PubSubMock
|> expect(:subscribe, fn (_module, "TRADE_EVENTS:XRPUSDT") -> :ok end)
|> expect(:broadcast, fn (_module, "ORDERS:XRPUSDT", _order) -> :ok end)There are multiple benefits to using the mox module instead of handcrafting the implementation:
- it allows defining functions that will pattern match values specific to each test(as in the case of the “usual” pattern matching, they will break when called with unexpected values)
- it allows defining implementations of the mocked functions based on incoming(test-specific) values
- it can validate that all defined mocked functions have been called by the test
- it comes with its own tests, so we don’t need to test the mocking framework itself - unlike a custom handcrafted implementation that we’d have to maintain and verify
But there’s a catch ;)
For the mox to know what sort of functions the module provides, it needs to know its behaviour.
In Elixir, to define a behaviour of the module, we need to add the @callback attributes to it:
defmodule Core.Test.PubSub do
@type t :: atom
@type topic :: binary
@type message :: term
@callback subscribe(t, topic) :: :ok | {:error, term}
@callback broadcast(t, topic, message) :: :ok | {:error, term}
endA behaviour can be defined in a separate module if we are working with a 3rd party module
that doesn’t provide it(like in the case of the Phoenix.PubSub module).
Note: The additional benefit of using the behaviours is that we could tell Elixir that our module implements the behaviour
by adding the @behaviour attribute:
Using the above will cause Elixir to validate at compile time that the MyPubSub module
implements all functions defined inside the Core.Test.PubSub module(otherwise it will raise a compilation error).
Let’s get back to the main topic. We figured out that we could mock all of the modules that the Naive.Trader depends on using the mox module.
But, how would we tell the Naive.Trader to use the mocked modules instead of the “real” ones when we run tests?
We could make all modules that the Naive.Trader depends on be dynamically injected from the configuration(based on the environment).
Another requirement to make mox work is to define the mocks upfront using the Mox.defmock/2 function
(called in test_helper.exs, which runs before tests).
It will dynamically define a new module that will be limited by the passed behaviour -
we will only be able to mock functions defined as part of that behaviour inside our tests.
To sum up, there are a few steps to get the mox running:
- implement behaviours that we would like to mock(as most packages [like
Phoenix.PubSub] don’t include them) - define mock modules using the
Mox.defmockfunction - modify the application’s configuration to use the mocked module(s)
- specify mocked module’s expectation inside the test
That’s the full mental model. Now let’s see how each step plays out in practice.
16.3 Add the mox package
First let’s add the mox package to the naive application’s dependencies:
We can now run mix deps.get to fetch the mox package.
[Note] As we will add the mox’s mocking code into the test_helper.exs file, we need to
make mox available in all test environments(both test and integration).
16.4 Identify the Naive.Trader dependencies
Let’s investigate the Naive.Trader module(/apps/naive/lib/naive/trader.ex). We are looking for all calls to other modules - we can see:
Logger.info/1Phoenix.PubSub.subscribe/2@binance_client.order_limit_buy/4Naive.Leader.notify/2@binance_client.get_order/3@binance_client.order_limit_sell/4Phoenix.PubSub.broadcast/3
So the Naive.Trader relies on four modules:
LoggerPhoenix.PubSubNaive.Leader@binance_client(eitherBinanceorBinanceMock)
We will need to work through them one by one.
16.4.1 Mock the Binance module
Let’s start with the binance client, as it’s already a dynamic value based on the configuration.
Neither the Binance nor the BinanceMock(our dummy implementation) module provides a behaviour -
let’s fix that by defining the @callback attributes at the top of the BinanceMock module before the structs:
# /apps/binance_mock/lib/binance_mock.ex
...
alias Binance.Order
alias Binance.OrderResponse
alias Core.Struct.TradeEvent
@type symbol :: binary
@type quantity :: binary
@type price :: binary
@type time_in_force :: binary
@type timestamp :: non_neg_integer
@type order_id :: non_neg_integer
@type orig_client_order_id :: binary
@type recv_window :: binary
@callback order_limit_buy(
symbol,
quantity,
price,
time_in_force
) :: {:ok, %OrderResponse{}} | {:error, term}
@callback order_limit_sell(
symbol,
quantity,
price,
time_in_force
) :: {:ok, %OrderResponse{}} | {:error, term}
@callback get_order(
symbol,
timestamp,
order_id,
orig_client_order_id | nil,
recv_window | nil
) :: {:ok, %Order{}} | {:error, term}In the above code, we added three @callback attributes that define the binance client behaviour.
For clarity, we defined a distinct type for each of the arguments.
As we now have a binance client behaviour defined,
we can use it to define a mock using the Mox.defmock/2 function inside the test_helper.exs file of the naive application:
# /apps/naive/test/test_helper.exs
ExUnit.start()
Application.ensure_all_started(:mox) #1
Mox.defmock(Test.BinanceMock, for: BinanceMock) #2First(#1), we need to ensure that the mox application has been started. Then(#2),
we can tell the mox package to define the Test.BinanceMock module based on the BinanceMock behaviour.
As we defined the binance client behaviour and mock, we can update our configuration to use them.
We want to keep using the BinanceMock module in the development environment, but for the test environment,
we would like to set the mocked module generated by the mox package:
16.4.2 Mock the NaiveLeader module
We can now move back to the Naive.Trader module to update all the hardcoded references to the Naive.Leader module
with a dynamic attribute called @leader and add this attribute at the top of the module:
# /apps/naive/lib/trader.ex
...
@leader Application.compile_env(:naive, :leader)
...
@leader.notify(:trader_state_updated, new_state)
...
@leader.notify(:trader_state_updated, new_state)
...
@leader.notify(:rebuy_triggered, new_state)
...As with the BinanceMock(our dummy implementation) module, the Naive.Leader module doesn’t provide a behaviour -
let’s fix that by defining the @callback attributes at the top of the module:
# /apps/naive/lib/leader.ex
...
@type event_type :: atom
@callback notify(event_type, %Trader.State{}) :: :okIn the above code, we added a single @callback attribute that defines the naive leader behaviour.
For clarity, we defined a distinct type for the event_type argument.
As we now have a naive leader behaviour defined, we can use it to define a mock using the Mox.defmock/2 function
inside the test_helper.exs file of the naive application:
In the above code, we’ve told the mox package to define the Test.Naive.LeaderMock module based on the Naive.Leader behaviour.
We are moving on to the configuration.
As the Naive.Leader wasn’t part of the configuration, we need to add it to the default config and test config files.
First, let’s add the :leader key inside the config :naive in the default /config/config.exs configuration file:
# /config/config.exs
...
config :naive,
binance_client: BinanceMock,
leader: Naive.Leader, # <= added
...and then we need to apply the same update to the /config/test.exs configuration file
(it will point to the module generated by the mox package - Test.Naive.LeaderMock):
16.4.3 Mock the Phoenix.PubSub module
Mocking the Phoenix.PubSub dependency inside the Naive.Trader module will look very similar to the last two mocked modules.
Inside the Naive.Trader module we need to update all the references to the Phoenix.PubSub to an @pubsub_client attribute
with value dependent on the configuration:
# /apps/naive/lib/trader.ex
...
@pubsub_client Application.compile_env(:core, :pubsub_client)
...
@pubsub_client.subscribe(
Core.PubSub,
"TRADE_EVENTS:#{symbol}"
)
...
@pubsub_client.broadcast(
Core.PubSub,
"ORDERS:#{order.symbol}",
order
)
...The Phoenix.PubSub module doesn’t provide a behaviour. As we can’t modify its source,
we need to create a new module to define the PubSub behaviour.
This is the same Core.Test.PubSub module we used as an example earlier - now we’ll actually create it.
Let’s add a new file called test.ex inside the /apps/core/lib/core directory with the following behaviour definition:
# /apps/core/lib/core/test.ex
defmodule Core.Test do
defmodule PubSub do
@type t :: atom
@type topic :: binary
@type message :: term
@callback subscribe(t, topic) :: :ok | {:error, term}
@callback broadcast(t, topic, message) :: :ok | {:error, term}
end
endAs previously, we defined a couple of callbacks and additional types for each of their arguments.
Next, we will use the above behaviour to define a mock using the Mox.defmock/2 function inside the test_helper.exs file of the naive application:
In the above code, we’ve told the mox package to define the Test.PubSubMock module based on the Core.Test.PubSub behaviour.
The final step will be to append the :core, :pubsub_client configuration to the /config/config.exs file:
and the test /config/test.exs configuration file:
16.4.4 Mock the Logger module
Before we dive in, we should ask ourselves: why mock the Logger module?
We could raise the logging level to error and be done with it.
Yes, that would fix all debug/info/warning logs,
but we would also miss an opportunity to confirm a few details (depends on what’s necessary for our use case):
- you can ensure that the log was called when the tested function was run
- you can pattern match the logging level
- you can check the message. This could be useful if you don’t want to put sensitive information like banking details etc. inside log messages
Mocking the Logger dependency inside the Naive.Trader module will follow the same steps as the previous updates.
Inside the Naive.Trader module we need to update all the references to the Logger to an @logger attribute with value dependent on the configuration:
# /apps/naive/lib/trader.ex
...
@logger Application.compile_env(:core, :logger)
...
@logger.info("Initializing new trader(#{id}) for #{symbol}")
...
@logger.info(
"The trader(#{id}) is placing a BUY order " <>
"for #{symbol} @ #{price}, quantity: #{quantity}"
)
...
@logger.info(
"The trader(#{id}) is placing a SELL order for " <>
"#{symbol} @ #{sell_price}, quantity: #{quantity}."
)
...
@logger.info("Trader(#{id}) finished trade cycle for #{symbol}")
...
@logger.info("Rebuy triggered for #{symbol} by the trader(#{id})")
...The Logger module doesn’t provide a behaviour. As we can’t modify its source,
we need to create a new module to define the Logger behaviour.
Let’s place it inside the Core.Test namespace in the /apps/core/lib/core/test.ex file
side by side with the PubSub behaviour with the following definition:
# /apps/core/lib/core/test.ex
defmodule Core.Test do
...
defmodule Logger do
@type message :: binary
@callback info(message) :: :ok
end
endAs previously, we defined a callback and additional type for the message argument.
Next, we will use the above behaviour to define a mock using the Mox.defmock/2 function inside the test_helper.exs file of the naive application:
In the above code, we’ve told the mox package to define the Test.LoggerMock module based on the Core.Test.Logger behaviour.
The final step will be to append the :core, :logger configuration to the /config/config.exs file:
and the test /config/test.exs configuration file:
This finishes mocking all four dependencies.
Every external module the Naive.Trader relies on is now based on configuration values.
We can progress to writing the test.
16.5 Implement a test of the Naive.Trader module
Finally, we can implement the unit test for the Naive.Trader module.
We will create a folder called naive inside the /apps/naive/test directory and a new file called trader_test.exs inside it.
Let’s start with an empty skeleton of the test tagged as unit:
# /apps/naive/test/naive/trader_test.exs
defmodule Naive.TraderTest do
use ExUnit.Case
doctest Naive.Trader
@tag :unit
test "Placing buy order test" do
end
endLet’s add the mox related code above the @tag :unit line:
# /apps/naive/test/naive/trader_test.exs
import Mox # <= 1
setup :set_mox_from_context # <= 2
setup :verify_on_exit! # <= 3In the above code, we are:
- importing the
moxmodule so we will be able to use functions likeexpect/3 - we use
set_mox_from_contextbecause our test is not running inasync: true mode, Mox allows us to share expectations globally. This is crucial because theNaive.Traderruns in a separate process from the test. If we ran this test asynchronously, the Trader process would crash because it wouldn’t have permission to call the mocks defined by the test process(unless we explicitly granted allowances). - telling
moxto verify that all the mocks defined in the tests have been called from within those tests. Otherwise, it will flag such cases as test errors
Inside our test, we need to define implementations for all the functions that the Naive.Trader relies on:
# /apps/naive/test/naive/trader_test.exs
...
test "Placing buy order test" do
Test.PubSubMock
|> expect(:subscribe, fn _module, "TRADE_EVENTS:XRPUSDT" -> :ok end) # <= 1
|> expect(:broadcast, fn _module, "ORDERS:XRPUSDT", _order -> :ok end)
Test.BinanceMock
|> expect(:order_limit_buy, fn "XRPUSDT", "464.360", 0.4307, "GTC" -> # <= 2
{:ok,
BinanceMock.generate_fake_order(
"XRPUSDT",
"464.360",
0.4307,
"BUY"
)
|> BinanceMock.convert_order_to_order_response()}
end)
test_pid = self() # <= 3
Test.Naive.LeaderMock
|> expect(:notify, fn :trader_state_updated, %Naive.Trader.State{} ->
send(test_pid, :ok) # <= 3
:ok
end)
Test.LoggerMock
|> expect(:info, 2, fn _message -> :ok end) # <= 4
...It’s important to note that we defined the mocked function with expected values in the above code.
We expect our test to subscribe to the trade events topic and broadcast to the orders topic(#1).
We are also expecting that the process will place an order at the exact values that we calculated upfront.
This way, our mock becomes an integral part of the test,
asserting that the correct values will be passed to other parts of the system(dependencies of the Naive.Trader module).
Another “trick”(#3) that we can use in our mocks is to leverage the fact that we can send
a message to the test process from within the mocked function.
We will use this idea to know precisely when the trader process finished its work as the notify/2 is
the last function call inside the process’ callback(handle_info/2 inside the Naive.Trader module).
We will assert that we should receive the message, and the test will be waiting for it before exiting(the default timeout is 100ms)
instead of using hardcoded sleep to “hack” it to work.
The final part(#4) tells the mox package that Logger.info/1 will be called twice inside the test.
The mox will verify the number of calls to the mocked function and error if it doesn’t match the expected amount.
The second part of the test is preparing the initial state for the Naive.Trader process, generating a trade event and sending it to the process:
# /apps/naive/test/naive/trader_test.exs
...
test "Placing buy order test" do
...
trader_state = dummy_trader_state()
trade_event = generate_event(1, 0.4318301, "213.10000000")
{:ok, trader_pid} = Naive.Trader.start_link(trader_state)
send(trader_pid, trade_event)
assert_receive :ok
endAs described above, the assert_receive/1 function will cause the test to wait for the message for 100ms before quitting.
Here are the helper functions that we used to generate the initial trader state and trade event:
# /apps/naive/test/naive/trader_test.exs
...
test "Placing buy order test" do
...
end
defp dummy_trader_state() do
%Naive.Trader.State{
id: 100_000_000,
symbol: "XRPUSDT",
budget: "200",
buy_order: nil,
sell_order: nil,
buy_down_interval: Decimal.new("0.0025"),
profit_target: Decimal.new("0.001"),
rebuy_interval: Decimal.new("0.006"),
rebuy_notified: false,
tick_size: "0.0001",
step_size: "0.001"
}
end
defp generate_event(id, price, quantity) do
%Core.Struct.TradeEvent{
event_type: "trade",
event_time: 1_000 + id * 10,
symbol: "XRPUSDT",
trade_id: 2_000 + id * 10,
price: price,
quantity: quantity,
trade_time: 5_000 + id * 10,
buyer_market_maker: false
}
endThe above code finishes the implementation of the test, but inside the test we used functions from the BinanceMock module that are currently private.
We need to update the module by making the generate_fake_order/4 and
convert_order_to_order_response/1 functions public(and moving them up in the module, so they are next to other public functions):
# /apps/binance_mock/lib/binance_mock.ex
...
def get_order(symbol, time, order_id) do
...
end
def generate_fake_order(symbol, quantity, price, side) # <= updated to public
...
end
def convert_order_to_order_response(%Binance.Order{} = order) do # <= updated to public
...
end
...We updated both of the methods to public and moved them up after the get_order/3 function.
16.6 Define an alias to run unit tests
Our unit test should be run without running the whole application, so we need to run it with the --no-start argument.
We should also select unit tests by tag(--only unit). Let’s create an alias that will hide those details:
We can now run our test using a terminal:
We should see the following error:
21:22:03.811 [error] GenServer #PID<0.641.0> terminating
** (stop) exited in: GenServer.call(BinanceMock, :generate_id, 5000)
** (EXIT) no process: the process is not alive or there's no process currently
associated with the given name, possibly because its application isn't startedOne of the BinanceMock module’s functions is sending a message to generate a unique id to the process
that doesn’t exist(as we are running our tests without starting the supervision tree[the --no-start argument]).
There are two ways to handle this issue:
- inside the
/apps/naive/test/test_helper.exsfile we could ensure that theBinanceMockis up and running by addingApplication.ensure_all_started(:binance_mock)function call - this is a hack - we could refactor the
BinanceMock.generate_fake_order/4to acceptorder_idas an argument instead of sending the message internally - this should be a much better solution. Let’s give it a shot.
First, let’s update the BinanceMock module:
# /apps/binance_mock/lib/binance_mock.ex
def generate_fake_order(order_id, symbol, quantity, price, side) # <= order_id added
...
# remove the call to GenServer from the body
...
end
...
defp order_limit(symbol, quantity, price, side) do
...
generate_fake_order(
GenServer.call(__MODULE__, :generate_id), # <= order_id generation added
symbol,
quantity,
price,
side
)Now we need to update our test to pass some dummy order id from the mocked function:
# /apps/naive/test/naive/trader_test.exs
...
test "Placing buy order test" do
...
{:ok, BinanceMock.generate_fake_order(
"12345", # <= order_id added
"XRPUSDT",
"464.360",
0.4307,
"BUY"
)
...
endWe can now rerun our test:
MIX_ENV=test mix test.unit
...
Finished in 0.1 seconds (0.00s async, 0.1s sync)
1 tests, 0 failures (1 excluded)Congrats! We just successfully tested placing an order without any dependencies.
To avoid explicitly passing the MIX_ENV=test environment variable, we will add the preferred environment for our alias inside the mix.exs file:
# /mix.exs
# add below under the `project` function
def cli do
[preferred_envs: ["test.unit": :test]]
endNow we can run our tests by:
mix test.unit
...
Finished in 0.06 seconds (0.00s async, 0.06s sync)
1 tests, 0 failures (1 excluded)We now have fast, isolated unit tests powered by mox. Here’s what we achieved:
- Defined behaviours for modules that lacked them (
BinanceMock,Naive.Leader,Phoenix.PubSub,Logger) - Used
Mox.defmock/2to generate mock modules constrained by those behaviours - Wired up environment-based configuration so tests use mocks while dev uses real modules
- Built a test for
Naive.Traderthat verifies the exact values passed to every dependency - Eliminated hardcoded
sleepcalls by using a message-passing feedback loop from mock to test
The key insight is that mox turns your mocks into assertions.
Instead of just stubbing out dependencies, each mock expectation verifies
that the right function was called with the right arguments.
Your mocks are doing double duty - enabling isolation and validating correctness.
It’s worth noting the trade-offs of the mox approach:
- behaviours must be defined for every module you want to mock - this is upfront work, but it pays off as a form of documentation
- module attributes are resolved at compile time, so you can’t swap implementations mid-test without recompilation
- mocked tests don’t catch integration issues - that’s what our end-to-end test from the last chapter is for
The two testing approaches complement each other: end-to-end tests prove the system works,
mox tests prove each piece works correctly in isolation.
So, What’s Next?
We’ve been building OTP processes and testing them, but we haven’t talked much about functional programming itself. In the next chapter, we’ll look at how to separate pure business logic from side effects. We’ll extract our trading strategy’s decision-making into pure functions that are trivial to test - no mocks needed at all.
[Note] Please remember to run mix format to keep things nice and tidy.
The source code for this chapter can be found in the book’s source code repository (branch: chapter_16).