Building an Elixir Rate Limiter with ExRated, ExUnit, and Doctests via TDD

This is a beginner-friendly step-by-step Elixir guide. If you would like to skip to the completed code on Github, scroll to the end of the guide for the link!

There are many reasons to implement request rate-limiting in a software system, including:

• Preventing brute force password cracking attempts.
• Preventing Denial of Service (DoS) attacks.
• Preventing billing attacks.
• Limiting requests to your public APIs to keep user accounts within their subscription tier usage limits (and preventing devs from accidentally DoSing your public endpoints*).

In this guide, we’ll use Test-driven Development (TDD) via ExUnit with doctests to write a bucket-based ExRated Elixir rate limiter for preventing brute force password cracking attempts.

ExRated is an Elixir port of the Erlang community’s raterlimiter library that uses a GenServer to track function calls in buckets saved to Erlang Term Storage (ETS).

Another option for bucket-based rate limiting in Elixir is ExHammer, a library to which I have contributed. While ExHammer is super easy to use, its default ETS backend is not yet ready for primetime.

*I worked for a Hong Kong startup long ago that accidentally took down a production LinkedIn APAC-region server after an intern wrote an infinitely-looped HTTP request. Rate limit your public APIs!

Project Setup

1. Install Elixir if you have not already.
2. On your command line, create a new Elixir project called RateLimiter:

mix new rate_limiter

This command will generate a /rate_limiter/ Elixir project directory with Hello World code in /lib/rate_limiter.ex .

An ExUnit unit test corresponding to /lib/rate_limiter.ex is generated in /test/, and the Logger OTP library is pre-included in :extra_applications in our mix config file, /mix.exs.

3. Open /mix.exs and add :ex_rated to our dependencies:

4. Back on your CLI, run mix deps.get to install our new dep.

Unit Testing Setup

Because ExRated uses GenServer and ETS under the hood, we must either mock ExRated function calls using a library such as Mox, or make sure our ExRated processes have started before we run our test scripts.

If we mock ExRated, we won’t really be testing our ExRated rate limiter; we’ll be testing an imitation of our ExRated rate limiter.

Let’s test ExRated. Open /tests/test_helper.exs and add line 3 below:

test_helper.exs runs before any of our tests, and Application.ensure_all_started(:ex_rated) ensures ExRated is available.

Now, open /tests/rate_limiter_test.exs and reduce it down to just doctest RateLimiter:

doctestwill look for and test code examples in @doc comment blocks in our module code — now let’s write function documentation that includes our first tests!

Rate Limiter Setup and the @doc Block

1. Open /lib/rate_limiter.ex
2. Replace the auto-generated code with the following:

The above values prefixed with @ are module attributes, variables we can use freely throughout this RateLimiter module.

3. Declare a function called allow_login_attempt?/2 with a preceding function @doc containing example usage and output:

Note the two example function calls preceded by iex>.

4. Back on the command line, run mix test at the root directory.

You should see our in-doc example function calls fail:

  1) doctest RateLimiter.allow_login_attempt?/2 (1) (RateLimiterTest)
     test/rate_limiter_test.exs:4
     Doctest failed
     doctest:
       iex> RateLimiter.allow_login_attempt?("doctest@example.com", 1)
       true
     code:  RateLimiter.allow_login_attempt?("doctest@example.com", 1) === true
     left:  nil
     right: true
     stacktrace:
       lib/rate_limiter.ex:14: RateLimiter (module)



  2) doctest RateLimiter.allow_login_attempt?/2 (2) (RateLimiterTest)
     test/rate_limiter_test.exs:4
     Doctest failed
     doctest:
       iex> RateLimiter.allow_login_attempt?("doctest@example.com", 0)
       false
     code:  RateLimiter.allow_login_attempt?("doctest@example.com", 0) === false
     left:  nil
     right: false
     stacktrace:
       lib/rate_limiter.ex:17: RateLimiter (module)


Finished in 0.00 seconds (0.00s async, 0.00s sync)
2 doctests, 2 failures

The Power of the @doc Block

As you can see above, the Elixir @doc block is extremely powerful; in a way, we are conducting Documentation-driven Development (Triple D), or Documentation-test-driven Development (DTDD) by writing our in-code doc first!

Besides providing a way to quickly write tests, @doc blocks also provide code introspection in IDEs and can be converted to sleek HTML documentation using ExDoc.

Implementing ExRated

While still in lib/rate_limiter.ex, write a case statement for ExRated.check_rate/3 within allow_login_attempt?/2:

Here, we use ExRated.check_rate(bucket_name, time_scale, rate_limit) to check the count in an email’s login attempt bucket ("login_attempt:#{email}") against our 10 minute @time_window and a given limit :

case ExRated.check_rate("login_attempt: #{email}", @time_window, limit) do

If no limit is passed as an argument, limit defaults to @email_request_limit as specified by limit \\ @email_request_limit in our declared arguments.

If the count within a bucket exceeds the specified limit, ExRated.check_rate/3 returns an {:error, limit} tuple, in which case we log a warning about the excessive login attempt and return false.

If the specified limit has not been exceeded, ExRated.check_rate/3 returns {:ok, count} and our function returns true :

{:ok, _count} -> true
{:error, _limit} -> false

Now, run mix test again — our doctests should pass:

..
Finished in 0.01 seconds (0.00s async, 0.01s sync)
2 doctests, 0 failures

Doctest Limitations

While doctests are super cool and useful, they can’t and shouldn’t replace explicit unit testing.

One of doctest’s limitations is the inability to control whether tests run asynchronously or synchronously. Doctests run concurrently by default, meaning if we wrote the following doctests:

 iex> RateLimiter.allow_login_attempt?("doctest@example.com", 1)
 true
    
 iex> RateLimiter.allow_login_attempt?("doctest@example.com", 1)
 false

…these tests would race each other every time, sometimes passing and sometimes failing.

The stable doctest for our false case, allow_login_attempt?("doctest@example.com", 0) , would never be called in real life, so let’s do some real testing!

Implementing Unit Tests and Logging

1. Open /tests/rate_limiter_test.exs and add the following at the top of the module:

2. Because we will use the same test email and bucket for multiple tests, let’s delete our login attempt bucket before every test using ExUnit’s setup callback:

3. Under our doctest call, write the following test:

With a @limit of 2, our test asserts that our rate limiter will allow two sequential login attempts for our @test_email.

4. Write another test that expects false with a log on a third attempt:

Here, once we are over the login attempt limit we assert that a message containing “Login attempt limit exceeded” will be logged, using our handy capture_log/1 function from ExUnit.CaptureLog.

Our full test file should look like this:

and when we run mix test, we should see our log test assertion fail:

  1) test allow_login_attempt?/2 returns false with a warning log when over the login attempt limit (RateLimiterTest)
     test/rate_limiter_test.exs:23
     Assertion with =~ failed
     code:  assert log =~ "Login attempt limit exceeded for email: #{@test_email}"
     left:  ""
     right: "Login attempt limit exceeded for email: test@example.com"
     stacktrace:
       test/rate_limiter_test.exs:31: (test)

...
Finished in 0.01 seconds (0.00s async, 0.01s sync)
2 doctests, 2 tests, 1 failure

Logging a Warning

1. Return to /lib/rate_limiter.ex and call Logger.warning/1 with our expected message:

2. Run mix test again — our tests should pass! 🎊

..
15:25:16.810 [warning] Login attempt limit exceeded for email: doctest@example.com
..
Finished in 0.01 seconds (0.00s async, 0.01s sync)
2 doctests, 2 tests, 0 failures

Bonus: Function Typespecs

Let’s add a @spec type specification to our allow_login_attempt?/2 function for all the type-heads out there:

As you can see, spec indicates a function’s argument and output types with @spec function_name(argument types…) :: expected_result_type.

@specs improve IDE type introspection, enhance ExDoc doc generation, and improve code readability in addition to enabling tools such as Dialyzer.

Bonus: Run-time Function Input Guards

Let’s add some run-time guards to ensure our allow_login_attempt?/2 function receives appropriate input:

is_binary(email) ensures that our email input is a valid binary, while we use is_integer and limit >= 0 to ensure only valid, non-negative limits are attempted by this function.

Thanks for reading!

If you are interested in a Part 2 in which I cover how to implement this rate limiter in a Phoenix project and track IPs from Phoenix conns, let me know in the comments below!

If you’re looking for an all-encompassing Elixir security solution that automatically blocks things like nefarious requests proxied through AWS, check out Paraxial.io. (This is not a sponsored post, I just like Paraxial.)

Check out this Github repository for the full code

Connect with me on LinkedIn if you’re looking for Elixir collaborators!