Shall we test our Router?

When I wrote How to do Outside-In TDD with Phoenix, I ran into a question: How to test our router definitions in Phoenix?

But I couldn't find a proper solution. Neither Phoenix nor Plug provides any test helpers for us to test the router.

With Rails, we certainly can

Rails provides us three built-in assertions to test our router definitions:

  • assert_generates for testing router helper functions
  • assert_recognizes for testing a path can be routed to a controller action
  • assert_routing for testing both

So it's natural for me to think about "Why can't Phoenix as a more functional web framework than Rails help me test this?"

I then opened a proposal to phoenix-core mailing list. Chris McCord (the creator of Phoenix) responds as following:

I don't feel such features would be helpful for testing, and I would actually discourage such tests. The fact that a router "generates" or "recognizes" a route is not something to test – we test requests through the application, and except results to come out the other side in the form of a response. So the fact that the router recognizes "/products" isn't a strong tests, rather we would test that GET "/products" sends us back a listing of products, so unit tests of the router itself are pointless.

Reading his response makes me thinking: am I going too far to pursue a higher test coverage and forgetting why we need tests?

Test, or not test?

Then I read What to test and not to test which answers this question perfectly.

It listed 2 main reasons for writing tests:

  1. Provide feedback about the API under testing (if doing TDD).
  2. Prevent regressions (but not errors).

So, can route tests justify their existence under these two reasons?

  1. Route tests provide little feedback about the API (the routes).

    assert_routing({ path: 'photos', method: :post }, { controller: 'photos', action: 'create' }) is way more complicated than resources :photos, only: [:create]

    In another word, we can already get enough design feedback by writing the routing definition itself.

  2. Route tests provide little value for preventing regressions.

    The router is the most straightforward code we can write in a web application. But it's also the most important part for a feature. If it's not working as expected, a feature would fail completely, then the feature test for this feature would fail even if we don't write any tests for this route.

    And let's see what kinds of bugs can happen to our router:

    1. A path cannot be routed.

      Then we add the missing route definition, the feature test passes, end of the story.

    2. A path is routed to a wrong controller.

      Then our feature test would fail if we write our feature test correctly and these two controllers are doing different things.

      We redirect the route to the correct controller, the feature test passes, end of the story.

    So, if we have at least one feature test for every one of our routing rules, we don't need any router tests.

More importantly, testing routers is like testing implementation details, which would add more unnecessary work if we want to refactoring. Just think of your server as a module and user requests as function calls1, everything would become clear:

  1. If you are writing a monolith application:

    Then the routing rule is absolutely an implementation detail. Every request is like a private function call. And we should never test a private function. So that we can refactor/rename these functions freely and our users can still use our application without noticing anything.

  2. If you are writing a backend API for your own clients:

    Then every request is like a public function call. But these functions are not documented to the outside world, which means they are private (this idea is familiar to an Elixir programming).

  3. If you are writing an API service for others:

    Then every request is a public function call. But we should be testing these "functions" with a more thorough feature test.

Using the "function call" metaphor, our route definitions are just "function names", and we almost never test a function is named correctly, right?

What should we do?

So, as you can see in the previous discussions: we should write feature tests for our applications, and since they've already covered the routing part, we don't need to test that part again.

You can find some great examples from plug/router_test.exs and phoenix/routing_test.exs.



I learned this idea from Programming Phoenix