When I started coding (waaay back in the late 90s) testing wasn’t even on the radar. I was living fast on Perl and PHP, having fun with how quickly I could code up some idea, reload Netscape, and see an instant change. Many years, books, projects, and tests later I now have some (hopefully well-grounded) opinions about the whys and hows of testing. Working at Seeq has deepened my appreciation for the value and art of testing as I’ve gotten to see how a great testing culture can enable a product to grow from inception to rollout without the wheels coming off (ok, sometimes the wheels still come off, but when we put them back on we make darn sure they’re thoroughly bolted on with some new tests!). Here’s what makes the top of my list when I think about what I’ve learned about testing over the years.
❤️ the details of the craft
The most important thing I’ve learned isn’t a technique or pattern, but a mindset. Once I started viewing the whole process, from design, to code, to test, to pull request, to documentation, as part of a creative process I was able to shift my mindset from drudgery and have-to to enjoyment and get-to. Testing, like the coding of the feature itself, takes creativity, can be aesthetically pleasing if done right, and often requires solving a unique set of problems. Of course, there are details that feel like wading through mud, but coding is a creative profession which means a solution can be sculpted. Tests are slow to run? Dig in to see why. Tests are repetitive or difficult to write? Design a mini domain specific language or abstract away the repetitive parts.
Test the API, not the implementation
The main purpose of a unit-level test is to give you confidence that the code can be refactored, and only secondarily to avoid a customer-facing bug. Test should only be concerned with the public interface of the thing under test and know as little as possible about the implementation details as possible. This is also known as black box testing, connoting the idea that the test should be clueless of what’s inside the box of the function under test. Moving from testing AngularJS code to React components this last year has brought this to light for me. Many of the tests for our AngularJS apps were for things like the controller, which is actually an implementation detail of the public-facing HTML component. We chose to use the React Testing Library as a way to force a new paradigm for frontend tests: there is no access to the internal state or useEffect callbacks, there is only the component that can be interacted with using events like click and setText, which is the public API the browser also uses to manipulate it.
Tests can be documentation
A well-written test and its description acts as a form of documentation, and one that is slightly better than a comment since it is more likely to be kept up to date as the code changes. Of course this means aiming for the same level of code quality in tests as in production with things like helpful variable and function names, avoiding repetitive code, etc. And like the test itself, the description should describe the public behavior, not implementation details. For example, it sets the isFoo flag to bar when called by X is going to have to change if isFoo is refactored and isn’t much help in describing why the behavior is that way.
The art of mocking
Mocks are a necessary part of most tests, and for good reason, since they allow the test to isolate how much of the system is being tested. Mocks are a form of the Law of Demeter in that they are decoupling the method under test from an implementation detail and limiting knowledge about how that other part of the system works. However, mocks are also a code smell, meaning they may indicate a deeper problem. They can indicate that there is tight coupling between two units of code and present an opportunity to decouple them so that mocking isn’t needed. Mocks make the test harder to understand and can drift from reality by returning incorrect data which allows for false-positive tests. If you find yourself mocking out so much that the test feels like its not actually giving any confidence for a later refactor and the code can’t be decoupled then it may be time to push the test up a level. Which brings us to…
Test at the lowest level, but no lower
Another maxim that is more art than science. The lower in the stack, or the closer to the unit of code under test, the faster it will run and the easier it will be to debug. However, lower-level tests will have a narrower scope or blast radius in terms of how much they test. Once you need to test the interaction of multiple components or find yourself adding too many mocks, it may be time to move up to an integration or end-to-end test. When writing these higher-level tests keep in mind the trade-offs: they have a wider blast radius so they can test much more, but because of that they can be harder to debug and slower to run.
Speed is key and time is relative
Fast tests are important to keep build times down — imagining the thousands of times a test will run and the developer hours waiting for it should give some pause when writing a slow test. Fast tests are also more likely to be useful when refactoring, because who wants to pause their workflow to let a minutes-long test set up and run?
For both speed and reliability avoid any test that relies on wall-clock time. That means no Thread.sleep(), mocking all debounces and timeouts with fake clocks, and finding a deterministic way to know when concurrent processes are all finished. For example, we had many integration tests that would hit the API with concurrent requests and poll every couple hundred milliseconds to see if they were finished. Straightforward enough, but as we wrote more in this style the slowness they added to the build was substantial and the number of hard-to-debug failures kept increasing. One of the developers on our team took a step back to design a better pattern: a special HTTP interceptor that collects each response before releasing them all back to the waiting test when they are ready. Speed and reliability for the win.
Good tests can make for better code
If some unit of code is hard to test it may be presenting an opportunity to write better code that is more loosely coupled and easier to test. This is one of the reasons I enjoy testing so much: not only does it find logic errors, it helps me craft cleaner, more re-usable, and easier to read code. It’s a forcing function for going back to design pattern fundamentals to find ways to reduce the complexity of some hard-to-test function or library. Some of my favorites are refactoring the code to pure functions that don’t rely on state and have no side-effects (so only the inputs and outputs need to be tested), breaking apart large units of code via the single responsibility principle, and reducing the number of conditional branches by encoding them as data (e.g. a map of options).
What’s your list of testing mantras? I’m sure I missed someone’s favorite, but a list, by definition, is bounded 😊