The Bumbling Assertion

As part of our catalogue of Test Smells, there’s already Assertion Diversion where the wrong sort of assertion is used, reducing the meaning of errors and the clarity of the assertion. There’s also the Over Exertion Assertion where a complex custom-made assertion is used inline in every single use case. The Bumbling Assertion sits between the cracks of these anti-patterns.

Consider this code:

Optional<Foo> result = service.getFoos(123);
assertNotNull(result);
assertThat(result).isNotEmpty();
assertThat(result.getBar()).isNotNull();
assertThat(result.getBar()).hasSize(1);
assertThat(result.getBar().get(0)).isEqualTo("buzz");

The above code is valid. There are five assertions, which work up to a climax of asserting that the first item in the bar list is “buzz”.

It’s a bit wordy. The verbosity of it means that as a spec the best we can say about it is that it’s thorough. The problem is that we can’t really see what the objective of this test is. Exhaustive doesn’t equal clear in purpose.

There are plenty of tests you could do in life. I could, for example, every time I try to start my car, check that the key hasn’t fallen off the key fob. On the whole, I prefer not to verify every last detail at all times. In this car example, if I didn’t have the key, the car wouldn’t start. Hopefully with a no key error!

The above code could be written more clearly:

Optional<Foo> result = service.getFoos(123);

// we're not testing for a null optional
// because WHO has THAT
// we'll also get errors if the optional is empty
assertThat(result.get().getBar())
    .containsExactly("buzz");

The above test runs the risk of failing with a null pointer exception or empty optional error before completing the assertion, and it’s a good idea to use the assertion libraries in the most powerful way to wrap up these sorts of errors. It can be tempting, to write the let’s test the ground under our feet assertions to debug any test failures, but that should be the job of other tests, not a test that’s trying to prove the end result.

For example, if we felt that there was ever a risk of getting a null back from the service, instead of an Optional.empty we could reasonably write a test:

@Test
public void whenNotFoundThenEmpty() {
    assertThat(service.getFoos(999)).isEmpty();
}

This cements the idea that the service returns a valid optional, making it unnecessary to assert for that in other tests.

What if there’s No Assertion?

How can an assertion be bumbling if there isn’t one?

if (size > 3) {
   throw new AssertionError("The size should not be more than 3");
} else if (size < 3) {
   throw new AssertionError("OMG the size is less than the required 3");
}

A general rule of thumb is that if you have to either throw new AssertionError or call Assert.fail you’re probably in the middle of a bumbling assertion. This may be a little less true if you’re WRITING an assertion library, but arguably most assertion libraries should be built on the shoulders of existing assertion library primitives.

Enough Bumbling

The TL;DR of this is that you should write just enough assertions at the end of your test to explain what it is that you’re trying to prove. Use the most powerful assertion features to help explain the failure and to express the purpose of the assertion as clearly as possible.

Remember, your test should, wherever possible, read like the spec of the component under test, so make the assertions clear and to the point.

So, one assertion per test?

Well, kind of. You may still need to use a few assertion calls to establish all the details of your final results, but they should, as a group, read as though they’re demonstrating the single purpose of the when bit of your given/when/then.

If you see multiple assertions, read carefully and simplify if you can.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s