Concurrent Unit Tests

In general, my heart sinks when I think I may have to write some unit tests for threaded code. This can be fiendishly difficult, and the secret with it is not to test things too absolutely, as timings and orders of events can be impossible to predict. Here are a few dos and don’ts.

Don’t

• Over assert everything – thing about the grand outcome you’re looking for and assert that
• Expect timings to be consistent between tests, even within margins
• Allow poor design in your code to lead to really hard to read tests – often concurrent things can make tight coupling in their code even harder to unit test – introduce interfaces to help with the testing
• Assume any particular execution order within your code
• Rely on Thread.sleep to keep things in sync – it can, but it won’t guarantee much.

Do

• Test on multiple machines, especially your CI server – you’ll find concurrency bugs that way
• Use sensible synchronization between your test code and any worker threads to keep control of things
• Keep it simple
• Use interfaces which can be mocked to see what your threads are doing to their outside world
• Keep it simple
• Keep it simple

and finally, here are two things you really should know when you’re doing concurrent testing

Tests Can Timeout

JUnit allows for a whole test to timeout. When you specify a timeout for the test, the test itself is run in a worker thread while JUnit watches to see if it takes too long, killing it with an exception if it overruns. This allows you to do things like join a thread that’s supposed to finish, in the hope that the whole test will end soon enough.

@Test(timeout=1000)
public void testSomething() {
    Thread thread = new Thread(new Runnable() {
        public void run() {
            waitForSomethingToHappen();
 
            // then we end
        }
    });
 
    // the worker is now started and is waiting for something to happen
    thread.start();
 
    // set the wheels in motion so that thing happens
    someService.triggerStartOfSomething();
 
    // now we wait for the worker thread to spot it
    thread.join();
 
    // if we get here without timeout then the system works.
}

Mockito’s Verify Can Wait Until Something Has Happened

When you have done the thing which should have provoked your concurrent code to get the result, you really need to wait for that result to propagate back your end point. If it’s a mockito mock, then use the timeout within the verify call to wait for everything to come back.

interface SomeInterface {
    // this is where the answer would be written by a worker
    void storeResultFromWorker(Result result);
}
 
@Test
public void workersCanConcurrentlyGetAnswer() {
    // the mock that will receive the answer
    SomeInterface someInterface = mock(SomeInterface.class);
  
    // create 100 worker threads to work on their answers
    for(int i=0; i<100; i++) {
        // let's assume the worker will do 
        // its thing and then tell someInterface about it
        createAndStartWorkerThread(i, someInterface);
    }
 
    // wait to see if all 100 threads got an answer
    verify(someInterface, timeout(1000).times(100))
      .storeResultFromWorker(any(Result.class));
}

In the above code, we’re testing that the workers all get an answer and tell the mock within the timeout period. This could well be enough testing. It saves us having to try to guess when they’ll all have run.