In general, my heart sinks when I think I may have to write some unit tests for threaded code. This can be fiendishly difficult, and the secret with it is not to test things too absolutely, as timings and orders of events can be impossible to predict. Here are a few dos and don’ts.

Don’t

• Over assert everything – thing about the grand outcome you’re looking for and assert that
• Expect timings to be consistent between tests, even within margins
• Allow poor design in your code to lead to really hard to read tests – often concurrent things can make tight coupling in their code even harder to unit test – introduce interfaces to help with the testing
• Assume any particular execution order within your code
• Rely on Thread.sleep to keep things in sync – it can, but it won’t guarantee much.

Do

• Test on multiple machines, especially your CI server – you’ll find concurrency bugs that way
• Use sensible synchronization between your test code and any worker threads to keep control of things
• Keep it simple
• Use interfaces which can be mocked to see what your threads are doing to their outside world
• Keep it simple
• Keep it simple

and finally, here are two things you really should know when you’re doing concurrent testing

Tests Can Timeout

JUnit allows for a whole test to timeout. When you specify a timeout for the test, the test itself is run in a worker thread while JUnit watches to see if it takes too long, killing it with an exception if it overruns. This allows you to do things like join a thread that’s supposed to finish, in the hope that the whole test will end soon enough.

@Test(timeout=1000)
public void testSomething() {
    Thread thread = new Thread(new Runnable() {
        public void run() {
            waitForSomethingToHappen();

            // then we end
        }
    });

    // the worker is now started and is waiting for something to happen
    thread.start();

    // set the wheels in motion so that thing happens
    someService.triggerStartOfSomething();

    // now we wait for the worker thread to spot it
    thread.join();

    // if we get here without timeout then the system works.
}

Mockito’s Verify Can Wait Until Something Has Happened

When you have done the thing which should have provoked your concurrent code to get the result, you really need to wait for that result to propagate back your end point. If it’s a mockito mock, then use the timeout within the verify call to wait for everything to come back.

interface SomeInterface {
    // this is where the answer would be written by a worker
    void storeResultFromWorker(Result result);
}

@Test
public void workersCanConcurrentlyGetAnswer() {
    // the mock that will receive the answer
    SomeInterface someInterface = mock(SomeInterface.class);
 
    // create 100 worker threads to work on their answers
    for(int i=0; i<100; i++) {
        // let's assume the worker will do its thing and then tell someInterface about it
        createAndStartWorkerThread(i, someInterface);
    }

    // wait to see if all 100 threads got an answer
    verify(someInterface, timeout(1000).times(100)).storeResultFromWorker(any(Result.class));
}

In the above code, we’re testing that the workers all get an answer and tell the mock within the timeout period. This could well be enough testing. It saves us having to try to guess when they’ll all have run.

Various releases of Java have introduced more sophisticated concurrency patterns to Java. You don’t need to build your own thread pooling techniques, or queues between threads. However, for a lot of uses, the synchronized keyword is pretty much all you need.

I had a recent use case, for which it seemed that there wasn’t a sophisticated library class, nor was a single line of code enough. I wanted an object on one thread to be able to signal when it was done and for other threads to be able to block until that signal was made.

This is not that difficult to do, but you can’t just do this:

private Object event = new Object();

// for consumers to use to wait for the event
public void waitForEvent() {
    // to do a wait, we need to acquire the monitor
    synchronized(object) {
        // wait until object notifies - which relinquishes the lock on the object too
        object.wait();
    }
}

private void signalEvent() {
    synchronized(object) {
        // release all waiters
        object.notifyAll();
    }
}

The above, which illustrates the synchronization pattern for notify and wait in Java, isn’t quite enough. The reason it’s not enough is that in concurrent processing, it’s possible for the event to be signalled before the waiting thread calls waitForEvent. If the event has already been signalled, and you start waiting for a notify that never comes, then you’re blocked either forever or until a timeout.

Reading up on the subject, it seems it’s common to put some sort of boolean alongside this pattern.

Furthermore, it turns out that wait can abort spuriously without being notified, so if you’re not checking a boolean in a while loop, you may wake up early and be in an error state. This code needs taming!

My next advice. Object orientation ALWAYS helps with thread safety stuff in Java. I created a Signal class, whose job is to store the signal and allow callers to wait for it.

I’ve shared this code here on GitHub. Feel free to copy it and use it as you wish.

Here’s a snippet:

public class Signal {
    private boolean signalled = false;
	
    /**
      * Set the done flag
      */
    public synchronized void setSignal() {
        signalled = true;
        notifyAll();
    }

    // ... other stuff

     /**
       * Wait indefinitely for the done signal
       * @throws InterruptedException on thread error
       */
     public synchronized void waitForSignal() throws InterruptedException {
         while (!signalled) {
              wait();
         }
     }
}

What’s quite nice about this is that the object orientedness of it means it’s easily synchronized with the function level synchronized keyword – when you synchronize a function, that makes the whole containing object lock for the duration of the synchronization. Using this signal is very easy – it just acts like a flag, but you can wait for it as well as read it.

The following unit test (also in GitHub) shows how the signal makes it easy for threads to set gates on each other. Note that the final signal in this example could have been done with thread.join(), but wasn’t for the purposes of the test. Also, note how Junit’s @Test(timeout= ) syntax is used to ensure that the test which is running waiting threads, doesn’t go on forever owing to a bug.

	@Test(timeout=TIMEOUT)
	public void interThreadSignalling() throws InterruptedException {
		final Signal threadStarted = new Signal();
		final Signal threadAllowedToProceed = new Signal();
		final Signal threadFinished = new Signal();
		
		Thread thread = new Thread(new Runnable(){

			@Override
			public void run() {
				// set that we've started
				threadStarted.setSignal();
				
				// wait until we're allowed to proceed
				try {
					threadAllowedToProceed.waitForSignal();
				} catch (InterruptedException e) {
					// do nothing here
				}
				
				// signal we've finished
				threadFinished.setSignal();
				
			}
			
		});
		
		thread.start();
		
		// wait for the thread to have started
		threadStarted.waitForSignal();
		
		// thread should not have finished - so wait to see if it has and check it hasn't
		threadFinished.waitForSignal(SHORT_TIME);
		assertFalse(threadFinished.isSignalled());
		
		// then signal the thread to finish
		threadAllowedToProceed.setSignal();
		
		// then wait indefinitely for the thread to say it's finished
		threadFinished.waitForSignal();
		
		// reaching here before the overall timeout is proof that the signalling works between threads
	}

Hope this is useful to someone.