A SureFire Classpath Fail!

Sadly, even the best plugins can work against you. We’ve been having a problem recently with Spring, Maven and SureFire. Specifically, it seemed that some of our unit tests ran absolutely fine with Spring and properties files when run via Eclipse, but got different behaviour when run via Maven. Even more specifically, the workaround seemed to be to duplicate values from .properties files that were in one jar file by putting them into a .properties file in the module whose tests were failing under Maven – usually by adding more properties to files in src/test/resources.

What this seemed to imply was that the classpath when running via eclipse, was set in such a way that the full module dependency structure was harnessed when looking for properties to evaluate in the @Value(“${someproperty}”) annotations, but was not being utilised at test time by Maven. Notably, these properties were used from the correct places at runtime in production.

Aaagh. What undocumented hell is this?

The solution is quite simple. You must not use the system class path in surefire if you want this to work. Here’s a snippet from our surefire configuration in the Maven pom.xml file:


As you can see, we’ve set this key to false. Now everything works. That’s it. You don’t want a default classloader: you want the one with all your dependencies in. That allows the Spring class path based dependencies to work correctly.

There was nothing about this on StackOverflow and it took months before we found the answer. Here it is now for posterity.

Posted in Uncategorized

When you want to disable a slow running test

A quick one. The code for this is here on GitHub.

When testing your software with automated unit tests, you want to be thorough to get code coverage and genuinely cover system behaviour, yet some of the things you want to test can take a fair bit of time to run through tests.

This poses a dilemma:

  • Make the test feedback process as quick as possible
  • Make the test process as thorough as possible

The best answer is to organise your code so that it can be thoroughly tested fast. If that’s not possible you may need a multi-tier automated test. The quick things can be tested straight away with a first-line of feedback, ideally within 5 minutes. Slower things should follow, with the whole end-to-end feedback in under 20 minutes if possible.

This leads to a dilemma – how and when do we disable long-running tests? And, given that we’re often using a combination of maven and within-IDE test runners, how can we disable a test in all contexts EXCEPT the one where it should be run. In my view moving a long-running test into the integration tests, just because you don’t like it, isn’t a great solution. Worse than that, you can jump through hoops trying to break things into categories, only to find that your IDE test runner is running them anyway.

A solution that worked for us recently was to disable certain JUnit tests based on a system property. If the property is not set, the test just doesn’t run. Simple as that. This can be done easily in JUnit by an assumption failure. More simple than that there’s the use of a JUnit rule to take the whole test class and make all of its tests abort if the system property isn’t set.

The code on GitHub is the JUnit rule I created. It is illustrated by this unit test:

 * Quick example of how to use this using a simpler test than that used to verify
 * that the mechanism works
 * The test within this class will only work if the system property "run.test" is set to "true"
 * To demo this running try setting the JVM option -Drun.test=true, which you can do in the maven
 * command line, or via your IDE's invocation of the junit runner
public class ExampleTest {
	public RequireSystemProperty property = new RequireSystemProperty("run.test", "true");
	public void systemPropertyDependentTest() {
		// do some sort of test we only want to do when a system property is set...

As you can see – the presence of this rule effectively stops that test running unless you specifically configure the JVM that’s running it with the system property that would activate it. We use this on our CI server in the integration test build where we enable everything and the build takes maybe 12-15 minutes. On the front-line CI build, we want the build time to be about 5 minutes, so we don’t run integration tests, or code coverage tools, or slow-running things.

Hope this is useful.

Tagged with: , ,
Posted in Uncategorized

Multi Module Maven Builds, Code Coverage and Sonar

I love SonarQube for its potential. The reality of it is somewhat different, you need a byzantine series of configurations to make it play nicely with Jenkins, Maven and Jacoco. It gets more difficult when you have a multi module build – i.e. a parent pom with modules set up in it.

I’m not going to claim any ownership for the decent how-to on this. The amazing David Valeri in https://davidvaleri.wordpress.com/2013/09/06/tracking-integration-test-coverage-with-maven-and-sonarqube/ has nailed this issue. Here is a quick summary of what you need to look out for:

  • You probably run your integration tests using the failsafe runner rather than surefire
  • If multiple projects have different bits of integration test, you’ll need to configure the jacoco plugin so that it shares a single jacoco.exec file across all modules
  • If one of your projects is an integration test project (our case) you just need to know where jacoco will write its output
  • In the parent pom.xml you will create a property for sonar to tell it where to find the integration test coverage report
  • Paths, especially once jenkins gets involved, can be complex

If you get it right, it works a treat.

See the linked post for details. Kudos to David!


Posted in Uncategorized

Handy Tips and Tricks For Finding Linux Disk Usage

I’ve no idea how I know that df -h is how you find disk free space… if you want to do more directory based stuff, then this blog is for you:


I’m glad they wrote it so I don’t have to.

Tagged with:
Posted in Uncategorized

Concurrent Unit Tests

In general, my heart sinks when I think I may have to write some unit tests for threaded code. This can be fiendishly difficult, and the secret with it is not to test things too absolutely, as timings and orders of events can be impossible to predict. Here are a few dos and don’ts.


  • Over assert everything – thing about the grand outcome you’re looking for and assert that
  • Expect timings to be consistent between tests, even within margins
  • Allow poor design in your code to lead to really hard to read tests – often concurrent things can make tight coupling in their code even harder to unit test – introduce interfaces to help with the testing
  • Assume any particular execution order within your code
  • Rely on Thread.sleep to keep things in sync – it can, but it won’t guarantee much.


  • Test on multiple machines, especially your CI server – you’ll find concurrency bugs that way
  • Use sensible synchronization between your test code and any worker threads to keep control of things
  • Keep it simple
  • Use interfaces which can be mocked to see what your threads are doing to their outside world
  • Keep it simple
  • Keep it simple

and finally, here are two things you really should know when you’re doing concurrent testing

Tests Can Timeout

JUnit allows for a whole test to timeout. When you specify a timeout for the test, the test itself is run in a worker thread while JUnit watches to see if it takes too long, killing it with an exception if it overruns. This allows you to do things like join a thread that’s supposed to finish, in the hope that the whole test will end soon enough.

public void testSomething() {
    Thread thread = new Thread(new Runnable() {
        public void run() {

            // then we end

    // the worker is now started and is waiting for something to happen

    // set the wheels in motion so that thing happens

    // now we wait for the worker thread to spot it

    // if we get here without timeout then the system works.

Mockito’s Verify Can Wait Until Something Has Happened

When you have done the thing which should have provoked your concurrent code to get the result, you really need to wait for that result to propagate back your end point. If it’s a mockito mock, then use the timeout within the verify call to wait for everything to come back.

interface SomeInterface {
    // this is where the answer would be written by a worker
    void storeResultFromWorker(Result result);

public void workersCanConcurrentlyGetAnswer() {
    // the mock that will receive the answer
    SomeInterface someInterface = mock(SomeInterface.class);
    // create 100 worker threads to work on their answers
    for(int i=0; i<100; i++) {
        // let's assume the worker will do its thing and then tell someInterface about it
        createAndStartWorkerThread(i, someInterface);

    // wait to see if all 100 threads got an answer
    verify(someInterface, timeout(1000).times(100)).storeResultFromWorker(any(Result.class));

In the above code, we’re testing that the workers all get an answer and tell the mock within the timeout period. This could well be enough testing. It saves us having to try to guess when they’ll all have run.

Tagged with: , ,
Posted in Uncategorized

Signalling between threads in Java

Various releases of Java have introduced more sophisticated concurrency patterns to Java. You don’t need to build your own thread pooling techniques, or queues between threads. However, for a lot of uses, the synchronized keyword is pretty much all you need.

I had a recent use case, for which it seemed that there wasn’t a sophisticated library class, nor was a single line of code enough. I wanted an object on one thread to be able to signal when it was done and for other threads to be able to block until that signal was made.

This is not that difficult to do, but you can’t just do this:

private Object event = new Object();

// for consumers to use to wait for the event
public void waitForEvent() {
    // to do a wait, we need to acquire the monitor
    synchronized(object) {
        // wait until object notifies - which relinquishes the lock on the object too

private void signalEvent() {
    synchronized(object) {
        // release all waiters

The above, which illustrates the synchronization pattern for notify and wait in Java, isn’t quite enough. The reason it’s not enough is that in concurrent processing, it’s possible for the event to be signalled before the waiting thread calls waitForEvent. If the event has already been signalled, and you start waiting for a notify that never comes, then you’re blocked either forever or until a timeout.

Reading up on the subject, it seems it’s common to put some sort of boolean alongside this pattern.

Furthermore, it turns out that wait can abort spuriously without being notified, so if you’re not checking a boolean in a while loop, you may wake up early and be in an error state. This code needs taming!

My next advice. Object orientation ALWAYS helps with thread safety stuff in Java. I created a Signal class, whose job is to store the signal and allow callers to wait for it.

I’ve shared this code here on GitHub. Feel free to copy it and use it as you wish.

Here’s a snippet:

public class Signal {
    private boolean signalled = false;
      * Set the done flag
    public synchronized void setSignal() {
        signalled = true;

    // ... other stuff

       * Wait indefinitely for the done signal
       * @throws InterruptedException on thread error
     public synchronized void waitForSignal() throws InterruptedException {
         while (!signalled) {

What’s quite nice about this is that the object orientedness of it means it’s easily synchronized with the function level synchronized keyword – when you synchronize a function, that makes the whole containing object lock for the duration of the synchronization. Using this signal is very easy – it just acts like a flag, but you can wait for it as well as read it.

The following unit test (also in GitHub) shows how the signal makes it easy for threads to set gates on each other. Note that the final signal in this example could have been done with thread.join(), but wasn’t for the purposes of the test. Also, note how Junit’s @Test(timeout= ) syntax is used to ensure that the test which is running waiting threads, doesn’t go on forever owing to a bug.

	public void interThreadSignalling() throws InterruptedException {
		final Signal threadStarted = new Signal();
		final Signal threadAllowedToProceed = new Signal();
		final Signal threadFinished = new Signal();
		Thread thread = new Thread(new Runnable(){

			public void run() {
				// set that we've started
				// wait until we're allowed to proceed
				try {
				} catch (InterruptedException e) {
					// do nothing here
				// signal we've finished
		// wait for the thread to have started
		// thread should not have finished - so wait to see if it has and check it hasn't
		// then signal the thread to finish
		// then wait indefinitely for the thread to say it's finished
		// reaching here before the overall timeout is proof that the signalling works between threads

Hope this is useful to someone.

Tagged with: , ,
Posted in Uncategorized

Mockito Chaining

Dynamic mocks are great. I believe it was Martin Fowler who first switched me onto them. What we should be testing is how an object interacts with its surroundings. Often this means setting it running and verifying that it made the right outbound calls given an input. Sometimes, with more autonomous code, you need to simulate how the outside world will appear to it, and this can get complex.

Mockito is the tool of choice these days for a lot of Java developers as it has a fairly self explanatory syntax and is powerful enough for doing most of what you need. Note: Mockito is not a panacea. It can be more complex to try to pre-record in a dynamic mock what’s expected of it, than it might be to implement a simple-in-memory service that will act the way you want your simulation of a real service to act. Maybe that’s worth an article of its own another day.

I want to show how more than one Mockito doReturn, doThrow, thenReturn, thenThrow can be chained, so you can simulate something.

Let’s imagine a bit of code that’s going to use a getNext method.

public interface Source {
    String getNext();

Now let’s imagine a simple test set up for simulating what that would do

    Source mockSource = mock(Source.class);


In the above, every time the getNext is called, it will return “hello”. Very easy. This syntax is nicest when the method returns something. The key thing is you can chain then. Let’s simulate a getNext which returns different values including throwing an exception.

      .thenThrow(new RuntimeException("I ran out of data"));

The calls to that mock will get different responses each time.

This works with the doThrow/doAnswer/doReturn family of stub set ups too. These are best when your function doesn’t return something. You can use them always, but I prefer the thenReturn/thenThrow family for clarity.

Let’s imagine a void function “doSomething” and see how to mock it:

    doThrow(new RuntimeException("blah"))

In the above, the first call to doSomething causes an exception and the next one doesn’t.

I found this useful in practice.

Once your test has run, you may then use Mockito’s verify to see which calls were made to which mocks.

Tagged with: , ,
Posted in Uncategorized