Crossing The JUnit Streams

How to avoid a situation where JUnit 4 stuff incorrectly enters JUnit 5 tests and everything gets enormously confusing!

One of the nice things about JUnit 5 migration is that you can run your JUnit 4 tests in vintage mode and everything’s still compatible. One of the down sides is that some of the annotations and methods have the same name in JUnit 4 and JUnit 5 and it’s very easy, when both sets of the library dependencies are available, to import the wrong stuff and produce a test that doesn’t make work.

It’s worse, though, when the test that doesn’t make sense also doesn’t fail the build.

Consider this test:

import org.junit.Test;
import org.junit.jupiter.api.BeforeEach;

import static org.junit.Assert.assertEquals;

public class AccidentalJUnit4Test {
    @BeforeEach
    public void beforeEach() {

    }

    @Test
    public void test() {
        assertEquals(1, 1);
    }
}

This is some horrific soup of JUnit 5’s annotations and JUnit 4’s. It runs in the IDE, but in the maven build it’s ignored as the @Test is from the wrong JUnit and I do not have junit-vintage running.

So run junit-vintage?

How Did This Occur?

In my case, I’ve imported TestContainers integration for JUnit 5, which has transitive dependencies to JUnit 4. That’s not great, but it’s not the end of the world. However, I only want JUnit 5 tests in my code, and yet I can accidentally write tests with JUnit 4 bits in, and nobody will notice!

These half formed tests were never meant to be, so I want them to fail the build.

What Doesn’t Work

  • Checkstyle – checkstyle could scan for forbidden import statements, but I don’t scan src/test with it, and the checkstyle rules for our project are shared with another project which uses junit-vintage in a valid way.
  • Macker – a complex scanner which seems not to have an answer out of the box
  • Enforcer – this would stop me from including the JUnit 4 dependency… except I can’t help but allow that

Why Should I Care?

Making things mistake proof by adding automation to spot known errors and tell you about them is much better than leaving warnings around the place, with the error still possible.

It’s like when someone puts up a sign saying warning this water is very hot rather than providing water at the right temperature!

Anything which can give us a forcing function is a benefit.

What Does Work

I found a silly and simple answer to this on GitHub.

This Maven Grep plugin works well:

  <build>
    <plugins>
      <!-- grep maven plugin set to filter naughty JUnit4 stuff -->
      <plugin>
        <groupId>net.radai</groupId>
        <artifactId>grep-maven-plugin</artifactId>
        <version>1.1</version>
        <executions>
          <execution>
            <goals>
              <goal>grep</goal>
            </goals>
            <phase>test</phase>
            <configuration>
              <greps>
                <grep>
                  <failIfFound>true</failIfFound>
                  <filePattern>src/test/java/**/*.java</filePattern>
                  <grepPattern>import\s+(static\s+)?org\.junit\.(Assert|Test|Before|After|AfterClass|Assume|BeforeClass|ClassRule|Rule|FixMethodOrder|Ignore|Rule)</grepPattern>
                  <outputPattern>Found JUnit 4 imports in file ${fileName} at line ${lineNumber} : ${line}</outputPattern>
                </grep>
              </greps>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>

<!-- you also need to add the distribution repo -->
  <pluginRepositories>
    <pluginRepository>
      <id>ossrh</id>
      <url>https://oss.sonatype.org/content/groups/public</url>
    </pluginRepository>
  </pluginRepositories>

The above will work for me to prevent a mistake happening, it may work for you.

I’ve placed a working (well, failing for the right reasons) example of the above code in GitHub.

Credit Where It’s Due

I nearly gave up on the above problem. Luckily, the open source community is brilliant.

Radai Rosenblatt wrote this plug-in in 2016. A contributor called Michal Lozinski added the file pattern scanning in 2017.

When we first tried to use the above configuration it didn’t work. The docs didn’t describe how to do this, but reading the code of the plugin showed that filePattern could be used. However, it didn’t work.

I contacted Radai today and he updated the release of the plug-in, and now it works.

Without open source, this wouldn’t be possible. Without authors taking responsibility for doing favours to strangers, this wouldn’t be possible.

Thanks!!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s