Automated Testing is a Broad Church

I’ve had the pleasure to be working on an automated testing framework recently. This seems to solve a problem that we’ve been having with Cucumber. I will write a more detailed piece on this in the near future, but here’s the elevator pitch.

JUnit and Mockito tests tend to be too technical so we can’t use them as an acceptance test framework directly.

Cucumber is the go-to technology for BDD/ATDD but implementing it can be cumbersome (cucumbersome perhaps? or an encucumbrance? – who knows!?).

In short, if you want the Given/When/Then and documentation friendly features of Cucumber you have to pay the price of using Cucumber.

What’s Wrong with Cucumber?

Nothing necessarily. Once you’re dealing with lots of similarly documented specs, especially if they’re simple, Cucumber can be a real boon.

However, for Cucumber to work you have to phrase your Gherkin right, implying but not implementing the test script. Then you need to write your glue code just write, and then you need one or two tiers of test execution code. This means you may have to cross 3 or 4 layers of software/script in various languages/styles to get to the code which reaches out to the system under test.

This is usually good, until you need to remember the outcome of one step in order to use it to verify a later one. At this point, you’ve no way of clearly putting that into the software layers. It ends up somewhere in the Orchestration or World code. It’s hinted at by the Gherkin and glue-code. It’s obscure and it’s caused entirely by the Achilles heel of Cucumber.

Cucumber’s Achilles Heel

To connect your spec with test execution code you have two degrees of separation. Plaintext Gherkin, used at runtime, plus whichever glue-code is kicking about. For tricky cases, this often obscures the intent of the spec/test implementation.

What can we do?

How about we write tests in Java but use the BDD syntax to structure them and report on them? With this in mind there are a few frameworks that offer just this:

Oleaster and Ginkgo4j both try to be equivalents of Jasmine and Mocha. Please see my post for more on these and other BDD frameworks in Java.

I have been working on Spectrum with its founder, Greg Haskins. In the current live release, there’s support for Jasmine/Mocha/RSpec like tests. In the next release (soon) there will be decent support for Gherkin syntax, and some rather neat ways of weaving in your favourite JUnit test frameworks (Spring, Mockito etc) via JUnit Rules.

Have a look.

The Right Test for the Right Job

Success comes not from finding the right tool, but from using the right tool for the job at hand. Where Cucumber succeeds, you should use it. It’s very helpful. Once it gets hard, change tactics.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s