In some version of the next few weeks, I’d adopt Serenity and Cucumber, and use them to write some natural language-based system tests that we can use as a final gate through our deployment process.
There are a few advantages of this:
- It’s a well known framework
- Its reporting is nice
- I’ve done it before
- It’s pretty feature rich
That said, Serenity has a few drawbacks:
- Its configuration is essentially voodoo, not being very well documented
- It runs as a JUnit 4 plugin, and is quite cranky to set up
- I can’t imagine how to run it in parallel
- For my current team there’s a learning curve, which I don’t see as necessary
- It doesn’t fulfil the brief of Selenium best-practice I’ve found in Mastering Selenium, a book I’ve read recently
- Nobody needs a natural language test suite for this use case (though, in fairness Serenity exists without Cucumber)
Why Use Serenity if it’s Not a Fit?
A question to ask is what the smallest possible framework might be. The less framework, the less to learn.
It seems to me that what I most like from Serenity is its dependency injection mechanism. That’s kind of all. Sure, the automagic about selenium is nice, but then it doesn’t comply with my aims to run browsers in parallel.
I’d quite like a JUnit 5 native concurrency model for these tests, which Serenity can’t come near.
So Can We Avoid Writing a Framework?
Yes. I’ve definitely not written a framework. I’ve coded something that’s available for copying/pasting/changing, which I’ll use as the basis of tests on the current project. However, it won’t be released as a new test framework.
However, it’s out there, and it works rather nicely if you want to play with it.
What Does It Do?
It keeps a stock of browser connections in a pool, for sharing across concurrent tests. It provides annotations to configure JUnit 5 to run tests in parallel with some Spring wiring. With a little additional configuration, it’s able to wire in:
- Page objects – objects that model interacting with the browser on a certain page
- Actions – objects that use page objects and test state to execute business processes
- Test state – objects that store values relevant to the current test, without having to pass them around the system
It has a
config.yml file to configure some basic settings (choice of browser for example), using Lightweight Configuration as the “engine” for interpolating environment variables and loading the values into Java.
It manages the lifecycle of the above objects, ensuring that the concurrent tests don’t crash into each other’s states.
Sounds pretty useful? I hope it is.