I have a love-hate relationship with Gherkin, the syntax underpinning Cucumber tests. On the one hand I love the readability, on the other I find the binding of the sentences to step definitions flaky, and the lack of formal structure in those sentences worries me. Like the Maven vs Gradle debate, I’m a fan of structure; it keeps stuff from derailing quickly. (Maven FTW.)

So when I decided to give Cucumber another go for one of my hobby projects, I attempted to get some structure in (more on that later on). But the result was that I wrote this:

    public void test() {
        Scenario.of("Modify Vacation Hours")
                .given(RosterPeriod.on("2022-09-19").exists())
                .and(User.of("peter").isLoggedin())
                .when(Overview.isAccessed())
                .and(VacationHours.forUser("peter").onDate("2022-09-19").isSetTo(20))
                .then(VacationHours.forUser("peter").onDate("2022-09-19").shouldBe(20))
                .and(WeekTotals.forUser("peter").inRosterPeriod("2022-09-19").shouldBe(20,0,0,0,0,0))
                .and(RunningWeekTotals.forUser("peter").inRosterPeriod("2022-09-19").shouldBe(20,0,0,0,0,0))
                .and(Event.who("peter").what("SetVacationHours").user("peter").rosterPeriod("2022-09-19").detailSubstring("hours=20").shouldExist());
    }

Which is the Java equivalent of this Cucumber test:

      Scenario: Modify Vacation Hours
        Given a rosterperiod 2022-09-19 exists
        And user peter is logged in
        When the overview is accessed
        And the vacation hours for peter on 2022-09-19 is set to 20
        Then the vacation hours for peter on 2022-09-19 should be 20
        And the week totals for peter in rosterperiod 2022-09-19 should be 20,0,0,0,0,0
        And the running week totals for peter in rosterperiod 2022-09-19 should be 20,20,20,20,20,20
        And an event with who "peter", what "SetVacationHours", user "peter", rosterdate 2022-09-19 and detail substring "hours=20" should exist

And I started wondering if the Java version is readable enough. Because it damn well fixes all my issues I have with Cucumber.

What do you think?

Let me explain how I got here. First off: Cucumber test should not include any technology specific information, aka functional testing or BDD. So my original goal of the exercise was to write Cucumber tests that could test both the UI and API with the same test, by simply switching the packages that implement the step definitions (this seems quite feasible). But while doing so, I got annoyed by the lack of structure for the feature files.

One of the main issues is how to differentiate between the Given and Then steps; both are stating facts, but the first needs to make the fact a reality, while the latter needs to verify (assert) that the fact is true. From a syntax perspective these statements can very well be identical. And given a clear structure in the sentences, these probably end up being identical:

Given the admin is logged in
Then the admin is logged in

Cucumber can’t deal with this duality, because it only does simple regexp pattern matching, and the context in which a step is used is not available in the pattern nor implementation of the step, so the syntax needs to differentiate between the two contexts. BTW, the When is always an action, which already differentiates it from the other two by the used verb.

One of the approaches to differentiate often found on the internet is to use time.

• Given: in the past
• When: in the present
• Then: in the future

For example:

Given the admin was logged in
When the admin logs in
Then the admin will be logged in

For solving the differentiation problem this is a great pattern, but it somehow reads wrong. This is a personal taste thing, but I’m not too happy with it; I prefer writing everything in present tense.

Another approach I thought of was to simply state the fact that Then is asserting.

• Given: fact
• When: action
• Then: “assert” fact

For example:

Given the admin is logged in
When the admin logs in
Then assert the admin is logged in

Again a fine pattern for differentiating, but it feels contrived. Then already contains the intent of asserting, adding that word explicitly is double. But it’ll work.

The third option was to introduce uncertainty in the Then:

• Given: absolute fact
• When: action
• Then: uncertain fact

For example:

Given the admin is logged in
When the admin logs in
Then the admin should be logged in

None of the three options is a slam dunk in my opinion, but I prefer the last one. It is closest to a natural reading language, and it is a nice and simple pattern to follow.

The next step was to see if it is possible to add some structure to the sentences, and this pattern merged:

<topic> [parameters] <verb> [values]

If we look at the tests at the top, topics are rosterperiod, user, overview, week totals, etc. Those are followed by parameters to specify what topic, like a date, or a name. And finally the verb, this is the defining element; the verb defines if a sentence is a Given, When or Then, and can possibly have values, to set or assert.

Once I had this structure in place, that last sentence started to haunt me; “the verb defines if a sentence is a Given, When or Then”. So I made Java classes to construct Given, When and Then interface implementations using a fluent API, aka step definitions, where the method that represents the verb returns the respective implementation. And then all that was needed was implementation of a Scenario, also using a fluent API, to make that into a Cucumber alike test. And behold: the code shown above and here emerged:

    public void test() {
        Scenario.of("Modify Vacation Hours")
                .given(RosterPeriod.on("2022-09-19").exists())
                .and(User.of("peter").isLoggedin())
                .when(Overview.isAccessed())
                .and(VacationHours.forUser("peter").onDate("2022-09-19").isSetTo(20))
                .then(VacationHours.forUser("peter").onDate("2022-09-19").shouldBe(20))
                .and(WeekTotals.forUser("peter").inRosterPeriod("2022-09-19").shouldBe(20,0,0,0,0,0))
                .and(RunningWeekTotals.forUser("peter").inRosterPeriod("2022-09-19").shouldBe(20,0,0,0,0,0))
                .and(Event.who("peter").what("SetVacationHours").user("peter").rosterPeriod("2022-09-19").detailSubstring("hours=20").shouldExist());
    }

I have never seen (unit) tests written like that, it resembles Spock somewhat, maybe. Did I reinvent a wheel? Someone must of thought of this before. Anyhow, I liked it, so made it into a small open source project ‘GiWTh’ (https://github.com/tbee/giwth).