Why use user story based testing tools like Cucumber instead of other TDD/BDD tools?

Tags:

When you think of writing tests, usually you would write them using a tool from the xUnit family, PHPUnit, JUnit, etc., or if you like a more BDD-style approach, perhaps you would use RSpec, Jasmine, or some other tool like that.

Then there’s Cucumber. Instead of writing your tests purely in code, with Cucumber you start by writing a human-readable user story. Then, you write code to run the story and perform test(s) based on it.

This seems like extra work right? That’s what I thought at first, so I hadn’t really looked into it that much. However, it turns out there might be a bit more to it…

How Cucumber differs from other tools

When working with most testing tools, you would write a test, perhaps like this:

class MyThingTest extends UnitTest {
    public function testSomething() {
        $this->assertEquals(1, 2);
    }
}

When working with Cucumber, you would instead write a user story, step definitions and a “world” where the steps run in.

Stories, or features, generally follow a format like this:

Feature: User login system
    A description of the feature, often written in style of
    "As a <user or role>
    I want to <do something>, 
    In order to <reach some business goal>"
 
    Scenario: Logging in as a user
        Given user "Bob" exists with password "Hello"
         When I log in with name "Bob" and password "Hello"
         Then login is successful

So you write human readable examples like above, and then you write step definitions in code. Step definitions are more like the traditional test functions you write in other testing tools, but they are used to match the scenarios. For example, we could write a step which matches Given user "Bob" exists with password "Hello", so that this step would insert a user with Bob and Hello into the database in preparation for the other steps.

Lastly, in Cucumber you have the world, which is a layer of abstraction for the step definitions. For example, you could have a Selenium world which runs your tests using Selenium, and you could have another world, PhantomJS world, which would run the tests using PhantomJS.

But why would I want to write three files instead of one?

That is pretty much what I thought initially as well!

One of the primary advantages from a practical point of view is this style of writing tests allows for easier reuse of code in the tests. Another advantage is the world system allows you to run your tests in different environments with relative ease. Of course there’s also things like the features can act as a basic form of documentation or such.

Code reuse

Often when working with xUnit style tools, you can end up repeating a lot of things in your tests. Perhaps it’s some initialization code or assertions, but in any case.

One of the nice things about Cucumber is that the separation on the three levels of feature, steps and world makes it much easier to reuse certain parts of the code.

Consider this example again:

    Scenario: Logging in as a user
        Given user "Bob" exists with password "Hello"
         When I log in with name "Bob" and password "Hello"
         Then login is successful

For this, we would write three steps: One for given, one for when and one for then. Now, let’s say we want to write a test which checks that the login fails with a wrong password…

    Scenario: Logging in as a user
        Given user "Bob" exists with password "Hello"
         When I log in with name "Bob" and password "Wrong!"
         Then I get a wrong password error

Note the feature is almost exactly the same as the first one, but we changed the password in the When step to “Wrong!” and changed the “Then” step.

This time we only need to write one step, for the then part – We already have definitions for the two steps before it, and because we can actually use regex matchers to get parameters directly from the feature text, we can write easily reusable steps.

This is in particular very useful if you are writing full stack tests for example with Selenium, where you often have to do a lot of interactions for clicking on links, buttons and whatnot. More about that later.

The world system

The idea with the steps and world is that instead of writing code in your steps to say run the Selenium browser, you instead write that code into the world file and use functions from the world in your steps. This allows you to write the step definitions more concisely, and also on a higher level of abstraction making it easier to grasp at a glance.

Since you wouldn’t deal with the specifics of the test runner in the step definitions, you can just replace the world with some other implementation – Selenium, unit test runner, phantomjs, whatever – and just like that you can test using a different environment.

That sounds great, from now on I’ll write all my tests using Cucumber!

Not so fast!

If you consider the advantages of Cucumber, it can indeed be pretty great. However, it’s not super great for writing lower level unit tests.

Why is this?

At least in my experience, tests which only work with a single class (eg. a unit test) are generally a bit more specific and unique. You would end up writing a lot of low level detailed features which would not really serve any useful purpose to have written down in docs like that, and you would also benefit less from the reusability of the steps since unit tests tend to be much more specific to a certain piece of functionality.

So while you definitely can write unit tests using Cucumber, I would recommend thinking twice before doing that.

Conclusion

Should you use Cucumber?

Maybe.

It’s pretty nice for writing integration tests or tests which you intend to run using something like Selenium. The separation of concerns it has allows you to reduce the amount of code you have to write for your tests, despite what it initially would seem like. You could even share the feature writing work with non-programmers according to some, but honestly I’m not entirely convinced about that.

While Cucumber might not be perfect for writing unit tests, there’s nothing that stops you from writing your full stack tests with Cucumber and your unit tests with your favorite “traditional” testing tool.

In any case, it is definitely worth checking out, you can find more from the following links:

  • Cucumber, Ruby based runner
  • Cucumber.js, pure JavaScript implementation – great for nodejs
  • Behat, a similar tool written in PHP