When working on a software project in a team that includes people with different roles, such as in agile environments, there is always a risk of misalignment in the understanding of end user requirements and what the software should do. The developer may not fully understand them because they may not be clearly formulated by the product owner. The product owner may not realize the complexity of the task being assigned for development and the impact it may have on its delivery. The tester may reason about different edge cases or scenarios that would have been easier to account for at an early stage of the development.
To help improve the development approach through better collaboration between business and developers, behavior-driven development (BDD) was established as a relatively recent software development approach, building on the main ideas of test-driven development (TDD), and using a higher level granularity in the test approach: instead of unit tests for classes and methods, the tests are acceptance tests that validate the behavior of the application. These acceptance tests are derived from concrete examples that are formulated by the team members, so that the behavior of the system is better understood. When these example scenarios are formulated during conversations between the different members, the requirements are likely to be expressed more clearly, the input of the developer will likely be incorporated into them, and the tester will contribute with more scenarios to cover in the tests.
Once these example scenarios are produced, they can be expressed in a format that is easy to read by non-developers, yet follows a certain template that makes it executable by a BDD tool such as Cucumber or JBehave. This format, called the Gherkin syntax, can serve multiple purposes at once:
- The scenarios act as executable specifications for the behavior of the feature under test.
- These specifications can be executed as automated regression tests.
- The scenarios act as documentation about the feature that follows the main code in a version control system.
In Cucumber, which supports several programming languages, such scenarios are written in .feature files that can be added in the project along with the test code. Each file contains scenarios for a specific feature, and each scenario consists of steps, where a step starts for example with Given, When or Then. These steps specify what the scenario is, what assumption(s) it uses, and how the feature will behave in terms of the outcome. In order to execute these steps, we also need the test code (also known as glue code) that will perform whatever action the steps should do. Each step in the feature files will be mapped to a Java method that contains its step definition.
Sample project
As a demonstration, let’s assume we have a simple food ordering application where we want to implement features for adding and removing a meal item from the user’s order. For convenience, let’s create a new project using Cucumber’s Maven archetype support, which should set up the project directory with the minimum code so that we can simply add feature files and step definition classes.
mvn archetype:generate -DarchetypeGroupId=io.cucumber \
-DarchetypeArtifactId=cucumber-archetype -DarchetypeVersion=2.3.1.2 \
-DgroupId=com.example -DartifactId=cucumber-example \
-Dpackage=com.example.cucumber -Dversion=1.0.0-SNAPSHOT \
-DinteractiveMode=false
This should generate a project with a POM file that includes dependencies on the Cucumber artifacts in addition to JUnit, which is itself relied upon to run the tests:
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-java</artifactId>
<version>4.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-junit</artifactId>
<version>4.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
Note: It seems the archetype generates dependency snippets referencing an old version of Cucumber, so in the above dependencies I updated them to the latest retrieved from Maven Central.
The entry point is in the file RunCucumberTest.java, which defines an empty class annotated with @RunWith(Cucumber.class) so that JUnit invokes the custom Cucumber runner, which will automatically scan for feature files and corresponding step definitions and execute them:
@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}
The CucumberOptions annotation specifies the built-in “pretty” formatter plugin for the report containing test results. This annotation can also be used to specify other options.
With the project set up and after importing it into an IDE, we can start adding our features to the food ordering service, which is assumed to already exist in a class FoodOrderingService (let’s imagine the application already existed before adding features to it). The features to be implemented are adding and removing an item from the current order, as shown in the below code (for conciseness, Lombok annotations are used):
@EqualsAndHashCode(of = "name") // items are identified by name
@AllArgsConstructor
public class Item {
@NonNull String name;
@NonNull String category;
}
@Getter
public class Order {
List<Item> items = new ArrayList<>();
BigDecimal price = BigDecimal.ZERO;
}
public class FoodOrderService {
private Order order = new Order();
public Optional<Order> getOrder() {
return Optional.ofNullable(order);
}
public void addItem(Item item) {
// TODO
}
public void removeItem(Item item) {
// TODO
}
}
Before implementing these features, we add corresponding .feature files that contain some scenarios to describe their behaviors. We can treat these as two features: adding an item to an order, and removing an item from an order. Here is a simple feature file for adding an item. For the sake of brevity, the feature file for removing an item is omitted (it can be viewed in the source code linked to at the end of this post).
Feature: Adding an item to order
I want to be able to add an item to a current order.
Scenario: Adding an item to an empty order
Given I have not yet ordered anything
When I go to the "Burgers" category
And I select a "Cheeseburger"
Then I have a new order
And the order has 1 item in it
Scenario Outline: Price of a single item order
Given I have not yet ordered anything
When I go to the "<category>" category
And I select <item>
Then my current order total is <price>
Examples:
| category | item | price |
| Sandwiches | a "Chicken Sandwich" | $9 |
| Dessert | an "Oreo Cheesecake" | $7 |
The file starts with the Feature keyword and a short description of the feature, followed by a more elaborate description that can serve as documentation, and two scenarios for adding an item. The second scenario (called a scenario outline) illustrates how to repeat a certain scenario for different values.
Next we need to add the step definitions for these steps (the lines starting with Given, When, And, Then, etc). We already have a file src/test/java/com/example/cucumber/Stepdefs.java which was generated with the Maven archetype, so we can add our step definitions there:
public class Stepdefs {
FoodOrderService foodOrderService;
String category;
@Given("I have not yet ordered anything")
public void no_order_yet() {
foodOrderService = new FoodOrderService();
}
@When("I go to the {string} category")
public void i_go_to_category(String category) {
this.category = category;
}
@When("I select a/an {string}")
public void i_select_item(String itemName) {
foodOrderService.addItem(new Item(itemName, category));
}
@Then("I have a new order")
public void i_have_new_order() {
assertTrue("Order was null", foodOrderService.getOrder().isPresent());
}
@Then("the order has {int} item(s) in it")
public void order_has_n_item_in_it(int itemCount) {
assertEquals("Wrong number of items in order",
itemCount, foodOrderService.getOrder().get().getItems().size());
}
@Then("my current order total is \\$([\\d\\.]+)")
public void current_order_total_is(String price) {
assertEquals("Wrong order price",
new BigDecimal(price), foodOrderService.getOrder().get().getPrice());
}
}
Note that the @Then annotated methods are typically where we do assertions against expected values.
Mapping steps to their step definitions
The way Cucumber maps each step to its definition is simple: Before a scenario is run, every step definition class will be instantiated and annotated methods (with @Given, @Then, etc) will be mapped to the steps by the expression in the annotation. The expression can be either a regular expression, or a Cucumber expression. In the above step definitions, some methods use Cucumber expressions, e.g. capturing integer parameters using {int}. To use these expressions, an additional dependency needs to be added to the POM:
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-expressions</artifactId>
<version>6.2.0</version>
<scope>test</scope>
</dependency>
Running the tests using mvn test results in the following expected errors:
Tests run: 3, Failures: 1, Errors: 2, Skipped: 0, Time elapsed: 0.561 sec <<< FAILURE!
Adding an item to an empty order(Adding an item to order) Time elapsed: 0.032 sec <<< FAILURE!
java.lang.AssertionError: Order was null
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at com.example.cucumber.Stepdefs.i_have_new_order(Stepdefs.java:30)
at ?.I have a new order(com/example/cucumber/adding_an_item.feature:26)
Price of a single item order(Adding an item to order) Time elapsed: 0 sec <<< ERROR!
java.util.NoSuchElementException: No value present
at java.util.Optional.get(Optional.java:135)
at com.example.cucumber.Stepdefs.current_order_total_is(Stepdefs.java:42)
at ?.my current order total is $9(com/example/cucumber/adding_an_item.feature:33)
Price of a single item order(Adding an item to order) Time elapsed: 0 sec <<< ERROR!
java.util.NoSuchElementException: No value present
at java.util.Optional.get(Optional.java:135)
at com.example.cucumber.Stepdefs.current_order_total_is(Stepdefs.java:42)
at ?.my current order total is $7(com/example/cucumber/adding_an_item.feature:33)
The next step is to implement the features to make the above tests pass. As a starting point, the price information are encapsulated in a BasicItemRepository class, which contains just enough logic code to make the tests successful. Later we can improve it by querying the information from a database, and re-running the tests to make sure that no regression occurred during the improvement. For now, we keep it simple by checking the item name and returning its appropriate price.
public class FoodOrderService {
private final ItemRepository itemRepository;
private Order order;
public FoodOrderService() {
itemRepository = new BasicItemRepository();
}
public Optional<Order> getOrder() {
return Optional.ofNullable(order);
}
public void addItem(Item item) {
if(order == null) {
order = new Order();
}
order.items.add(item);
BigDecimal itemPrice = itemRepository.getItemPrice(item);
order.price = order.price.add(itemPrice);
}
public void removeItem(Item item) {
getOrder().ifPresent(order -> {
order.items.remove(item);
order.price = order.price.subtract(itemRepository.getItemPrice(item));
});
}
}
interface ItemRepository {
BigDecimal getItemPrice(Item item);
}
public class BasicItemRepository implements ItemRepository {
@Override
public BigDecimal getItemPrice(Item item) {
if(item.name.equalsIgnoreCase("Chicken Sandwich")) {
return new BigDecimal(9);
} else if(item.name.equalsIgnoreCase("Oreo Cheesecake")) {
return new BigDecimal(7);
} else if(item.name.equalsIgnoreCase("Cheeseburger")) {
return new BigDecimal(9);
}
throw new IllegalArgumentException("Unknown item " + item.name);
}
}
Running the scenarios again with mvn clean test result in a build success.
Some improvements to the organization of scenarios and step definitions
Background steps
In the previous feature file, the same Given step was used. If at least one Given is shared by all scenarios in the feature, it can be moved to a Background:
Feature: Adding an item to order
I want to be able to add an item to a current order.
Background:
Given I have not yet ordered anything
Scenario: Adding an item to an empty order
When I go to the "Burgers" category
And I select a "Cheeseburger"
Then I have a new order
And the order has 1 item in it
Scenario Outline: Price of a single item order
When I go to the "<category>" category
And I select <item>
Then my current order total is <price>
...
Organizing step definitions and their dependencies
The mapping between steps and the methods containing their definitions does not depend on the class in which the method is defined. As long as Cucumber finds one method with a matching expression, it will run that method. This leaves the decision of where to place step definitions up to the developer. As is the case with the classes of the main code, step definition classes should be organized in a logical way to make their maintenance easier, especially when the number of tests increases.
One of the biggest challenges when writing step definitions is in maintaining the state between dependent steps in a given scenario. As shown in the Stepdefs class, a field category was used to save the parameter passed to the “When I go to the {string} category“. The field was subsequently used in the next step. This is a simple way to maintain state if every feature file has a separate class that encapsulates all of its step definitions.
Sometimes, however, we may want to split step definitions into more than one class for better maintainability. The best way to share state between inter-class step definitions is to use a shared object, and use dependency injection to pass that object to every instance that needs it. The Cucumber project has bindings to several dependency injection frameworks, including Spring and Guice. If the project is already using a DI framework, it’s probably better to use it in the tests. Otherwise, the simplest one to use is PicoContainer.
To carry out this state management between several classes, let’s assume that we want to split the Stepdefs class into two classes: ItemStepdefs and OrderStepdefs. The first class fills the object with state, and the second uses that state in the steps that need it. This may not normally make sense for this feature. For this example, let’s use the Spring solution; the PicoContainer one is straightforward and does not require any configuration or annotations. First we add the required dependencies. We need both the Cucumber binding and Spring dependencies because our sample project did not initially use Spring:
<dependency>
<groupId>io.cucumber</groupId>
<artifactId>cucumber-spring</artifactId>
<version>4.2.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>5.1.3.RELEASE</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>5.1.3.RELEASE</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>5.1.3.RELEASE</version>
<scope>test</scope>
</dependency>
Note also the dependency on spring-test.
First we create a class that contains the state to be shared between the step definitions, and annotate it with @Component:
@Component
public class ItemOrderInfo {
String category;
FoodOrderService foodOrderService;
}
We also need a configuration class for Spring. We assume that the above <codeComponent class is in the same package of this configuration class:
@Configuration
@ComponentScan
public class SpringTestConfig {
}
Next we annotate one of the two step definition classes with @ContextConfiguration from the spring-test, pointing to the test configuration class that was just created. At this point we can use Spring’s dependency injection mechanism to provide a singleton instance of ItemOrderInfo, the class containing state:
@ContextConfiguration(classes = SpringTestConfig.class)
public class ItemStepdefs {
@Autowired
ItemOrderInfo itemInfo;
@Given("I have not yet ordered anything")
public void no_order_yet() {
itemInfo.foodOrderService = new FoodOrderService();
}
@When("I go to the {string} category")
public void i_go_to_category(String category) {
this.itemInfo.category = category;
}
}
We can use the same object in the other step definition class:
public class OrderStepdefs {
@Autowired
ItemOrderInfo itemInfo;
@When("I select a/an {string}")
public void i_select_item(String itemName) {
itemInfo.foodOrderService.addItem(new Item(itemName, itemInfo.category));
}
@Then("I have a new order")
public void i_have_new_order() {
assertTrue("Order was null", itemInfo.foodOrderService.getOrder().isPresent());
}
...
}
Hooks
There are some annotations that can be used to hook into the lifecycle of the scenario. For example, to prepare something before every scenario, we can add it in a @Before annotated method (this is different than the org.junit.Before annotation provided by JUnit):
@Before
public void prepare(){
// Set up something before each scenario
}
Normally this is where things like initializing a resource or preparing a test database can be done.
On the other hand, the @After annotation allows executing code after each scenario. There are also @BeforeStep and @AfterStep annotations.
Filtering scenarios using tags
In some cases we want to run only a subset of scenarios. A handy feature called tags allows labeling specific features or scenarios such that we can reference them when running the tests. The feature file we have so far can be enriched with tags as follows:
@addItem
Feature: Adding an item to order
I want to be able to add an item to a current order.
@empty
Scenario: Adding an item to an empty order
Given I have not yet ordered anything
When I go to the "Burgers" category
And I select a "Cheeseburger"
Then I have a new order
And the order has 1 item in it
@price
Scenario Outline: Price of a single item order
Given I have not yet ordered anything
When I go to the "<category>" category
And I select <item>
...
To run only scenarios tagged with @price, we can pass the tag in the cucumber.options system property:
mvn clean test -Dcucumber.options='--tags "@price"'
The hook annotations (@Before and @After) shown earlier can also take tag expressions to restrict their execution.
Conclusion
The above sample project illustrates a simple workflow that follows behavior-driven development practices: deriving scenarios about our features, formulating them in a natural language syntax, and using them to drive the implementation. The source code can be found here.
Further resources
https://dannorth.net/introducing-bdd/
https://docs.cucumber.io/cucumber/
https://github.com/cucumber/cucumber-jvm/
Learning Quest 