How to integrate Cypress and Cucumber in your development flow in just a few weeks.

At INIT Group we started using Cypress and Cucumber in one of our team’s project about a couple months ago, and we believe we somehow mastered the process. And we love it. There are quite a few incremental learning steps to integrate it into the development flow. But it really is quite straight forward once you know them. I believe any team can start working like this in 2–4 weeks. That’s why I’m writing this post.

cypress loves cucumber

Here you can find a git skeleton repo of all you need to start, with an explanation of all these incremental learning steps. It’s tested on MacOS and Linux. We think some parts might not work with Windows.

This is the result of our learnings during these last months. It’s probably not perfect, and we probably will change this repo as new learnings occur. Please, do not hesitate to give any kind of feedback.

Also, take into account that this is not a Cypress nor a Cucumber tutorial. If you don’t know any of them, I would suggest you to do some Cypress tests directly with mocha first, while you check these links:

• https://docs.cypress.io/examples/examples/tutorials.html
• https://docs.cucumber.io/gherkin/reference/

Ready? Then let me tell you how we use Cypress and Cucumber together so easy, that anyone can do it.

How to install the skeleton / example repository

I created a skeleton repository to let you start as fast as possible, and to understand all the mechanics involved. Clone it and play with it to know what this is all about.

git clone git@github.com:jmarti-theinit/cypress-cucumber-example.git
git submodule init
git submodule update
npm install

Once you install all dependencies, go ahead and execute the tests:

npm run test:prod

You should see all 3 specs passed. Well done, let’s see how this works!

All 3 specs passed!

Where do we write features: using an external “gherkin-features” repository

When we started the project, we needed two things to happen:

• We wanted features to be available not only for end to end tests, but also in unit tests and integration tests in other repositories, so features could not be attached to just one repository.
• We wanted business people and Product Owner to read them all by themselves, trying not to mix the Gherkin specs with some javascript fuzz code.

That’s why we decided to use an external repository for features. In the skeleton repository you just downloaded, the features repo is (as an example) git@github.com:jmarti-theinit/cypress-cucumber-examples-features.git

The way we synchronize features in this repo to the features in Cypress + Cucumber project is as follows:

• The repo is included as a git submodule in a “gherkin-features” folder. That’s why you need to “git submodule init” at the beginning.
• The features inside this folder are synchronized into cypress/integration folder with a npm run test:pull-features command, which: (1) pulls and updates the gherkin-features submodule, (2) copies new features, (3) overwrites any feature modified in cypress/integration and (4) removes any feature in cypress/integration that does not exist in gherkin-features. Check the “test:pull-features” script in package.json to see how all that is done.

The counterpart of this is that any change to features must be done in the original git repo, not in gherkin-features or cypress/integration. This can be at some point a little bit tedious, but the benefits are higher.

Every time we want to create a new test, we just need to npm run test:pull-feature before.

This works with GIT submodules. If you use another Control Version System, you should check how to make it work.

If you see any other better way to do this, please give us some feedback.

The gherkin-features folder (which is a git submodule of the original repo) and the cypress/integation folder.

If you want to use your own gherkin-features repo, just remove the current one and add yours under the “gherkin-features” folder like this:

git submodule deinit gherkin-features
git submodule add (YOUR_REPO_URL) gherkin-features
git add --all
git commit -m "Change repo url"

If you want to know how GIT submodules work, check this link. I think it’s very well explained: https://git-scm.com/book/en/v2/Git-Tools-Submodules

Take into account that any execution of npm run test:pull-features should be added, commited and pushed into your repository, as the submodule needs to reflect all the time where is pointing to its own git repository.

How specs work with Cucumber and Cypress

This repo uses cypress-cucumber-processor, which is a plugin for Cypress. Installation is really easy (already done in the repo you just downloaded):

• In package.json you can find the dependency.

"cypress-cucumber-preprocessor": "^1.9.1",

• In cypress/plugins/index.js you can find the wiring with Cypress as a file preprocessor.

module.exports = (on, config) => {
  on('file:preprocessor', cucumber());
  ...
};

• You can also tell Cypress not to show javascript files, and to show only .feature files through cypress.json.

{
...
"ignoreTestFiles": ["*.js", "*.md"],
...
}

The way to organize features and javascript code is quite easy:

feature files and javascript files

• All your tests must go in the cypress/integration folder.
• At the same level as the feature in that folder, create a folder with the same name of the feature. For example, if you have “feel-lucky.feature”, create a folder “feel-lucky”.
• Inside that folder, create as many javascript files as scenarios your feature has.

The way to write code for the Cucumber steps in javascript is also really easy.

import { Given, When, Then } from 'cypress-cucumber-preprocessor/steps';
Given(/^One given step$/, () => {
   ...
});
Given(/^another given step$/, () => {
   ...
});
When(/^Some other when step$/, () => {
  ...
});

Then(/^I have some results$/, () => {
  ...
});

In the javascript file of the scenario you can add as many Given, When and Then steps as you need. They should match the Given, When, Then (and And) steps that you have in your corresponding .feature file.

Using Page Object Pattern

I really like the Page Object Pattern, which you can read more about it here. This pattern allows maintainability, as it proposed to encapsulate all HTML stuff (css selectors, clicks, etc) in a Page object. This way, when you do several scenarios related to the same page, all the HTML is in one specific file; and if these tests break because of an HTML change, then you can go to that Page, change it and fix several tests at once.

This way, I get the tests to express things like this, which is quite more verbose:

Given(/^I'm at Google$/, () => {
  GoogleSearchPage.visit();
});

When(/^I type search word 'github'$/, () => {
  GoogleSearchPage.type('github');
});
When(/^Press 'Search'$/, () => {
  GoogleSearchPage.pressSearch();
});

Then(/^I have some results$/, () => {
  GoogleResultsPage.expect().toHaveResults();
});

And in your page, you have this code. As you can see we usually expose all the HTML related code through constants in the beginning to be able to locate it easier.

const SEARCH_FIELD = 'input[type=text]';
const SEARCH_BUTTON = 'input[type=submit]';
const SEARCH_TEXT = 'Buscar';

class GoogleSearchPage {
  static visit() {
    cy.visit('/');
  }
  static type(query) {
    cy.get(SEARCH_FIELD) // 2 seconds
      .type(query);
  }
  static pressSearch() {
    cy.get(SEARCH_BUTTON).contains(SEARCH_TEXT)
      .click();
    return new GoogleResultsPage();
  }
}

You may be worried about the strong dependencies of these pages with the global “cy” variable, and the usage of static methods in these classes. We found this, among all the different combinations, the one that satisfied all of us the most.

Using common steps of Cucumber

One thing that you start finding out, and where the power of Cucumber emerges, is that lots of your steps in your feature files start repeating. Mainly those in the Given part, and in the Background part.

Cucumber allows you to use “common” steps. With Cypress and Cucumber preprocessor, you need to put them in the cypress/integration/common folder. I don’t like that structure much, but is the one that you can do right now.

common folder inside cypress/integration

You can add as many javascript files as you want to inside the cypress/integration/common folder, and as many steps inside each one of those js files as you want.

We generally try to use a visually hierarchical name of files. For example, “google.js” for those steps common to the “google” folder. “google-search.js”, for those steps common to the “google/search.js”, etc.

From our perspective, it would be better if this common directory could be added into the cypress/integration/google directory, in the example above. But that’s not achievable right now (or we didn’t achieve it).

Server fixtures: Setting server in an known state for tests

When using tests, you generally need to set the server database in a known state. This usually means executing some database operations.

In our case, we have set up this framework:

• We created a “e2e database commands” module in our backend. The responsability of this module is to expose an API to execute database commands for these end to end tests.
• A database command is a set of database operations. A database command is called through a string name. For example, “set-test-user-height-to-175” is equal to “update user set height=175 where id=1”.
• We have in the database some predefined fixtures that you can count on. In this example, the user. It’s all we have.
• We keep the commands in the database, and we can call them through an API exposed. This way, from our node code in cypress, we do executeCommand('set-test-user-height-to-175') and we know that the user will have this height for the tests.
• We create new commands as our tests need it.

We are not worried about these commands to be exposed through an API, even on production, as they only do operations for test users. So they do not mess up the database at all.

From Cypress and Cucumber point of view, the only problem that we needed to solve is that we needed to call to this commands (or requests) while setting tests data up. And, as they are asynchronous operations, we need to make sure that these operations are done synchronously before proceeding tests.

To do that, cypress has the powerful “cy.task”. It makes sure that all you do is inside the Cypress chaining mechanism. In the repo you just downloaded, you can check how we call an asynchronous task that takes 5 seconds, and how Cypress waits for it before proceeding.

• In “cypress/common/execute-command.js” there is this “executeCommand” function that call the task:

const executeCommand = (command) => {
  cy.task('pluginExecuteCommand', command);
};

• In cypress/plugins/index.js, we add that task to our Cypress configuration:

module.exports = (on, config) => {
  ...
  on('task', {
    pluginExecuteCommand,
  });
  ...
};

• And you can see the “pluginExecuteCommand” in cypress/plugins/plugin-execute-command.js. Make sure this always returns a promise, to make sure that Cypress waits for it. In our example, we do an asynchronous sleep for 5 seconds. This should be replaced with your call to your API from e2e database job (you can use your own request framework with node, for example, axios; or cy.request commands).

const pluginExecuteCommand = command =>
  new Promise((resolve, reject) => {
    setTimeout(() => {
      resolve(`${command} execution simulated after 5 secs`);
    }, 5000);
  });

This is what will probably take most time for you to start with Cypress and Cucumber. Do it incrementally, start exposing an API to execute a command, and you will add richer functionalities in the future (parametrized queries, scaffolding of these database commands, etc).

Launching tests against different environments

Another common situation is to be able to execute tests against different environments: localhost while developing, integration environment for each code integration and even production to make sure everything works.

To be able to handle this, Cypress offers a great functionality:

• By default, it uses cypress.json variables in your root folder.
• But cypress/plugins/index.js can overwrite the variables that you want, returning an object with the ones that you want to overwrite.

In our case, through npm scripts we set a CYPRESS_ENV environment when calling the tests (defined by us, it’s not related to CYPRESS).

Then, we return it in cypress/plugins/index.js:

module.exports = (on, config) => {
  ..
  // `config` is the resolved Cypress config
  return cypressConfigResolver();
};

This cypressConfigResolver function is defined in cypress/config/cypress-config-resolver.js file. Which reads the xxxxxx.json in cypress/config directory, according to the CYPRESS_ENV file (or “localhost” if not defined).

cypress/config/cypress-config-resolver.js file

This cypressConfigResolver is also used in the cy.tasks that call the e2e database jobs (like the one in pluginExecuteCommand); as you need to make sure that fixtures and data is set only in the servers that you are doing the tests against.

This way, we can open cypress like (generally when developing):

npm run cypress:open:local
npm run cypress:open:prod

Or we can launch headless tests like (generally to make sure we didn’t break any test):

npm run test:local
npm run test:prod

Or we can launch tests with a Chrome like (to debug):

npm run test:debug:local
npm run test:debug:prod

You can check it is working in the current repository, because when you execute npm run cypress:open:pro, you will see Cypress opening with “www.google.com” and the API of executeCommand working against “https://some-server-production.com/e2e-api/”, as it is defined cypress/config/production.json.

Cypress baseUrl is “www.google.com” and the pluginExecuteCommand outputs “db-command-long-task executiom simulated after 5 secs, thrown to https://some-server-production.com/e2e-api/”, which is the e2eDatabaseJobApi property of production.json.

Which is different that is we execute npm run cypress:open:local , in which you see the following, based on cypress/config/localhost.json environments.

Cypress baseUrl is “www.google.es” and the pluginExecuteCommand outputs “db-command-long-task executiom simulated after 5 secs, thrown to https://some-server-localhost.com/e2e-api/”, which is the e2eDatabaseJobApi property of localhost.json.

Integration with Jenkins

To support our Continuous Integration workflow, we use Jenkins as the automation server.

To be able to anyone launch the tests and to be able to automatically launch them, we created a pipeline. The pipeline definition is included in the repository in the file build/Jenkinsfile.

Three things are remarkable in that pipeline configuration:

• Only tests marked with @e2e-test are executed. This is done because we need to exclude tests that have been written for future user stories, and are not automated yet; so that the tests that are not automated don’t throw errors. To be able to do that, we tell Cypress to execute only the tests marked as “@e2e-tags”. That is done through the npm run test script in package.json.

{
...
"test": "cypress run --env TAGS='@e2e-test' --spec 'cypress/integration/**/*.feature'",
...
}

• We also want test reports to be able to be read easily after execution. To do that, we use a reporter. As Cypress uses mocha internally, it is enough to tell Cypress to use the “junit” reporter. And we tell Jenkins, through the pipeline, to use these reports.

In cypress.json:

{
  ... 
  "reporter": "junit",
  "reporterOptions": {
    "mochaFile": "test-results/test-output-[hash].xml"
  },
  ...
}

In Jenkinsfile:

{ 
...
  post {
    always {
      junit keepLongStdio: true, testResults: 'test-results/*.xml', allowEmptyResults: true
    }
  }
...
}

• We also want to export the Cypress videos, to be able to download them easily if anything happens. To do that, we expose those artifacts through Jenkinsfile.

post {
  always {
    ...
    archiveArtifacts artifacts: 'cypress/videos/**/*.mp4', onlyIfSuccessful: false
    ...
  }
}

Integration with IntelliJ IDEA

The best way we found to integrate Cypress and Cucumber with IntelliJ IDEA is to install the “Cucumber.js” plugin.

But we find quite some disturbing lacks:

• We cannot create steps from the IDE, as it does not recognize the structure of the tests, and it is not configurable.
• When we start the IDE, it does not recognize the features immediately. We need to do some change in the feature file and the js file to find it (it probably indexes steps only when files are changed, but not when the IDE is loaded).

When the second point works as it should, it is really useful to click on the steps from the feature and jump into the code that implements it.

It’s a pity the lack of support for this.

How to make sure all developers and product owners jump into this new development workflow

We use Cypress + Cucumber as an Acceptance Testing tool, and it’s going quite well. This means that in our workflow, all user stories have specifications, and that all features that are implemented have their acceptance testing automated.

Three things we need to have in mind, and make sure that all the team members know and remember this:

• Specs are not meant to be unidirectionally written and implemented. They are conversation examples that are explicitly written down in the user stories. When user stories are complex, we can do an Example Mapping session to clear doubts out.
• All stories should have, at least, a specification file before being developed (pointing to the gherkin-features repo). If they do not have it, anyone can add a new spec, not only the product owner.
• We need to make sure that the acceptance test is automated. In order to achieve this, we have created a new “Test Automated” column in our Kanban / JIRA / Sprint board. We are not proud of this, as we should already know that “Done” means “with tests automated”. But as this practice is something emerging and new in the team, we decided to explicitly declare it through a column. Maybe, in the future, this column disappears.

Continuos Delivery and Rollback

If you read this article just to know how we automatically launch the tests against our server, and rollback in case it fails, I have bad news for you: we are still missing this feature.

We still launch the tests manually in development servers, we manually do a release to production in case everything works ok, and we manually launch the tests into production servers to make sure nothing is broken.

Conclusions and how I would recommend you to start

In these first two months, we achieved something that we were wishing from a lot of time: (1) to automate our acceptance tests with a language that even business people could understand, and (2) have more confidence with the released versions through these tests.

For us it’s been quite an achievement, and after reflecting all the learnings that we have done, we really believe that is achievable for anyone now that Cypress and Cucumber technologies have emerged and they have such an easy entry barrier. They really do, trust me.

If you want you and your team to achieve this in 2–4 weeks, I would suggest you to do the following:

• First of all, have a conversation inside your team about why you need this. Make sure you all believe that this is needed and that it can be done. Make sure everybody understands that this will slow you down a bit at least a couple of months in the beginning. Yes, you will do some new user stories, but you will do them slower, in the beginning.
• I would recommend one person (the Product Owner? QA people?) to start creating a couple of specs for your next user stories, just to practice Gherkin. This should be the “Person of Reference about Gherkin”. Start with the easiest ones. This will allow this “Person of Reference” to help developers also write some specs later.
• Create a separate repository for the e2e-tests. Clone my repository and use it as a skeleton. Remove my gherkin-features repo and use yours. If in the beginning, the “gherkin-features” external repository adds unneeded complexity, forget about it, and add the features in an ordinary gherkin-features folder, or even inside cypress/integration folder.
• Have a session in your team to make sure everybody understands the basics behind this repository, and how Cypress works.
• Develop some basic tests with their specs, based on past features, not future ones. Make sure every developer of the team has created and developed a spec (with the help of this Person of Reference).
• During this moment, make sure that the Product Owner has quite a few future user stories with specs.
• Once every developer has developed at least a spec, then add the new “Test Automated” column in your board.
• This way you should start developing the user stories that have specs.
• From this point, make sure no future user story is without a spec file, and make sure that all user stories developed go through the “Test Automated”.
• Add a Job about this to the Integration Server, to make sure everybody visualizes the increasing “cobertura” of the tests (based on the number of tests executed). Execute it and try it to automate it.

Some things to keep in mind:

• If you have different frontends, I would suggest only to focus on one (the most common one). And repeat all this on the second one after you master this first frontend.
• It doesn’t matter if you implement the tests before or after. You might do BDD while developing, or you might not. Just make sure all tests are automated when passing through the “Test Automated” column.
• It has been very common for us that spec files are not perfect, or they need to be changed while developing. Everybody should be empowered to change any spec as needed. If a spec is wrong, there is no excuse to stop the automation of a test. Actually, it will probably be wrong, but at least, it will reflect the need behind the user story.

This is more or less the steps that worked for us, in our context and in our team. And we are quite happy with the results. Do not loose the opportunity to try this in your team!

If you followed this steps, or you did others, please, don’t hesitate to give some feedback. We are here to learn!

Thanks!