Cypress Fail Fast

Enables fail fast in Cypress, skipping the rest of tests on first failure.

It can be configured to skip all remaining tests in current spec file, in current run, or even in parallel runs.

Table of Contents

Installation

Add the plugin to devDependencies

npm i --save-dev cypress-fail-fast

Inside cypress/plugins/index.js:

module.exports = (on, config) => {
  require("cypress-fail-fast/plugin")(on, config);
  return config;
};

At the top of cypress/support/index.js:

import "cypress-fail-fast";

From now, if one test fail after its last retry, the rest of tests will be skipped:

Configuration

Environment variables

• FAIL_FAST_STRATEGY: 'spec'|'run'|'parallel'
  • If spec, only remaining tests in current spec file are skipped.
  • If run, all remaining tests in all spec files are skipped (default value).
  • Use parallel to provide your own callbacks allowing to notify from one run to the others when remaining tests should be skipped.
• FAIL_FAST_ENABLED: boolean = true Allows disabling the "fail-fast" feature globally, but it could be still enabled for specific tests or describes using configuration by test.
• FAIL_FAST_PLUGIN: boolean = true If false, it disables the "fail-fast" feature totally, ignoring even plugin configurations by test.

Examples

CYPRESS_FAIL_FAST_PLUGIN=false npm run cypress

or set the "env" key in the cypress.json configuration file:

{
  "env":
  {
    "FAIL_FAST_STRATEGY": "run",
    "FAIL_FAST_ENABLED": true
  }
}

Configuration by test

If you want to configure the plugin on a specific test, you can set this by using the failFast property in test configuration. The plugin allows next config values:

• failFast: Configuration for the plugin, containing any of next properties:
  • enabled : Indicates wheter a failure of the current test or children tests (if configuration is applied to a suite) should produce to skip the rest of tests or not. Note that the value defined in this property has priority over the value of the environment variable CYPRESS_FAIL_FAST_ENABLED (but not over CYPRESS_FAIL_FAST_PLUGIN, which disables the plugin totally).

Example

In the next example, tests are configured to "fail-fast" only in case the test with the "sanity test" description fails. If any of the other tests fails, "fail-fast" will not be applied.

describe("All tests", {
  failFast: {
    enabled: false, // Children tests and describes will inherit this configuration
  },
}, () => {
  it("sanity test", {
    failFast: {
      enabled: true, // Overwrite configuration defined in parents
    },
  }, () => {
    // Will skip the rest of tests if this one fails
    expect(true).to.be.true;
  });

  it("second test",() => {
    // Will continue executing tests if this one fails
    expect(true).to.be.true;
  });
});

Configuration examples for usual scenarios

You want to disable "fail-fast" in all specs except one:

Set the FAIL_FAST_ENABLED key in the cypress.json configuration file:

{
  "env":
  {
    "FAIL_FAST_ENABLED": false
  }
}

Enable "fail-fast" in those specs you want using configurations by test:

describe("All tests", { failFast: { enabled: true } }, () => {
  // If any test in this describe fails, the rest of tests and specs will be skipped
});

You want to totally disable "fail-fast" in your local environment:

Set the FAIL_FAST_PLUGIN key in your local cypress.env.json configuration file:

{
  "env":
  {
    "FAIL_FAST_PLUGIN": false
  }
}

Configuration for parallel runs

The plugin configuration supports defining two callbacks that, used in combination, allow to skip tests in one run when other run starts skipping tests also. Where, or how do you store the "flag" that allows to communicate your runs is in your hands, the plugin does not care about it.

To implement it, the plugin can receive an object with extra configuration as third argument when it is registered in the cypress/plugins/index.js file:

• parallelCallbacks: Object containing next properties:
  • onCancel: function() This callback is executed on first test failure that produces the plugin starts skipping tests.
  • isCancelled: function(): boolean If this callback returns true, the plugin skips remaining tests.

These callbacks are executed only when the environment variable FAIL_FAST_STRATEGY is set to parallel.

Here is an example of configuration that would skip tests on many parallel runs when one of them starts skipping tests. It would only work if all parallel runs have access to the folder where the isCancelled flag is being stored as a file (easy to achieve if all of your parallel runs are being executed on Docker images on a same machine, for example). Note that this is only an example, you could also implement it storing the flag in a REST API, etc.

const fs = require("fs");
const path = require("path");

// Flag file is stored in the /cypress folder
const isCancelledFlagFile = path.resolve(__dirname, "..", ".run-is-cancelled");

module.exports = (on, config) => {
  require("cypress-fail-fast/plugin")(on, config, {
    parallelCallbacks: {
      onCancel: () => {
        // Create flag file when the plugin starts skipping tests
        fs.writeFileSync(isCancelledFlagFile);
      },
      isCancelled: () => {
        // If any other run has created the file, start skipping tests
        return fs.existsSync(isCancelledFlagFile);
      },
    },
  });

  return config;
};

Note that this example requires to remove the created file when all of the runs have finished, or tests will always be skipped whenever any run starts again. So, the FAIL_FAST_STRATEGY environment variable should be set to parallel only in CI pipelines where the workspace is cleaned on finish, for example.

Usage with TypeScript

If you are using TypeScript in the Cypress plugins file, this plugin includes TypeScript declarations and can be imported like the following:

import cypressFailFast = require("cypress-fail-fast/plugin");

export default (on: Cypress.PluginEvents, config: Cypress.PluginConfigOptions): Cypress.PluginConfigOptions => {
  cypressFailFast(on, config);
  return config;
};

Tests

To ensure the plugin stability, it is being tested with Cypress major versions 5.x, 6.x and 7.x, and new releases will be published for each new Cypress minor or major releases, updating the package E2E tests.

Latest versions used in the E2E tests can be checked in the devDependencies of the package.json files of the E2E tests:

Anyway, if you find any issue for a specific Cypress version, please report it at https://github.com/javierbrea/cypress-fail-fast/issues.

Acknowledgements

This plugin has been developed based on the solutions proposed by the community on this Cypress issue, so thanks to all! I hope this plugin can be deprecated soon, as soon as the Cypress team adds native support for this feature.

Contributing

Contributors are welcome. Please read the contributing guidelines and code of conduct.

License

MIT, see LICENSE for details.