Codeception 5

Codeception 5.0 is out!

This release is PHP 8+ only, so we are back on track with modern PHP. We are dropping support for PHPUnit < 9, and are technically ready for PHPUnit 10. And we also support Symfony 6 without dropping support of previous Symfony versions. As always, we did our best to keep backward compatibility so if you can update your dependencies, all tests should be working for you.

So let’s take a look at new features:

New Directory Structure

Codeception 5 will match PSR-12 standard. So all tests and classes will have their own namespace Tests. The directory structure was updated accordingly:

tests/
  _output
  Acceptance
  Functional
  Support/
    Data/
    _generated/
    Helper/
  Unit/

All suite name will have their own namespace, as well as actor and helper classes:

<?php

namespace Tests\Acceptance;

use \Tests\Support\AcceptanceTester;

class LoginCest
{
    public function tryToTest(AcceptanceTester $I)
    {
        $I->amOnPage('/');
    }
}

New directory structure will be generated by running codecept bootstrap. The directory structure is set with a new default config, so the previous directory structure is still valid.

Attributes

Annotations were an essential part of Codeception testing framework. Even though they were not native language constructs, they proved to be quite good to separate a test from its metadata. We believe that test should not include code that doesn’t belong to the test scenario.

So we were glad that native Attributes have landed PHP world. In this release we encourage our users to start using them:

#[Group('important')]
#[Group('api')]
#[Examples('GET', '/users')]
#[Examples('GET', '/posts')]
#[Env('staging-alpha')]
#[Env('staging-beta')]
#[Env('production')]
#[Prepare('startServices')]
public function testApiRequests(ApiTester $I, Example $e)
{
  $I->send($e[0], $e[1]);
  $I->seeResponseCodeIsSuccessful();
  $I->seeResponseIsJson();
}

As you see, attributes decouple all preparation steps, keeping the test scenario minimal. We also keep supporting annotations, so an urgent upgrade is not needed. Attributes can’t do something that traditional annotations can’t, they are just a modern alternative.

List of available attributes (all under Codeception\Attribute) namespace:

• Before - specifies the method that should be executed before each test
• After - specifies the method that should be executed after each test
• Group - set the group for the test
• Skip - skips the current test
• Incomplete - marks test as incomplete
• Depends - sets the test that must be executed before the current one
• Prepare - sets a method to execute to initialize the environment (launch server, browser, etc)
• DataProvider - specifies a method that provides data for data-driven tests
• Examples - sets data for data-driven tests inside the annotation
• Env - sets environment value for the current test
• Given, When, Then - marks a method as BDD step

Debugging

Do you remember, Hoa\Console? Unfortunately, this library was deprecated and we were looking for a modern alternative that could power codecept console and $I->pause(); commands. We switched to PsySH a PHP REPL.

An interactive console is used to pause a test in the given state. While in pause you can try different Codeception commands, and check variable values. Instead of fixing tests blindly, you can start an interactive session. This is quite a similar effect you can get with a real debugger like XDebug but focused on Codeception commands. Especially this is helpful to write acceptance tests as the test scenario can be planned while executing a test. So basic scenario can be written as:

$I->amOnPage('/');
$I->pause();

After opening a page you will be able to try commands in a browser. If a command succeeds you can use it in your tests.

Also new functions were added:

• codecept_pause() - starts interactive pause anywhere in debug mode
• codecept_debug() - prints a variable into console using Symfony VarDumper

Sharding

Parallel Execution guide has been rewritten and focused on a new feature: sharding. It is the simplest way to run slow tests (think of acceptance tests first) in parallel on multiple agents.

In this case, you specify the batch of tests that should be executed independently and each job picks up its own not intersecting group of tests to run them.

# first job
./vendor/bin/codecept run --shard 1/3

# second job
./vendor/bin/codecept run --shard 2/3

# third job
./vendor/bin/codecept run --shard 3/3

This feature reduces the need for complex configuration and usage of robo task runner to split tests.

It is recommended to use sharding to parallelize tests between multiple jobs as the simplest approach. Unfortunately, PHP doesn’t have native multi-threading for test parallelization, and even if it had, it doesn’t solve the problem of running slow browser tests that interacts with a whole application. So only horizontal scaling by jobs can be suggested as a long-running approach. The more build agents you add to your Continuous Integration server, the fastest tests will run. That’s it!

Grep and Filter

New options --grep and --filter were introduced to select tests by part of their name. Actually, it is the same option and an alias. --grep is a common way to select tests to execute in NodeJS test runners, so we ported it to Codeception. But as usual, specific tests can also be executed by group or specifying a test signature.

./vendor/bin/codecept run --grep "user"

Other Changes

Please go through the list of changes introduced to see if they don’t affect your codebase:

• Requires PHP 8.0 or higher
• Compatible with PHPUnit 9 and ready for PHPUnit 10
• Compatible with Symfony 4.4 - 6.0
• Stricter check for phpdotenv v5 (older versions are not supported)
• Throw exception if actor setting is missing in suite configuration
• Removed generate:cept command (Cept format is deprecated)
• Removed settings disallow_test_output and log_incomplete_skipped.
• Removed setting paths.log (it was replaced by paths.output in Codeception 2.3)
• Removed suite setting class_name (replaced by actor in Codeception 2.3)
• Removed global setting actor (replaced by actor_prefix in Codeception 2.3)
• Removed Configuration::logDir method (replaced by Configuration::outputDir in 2.0)
• Moved XmlBuilder class to module-soap
• Decoupled test execution and reporting from PHPUnit
• Custom reporters implementing TestListener are no longer supported and must be converted to Extensions
• Added optional value to fail-fast option (#6275) by #Verest
• Removed JSON and TAP loggers
• Removed code coverage blacklist functionality
• Removed deprecated class aliases
  • Codeception\TestCase\Test
  • Codeception\Platform\Group
  • Codeception\Platform\Group
  • Codeception\TestCase
• Introduced strict types in the code base.

Complete Changelog

We really happy that we are finally here with Codeception 5. This release was crafted during wartime, which happens in Ukraine. It is mentally and morally hard to work on tech products knowing that at any point this peaceful virtual life can end at any moment by a random missile. Codeception was created in 2011 by Michael Bodnarchuk in Kyiv, and today in 2022 he also stays there writing this post. If you want to support Codeception, all the Ukrainian PHP community, and all our brave nation who stands for democracy against barbaric Russian invasion, consider donating to Ukrainian charities. Not a single time. Every month until the war ends. Every time you travel or enjoy tasty food in a restaurant think of people who are forced to defend their land, or who fled their homes. Glory to Ukraine!

This release wouldn’t be possible without the hard work of Gintautas Miselis who keeps constant work on modernizing internals and keeping Codeception up to date. Also we are really thankful to Gustavo Nieves who did a lot of work transitioning Codeception to new Symfony and more! Thanks to our maintainers! If you want to support our work we have OpenCollective!