CONTRACT TESTING

Photo by fabio on Unsplash

I have published this article originally as a guideline for PACT tests in Go. As writing Microservices in NodeJS is equally or even more present in today's market, I have decided to republish this article with instructions for JavaScript (NodeJS).

My favorite part of developing software is writing tests. It does not matter if we talk about the unit or integrational tests. I love them all.

That feeling when you write a test case for which function fails. It gives me a lot of joy when I find a bug in that stage of development just because I had fixed a bug before someone found it on some test environment, or even worse, production.

I stay up late to write some more tests from time to time. It is like a hobby. I spent around 30 minutes writing some unit tests for my private project on my wedding day. Do not tell my wife!

The only thing which bothered me for some time was integrational issues between multiple Microservices. How do I ensure that two Microservices with particular versions do not have integration issues? How to ensure that the new version of Microservice does not break the API interface, so the others can not use it anymore?

This information is crucial before we start massive scenarios in our end-to-end testing pipeline. Otherwise, we wait one hour to get feedback that we have broken the JSON scheme.

Then, one day in the office, I heard a rumor that we plan to use some Contract Testing. I quickly checked the first article, and I was amazed. It was a breakthrough.

Contract Testing

There are many excellent articles about Contract testing, but I like the most one from Pactflow. Contract testing makes sure that two parties can communicate together by checking them in isolation to test if both sides support the messages they exchange.

One party (Consumer) captures communication with the other party (Provider) and creates the Contract. That Contract is a specification of expected requests from Consumer and responses from Provider.

The application code generates Contracts automatically (in most cases, during the unit testing phase). Automatic creation guarantees that each Contract mirrors the latest reality.

Contract testing

After Consumer publishes the Contract, the Provider can use it. In its code (probably also unit tests), the Provider performs Contract verification and publishes results.

In both phases of Contract testing, we work only on one side, without any real interaction with the other party. Practically, we are making sure that both parties can communicate with each other in their separate pipelines. So, the whole process is asynchronous and independent.

If any of those two phases fails, both Consumer and Provider should negotiate about fixing integration issues. In some cases, the Consumer should adapt integrational code. In some other, the Provider needs to adjust its API.

Contract testing is NOT Schema testing. Schema testing is bound to one party without connection to any other. Contract testing verifies interaction on both sides and makes sure that any desired versions of them are compatible.

Contract testing is NOT End-to-End testing. We perform End-to-End testing on a group of services running together, where we test probably the whole system, starting from UI down to storage. Contract testing executes tests against each service independently. They are isolated and do not require more than one service running.

So, let us see what Contract testing is.

PACT is a tool for Contract testing. We use it to support checking communication between Consumers and Providers over HTTP protocol. It also supports testing message queues like SQS, RabbitMQ, Kafka, etc.

We provide Contracts by using PACT DSL for a specific programming language on the consumer side. The Contract contains interactions that consist of expected requests and expected minimal responses.

During test execution, the Consumer fires requests to Mock Provider, which uses the defined interaction to compare actual and expected HTTP requests. Once requests match, Mock Provider returns the expected minimal response, so the Consumer can verify if it meets expectations.

Consumer testing

On the Provider side, we use created Contracts to verify if the server can meet expectations. Verification results can be published to track which versions of Consumers and Providers are compatible.

When we run tests on the Provider side, Mock Consumer sends the expected request to the Provider. There it checks if the HTTP request meets expectations and returns a response. In the end, Mock Consumer compares actual response with expected minimal response and provides the result of verification.

Provider testing

We can store all Contracts and Verification results on PACT Broker. It is a tool that developers should host and maintain on their own on most projects, but it is also possible to use a public one, Pactflow.

Simple server and client in NodeJS

To write the first Contract, we should provide some code for a simple server and client. In this case, the server should provide one endpoint /users/:userId for returning Users by ID. Code generates results to avoid any more complex logic, like communication with a database.

Code for server

The complete code for the server is in a single file server.js. For this demonstration, I have used the Express web framework for NodeJS, but any other framework (or no framework at all) is good enough.

Client code is even more straightforward, and I placed it inside the file client.js. It creates and sends a GET request to the /users/:userId endpoint. After getting the result, it simply returns its payload. It relies on the Axios library to send HTTP requests to our server.

Code for client

I have provided dedicated functions for handling requests and responses in both client and server code. This code structure is vital to provide dedicated test functions later for the code we want to test.

PACT test for the client

Writing the PACT test for NodeJS is the same as writing unit tests. In this case, we should also rely on the NPM package from Pact Foundation. Besides it, I recommend using packages Mocha and Chai.

Pact test for a client

As you can see in the example above, I have provided the PACT test for NodeJS in a simple unit test. At the begging of the test, I defined a PACT Provider which runs Mock Server in the background.

The critical point of the test is defining an Interaction. Interaction contains the state of the Provider, name of a test case, expected request, and expected minimal response. We can define many attributes for both request and response, including body, header, query, status code, etc.

After we define the Interaction, the next step is verification. PACT test runs our client, which now sends a request to PACT Mock Server instead of a real one. I have made sure this would be the case by providing Mocks Server’s host to the getUserByID method.

If the actual request matches the expected one, Mock Server sends back the expected minimal response. Inside the test, we can make a final check if our method returns the right User after extracting it from the JSON body.

The last step is all about writing Interaction in the form of a Contract. PACT stores Contract inside the pacts folder by default, but we can change that in PACT Provider initialization. After we execute the code, the final output should look like this:

[2022-04-23 20:15:16.158 +0000] INFO (92784 on TS-NB-089): [email protected]: Pact running on port 57645
[2022-04-23 20:15:16.468 +0000] INFO (92784 on TS-NB-089): [email protected]: Setting up Pact with Consumer "example-client" and Provider "example-server"
    using mock service on Port: "57645"
      ✔ should get the right user data1 passing (2s)

PACT test for the server

Writing the PACT test for the server is easier. Here is the idea only to verify desired Contracts that the client already provides. We also write PACT tests for servers in the form of unit tests.

Pact test for a server

The client has already provided a Contract as a JSON file that contains all interactions. Here we need to define PACT Verifier and then execute verification of the Provider for the desired Contract, provided as a parameter pactUrls.

During Contract’s verification, the PACT Mock Client sends expected requests to the server, specified in one of the Interactions in the Contract. The server receives the request and returns the actual response. Mock Client gets the response and matches it with the expected minimal response.

If hole process of verification is successful, we should get an output similar to this:

Pact test for server
    handleUser
[2022-04-23 20:25:49.993 +0000] INFO (92887 on TS-NB-089): [email protected]: Verifying provider
[2022-04-23 20:25:50.056 +0000] INFO (92887 on TS-NB-089): [email protected]: Verifying Pacts.
[2022-04-23 20:25:50.058 +0000] INFO (92887 on TS-NB-089): [email protected]: Verifying Pact Files
[2022-04-23 20:25:51.616 +0000] WARN (92887 on TS-NB-089): [email protected]: No state handler found for "User Alice exists", ignoring
[2022-04-23 20:25:51.709 +0000] INFO (92887 on TS-NB-089): [email protected]: Pact Verification succeeded.
      ✔ should get the right user data (1716ms)1 passing (2s)

Usage of PACT Broker with Pactflow

As mentioned previously in this article, usage of PACT testing can not be completed without the PACT broker. Of course, we do not expect to have access to Contracts in physical files from clients inside pipelines of our servers.

Development teams should use one standalone PACT broker dedicated to that project for that purpose. It is possible to use the Docker image provided by PACT Foundation and have it installed as part of your infrastructure.

Also, if you are willing to pay for a PACT broker, the solution from Pactflow is perfect, and registration is simple. For this article, I have been using the trial version of Pactflow, which allows me to store up to five Contracts.

Dashboard Overview of Pactflow

To publish Contracts to Pactflow, I need to make minor adaptations to the file client.pact.test.js. Adaptations include the new part where I have defined a PACT publisher that uploads all Contracts after test execution.

Adaptation of client test

After you register with Pactflow, you should get the new host of your PACT broker. In addition, you should use your API token to finalize the definition of publisher in the code. You can find API tokens inside the settings of the dashboard overview.

When we execute the new client test, it adds the first Contract on Pactflow. This Contract has tags 1.0.0, latest, and master (which is added by default).

The first Contract on Pactflow

For having a difference in the client, I have adapted the test to send a request to the other endpoint /user/{userId} instead of /users/{userId}. In addition, I have also changed Tag and ConsumerVersion, to be 1.0.1 instead of 1.0.0. After I execute the test, the additional Contract will appear.

Two Contracts on Pactflow

Next, I have to adapt the test for the server. The new changes are present only for the verification process. It should now contain the PACT Broker host, API token, and decision to publish the verification result.

Adaptation of server test

In addition, it should also have a selector for the Consumer name and version to perform verification with the correct version of the Contract. The first execution, which passed successfully, will run checking for the client version 1.0.0.

Next, the second execution of the test, which failed, is to check the client version 1.0.1. The second execution must break, as the server still listens to the /users/{usersId} endpoint.

Verification results on Pactflow

To fix integration between the newest client and server, we should fix either. In this case, I have decided to adjust the server in a way to listen to the new /user/{usersId} endpoint.

After updating the server to the new version 1.0.1 and executing PACT verification once again, the test is ok once again, and it publishes new verification results on the PACT broker.

Updated Contract verifications

On Pactflow, like on any other PACT broker, we can also check the history of each Contract’s verification process that we can achieve by accessing a particular Contract from the dashboard overview and then checking its Matrix tab.

Contract’s Matrix

Conclusion

Writing the PACT test is a quick and cheap way to execute our pipeline. By verifying Consumers and Providers in the early stages of our CI/CD process, we save more time by getting early feedback about our integration results.

Contract tests allow us to use actual versions of our clients and servers and check them in an isolated process to determine if those particular versions are compatible.

What is your experience with Contract testing? And specifically, PACT?