JavaScript Test Automation Frameworks

To understand JavaScript automated testing frameworks, we must first have a foundational understanding of how to approach testing in the general sense, including the purpose and structure of test cases and the mindset we use to create effective test cases.

A General Approach to Testing

A test framework is primarily responsible for running a test suite or a collection of test cases. Each test case is designed to exercise a single, logical unit of behavior in an application and ensure that the targeted behavior operates as expected. Generally, test cases are structured in three parts:

1. Preconditions – Assertions that must be true before the test can execute
2. Command – The logic of the test case that is intended to verify some behavior
3. Postconditions – Assertions that must be true once the test is complete

Written formally, using Hoare Logic, a test case can be expressed as:

{P} C {Q}

where:

• {P} are the preconditions of the test case
• {C} is the command to be executed
• {Q} are the postconditions of the test case that we expect to be to be true

We can write tests in natural language — called Behavior-Driven Development (BDD) — using the following clauses:

1. Given – The environment and context of the test case
2. When – The logic that is under test
3. Then – The assertions about the results of the test

For example:

Given I have addends of 3 and 5
When I sum them
Then the result is 8

Lastly, we can translate our test cases into code, as seen in the pseudocode example below of our previous BDD test case:

// Given

a = 3

b = 5

​

// When

result = a + b

​

// Then

assert result == 8

Since our test case is now expressed as code, we can execute it the same way we execute our application code. In practice, this means executing all, or a specific subset, of our test cases each time we make changes to our application. This ensures that regressions — bugs that cause previously working components to fail — are not introduced in our application.

Reporting is another important responsibility of a test framework. Apart from simply denoting whether each test passed or failed, frameworks are also responsible for describing why a test case failed. For example, if the value of result in our previous test case was 7, we expect that our report would include a description of the failure, informing us that the actual result of the test, 7, does not match our expected result, 8.

Testing JavaScript Applications

The generalized approach to testing and test frameworks above holds true for nearly all programming languages and ecosystems. While JavaScript can be executed outside of a browser using Node.Js (Node), a vast majority of the JavaScript applications we see today are browser-based. This is unsurprising, as JavaScript has a significant advantage over nearly all other programming languages in this realm: All modern browsers include a JavaScript engine that allows JavaScript code to be executed natively within the browser, which brings its own set of advantages and challenges.

This tight relationship between JavaScript and the browser allows us to programmatically execute basic actions we take for granted, such as clicking a button or hovering over a specific Hypertext Markup Language (HTML) element. Due to JavaScript's capability to interact with the browser and execute these actions, JavaScript test frameworks must also be able to inspect the results of these actions, as seen in the following diagram:

For example, if we click a button, we may want to assert that a different HTML element changes to a specific color, a dialog box is displayed, or a Representational State Transfer (REST) request is made to some back-end service. Additionally, we may want to take screenshots or videos of a test case's results — or the entire execution of a test case — to ensure that the results match our aesthetic specifications (e.g., graphic designs).

Since users can choose from a host of different browsers (e.g., Chrome, Firefox, Safari, Edge), we may also want to ensure that our application behaves and appears as expected in multiple browsers. This adds an additional requirement to our test framework: interoperability with different browsers.

Conceptually, each browser has its own application programming interface (API) that allows an application to inspect the results of an interaction with the browser. Over the years, as more and more browsers with vastly different APIs became popular, standards were devised that abstract the details of each browser and allow test frameworks to interact with browsers in an agnostic fashion.

These advancements have led to three different types of JavaScript test frameworks:

1. Standardized – Uses a standard API, which ensures interoperability with all browsers that support the standard, but it can be inefficient and lack support for browser-specific features
2. Non-Standardized – Uses browser-specific protocols that have not been standardized, which may be more efficient and feature-rich, but it may not support all browsers
3. Proprietary – Uses a customized mechanism — such as proxying or code injection — to interact with browsers, which can be efficient and feature-rich, but it is more complex and may introduce quirky limitations

Each of these categories has its advantages and disadvantages, and the frameworks that use them excel and deteriorate in different ways. In the following section, we will look at six of the more prominent frameworks, exploring how they utilize one of the three approaches above to allow us to create automated tests in JavaScript.