Why trying to do too much E2E testing might have the same result as eating too much ice cream = it goes bad

In the complex world of SAP S/4HANA transformations, particularly when dealing with Selective Data Transitions/BlueField or Complex Brownfield projects, End-to-End (E2E) testing might seem like a panacea. However, this might be a classic case of a Greek myth of Sisyphus – a never-ending task. This article will delve into the why E2E testing for SAP projects is often a trap, the anti-patterns in testing, and a potential solution to the predicament.

The Pitfalls of E2E Testing in SAP S/4HANA Projects

The initial impression of E2E testing is undeniably appealing. Imagine a scenario where every modification made by your SAP functional or development team, is automatically validated, isn’t that a dream scenario?

However, it’s crucial to comprehend the ramifications of E2E testing in this context. Let’s consider the implications of having 50-100 third-party systems. Can you do a real E2E testing of that? The augmentation in the project scope due to E2E testing is substantial, not even doubling or even tripling the initial plan but just making it impossible to achieve.

Moreover, SAP E2E testing setups are usually deployed by personnel unfamiliar with the SAP business processes or the functionality of the integrated third-party systems. This unfamiliarity frequently leads to exhaustive, slow, and expensive testing out of fear of potential issues, without necessarily enhancing the SAP S/4HANA project quality. In essence, the testing phase happens too late in the project to prevent quality problems effectively.

The Test Pyramid and its Anti-patterns

Unfortunately, many organizations implementing SAP S/4HANA projects fall into the trap of an inverted pyramid, also referred to as the Ice Cream Cone anti-pattern developed by Alastair Scott. This model, while common, is usually a result of neglecting a culture that encourages developers to create unit tests, discourage isolated testing and leads to the creation of more and more E2E UI tests to compensate, hence creating the inverted pyramid.

To better comprehend the issues at hand, we can look at the original Test Automation Pyramid introduced by Mike Cohn. This pyramid suggests more unit and component tests (due to their simplicity and cost-effectiveness), some service and API tests, and much, much fewer E2E UI tests because of their complexity and relative expense.

A Shift in Approach: Simulate to Simplify (S2S) by Isolating and Decoupling

So, if E2E testing is impractical, and the Test Pyramid’s rigid adherence may lead to the Ice Cream Cone, what’s the solution?

The answer might lie in simulation of 3rd party systems and doing “SAP E2E testing” only. In an ideal scenario, we’d use simulation to isolate and decouple SAP projects from third-party systems and EDI/B2B partners. By focusing on testing the SAP system for SAP S/4HANA projects, we can bypass the complexities of the third-party systems, thereby reducing project scope, speeding up the process, and mitigating the need for simultaneous testing across multiple parties.

Moreover, a testing strategy should be risk-oriented. Instead of fixating on the number of tests at each level, focus on addressing system risks. Use the testing pyramid as a guide, but remember that it’s a model, not a definitive rule.

Ultimately, the goal should always be to provide value through testing, even if that contradicts the traditional pyramid model. By utilizing simulation, we can streamline the testing process and make it more efficient, while addressing the significant risks, leading to a successful SAP project.

Where I can learn more about this topic of Simulations/Service Virtualization for SAP transformation programs?

There’s a 3h course on openSAP just on this topic called “Avoid SAP S/4HANA Project Delays with Third-Party Systems Service Virtualization” – In this course we help you understand why SAP-tailored service virtualization is a hidden gem of SAP S/4HANA projects, who can use it and when to use it, and most importantly, how to benefit from it. In addition, you’ll learn how service virtualization can make your projects more sustainable by significantly minimizing their carbon footprint.

Further reading

E2E testing for SAP is a mouse trap! Don’t fall for it – Decouple instead

Why is End to End (E2E) testing blocking your SAP test automation adoption journey?

How to run many SAP S/4HANA rollouts at the same time? Use project tracks!

Simulate 3rd party, non-SAP systems and EDI partners to remove delays from complex SAP S/4HANA transformations

Composable SAP ERP – Decentralized Warehouse and 3PL scenarios