Synthetic User Monitoring (SUM) in SAP Cloud ALM in action

Synthetic User Monitoring is a newest app in SAP Cloud ALM. In this app you are informed about availability and performance issues of your web applications before end users are getting affected by it. It does so, by executing on a regular schedule custom Selenium IDE scripts.

By the way: SUM is not meant for application testing or for task automation. For this, there are other nice capabilities planned or already available.

Selenium is an open-source framework for testing web applications. It consists of Selenium IDE (tool that is used to create Selenium scripts) and Web-browser-specific Selenium WebDriver (component that executes the Selenium scripts) with Selenium Grid (smart proxy server to provide easy way to run tests in parallel on multiple machines).

Synthetic User Monitoring (SUM) provides us with 2 main benefits.

It provides us with Proactive detection which enables us to detect issues early and even before the end-users encounter problems. Synthetic User Monitoring also provides us with Factual Metrics. SUM re-executes same scenarios, and it allows us to monitor the evolution of the performance and availability based on metrics collected.

Let’s have a look into the SUM main features:

• Monitoring of web applications with custom Selenium IDE scripts. These Selenium IDE files are containing groups of test scripts.
• Selenium remote web driver infrastructure is used for executing Scenario commands. For that you have to define runners, which are the components executing the tests of the scenarios.
• Scenario execution intervals are configurable.
• Metrics are captured at the scenario step level. Which means you can drill down a scenario to display performance and availability values on runner and step level.
• Performance evaluation is based on configurable thresholds.

Currently SUM keeps the scenario’s execution metrics for 14 days.

So what do we need to do now to get started with Synthetic User Monitoring?

To setup Synthetic User Monitoring you only have 2 prerequisites:

• You need Selenium Remote WebDriver infrastructure to configure runners – a client and a server. The client is your WebDriver test and the server is simply a Java servlet, which can be hosted in any modern JEE app server.
• Selenium SIDE file to configure the scenarios.

The configuration of Synthetic User Monitoring involves 3 components:

• Runners to define and configure where the monitoring scenarios are executed.
• Resources to upload and handle the Selenium IDE files.
• Scenarios to define and configure the monitoring scenarios.

Configuration of Synthetic User Monitoring is performed in the application itself.

Setup and Configuration is done in 4 different steps:

1. In Step 1 you configure Runners.

The Runner endpoint URL must be a trusted infrastructure because SUM sends the scenario commands to the runner endpoint, and it includes sensitive data maintained in the scenario configuration.

Currently are only managed runners supported which means that the scenario execution is scheduled and triggered by the SUM backend.

1. In Step 2 you Import Resources.

Upload the Selenium SIDE files that contains your scripts. After import it becomes a new Resource entry in the SUM configuration. During this process, file content is analyzed, and test becomes a SUM Selenium script.

In this step SUM detects the SUM variable annotations and automatically creates the corresponding SUM script variables.

If the updated file gets re-uploaded in Synthetic User Monitoring, it is updated.

1. In Step 3 you Maintain Scenario.

It means you create a scenario for your SUM scripts. Keep in mind that one runner can run several scenarios, but each scenario can only be assigned to one runner.

Following good practice during SUM scenario creation it will also be necessary to appropriately set secured variables via the SUM configuration in the parameters section of the Scenario details.

In the end of step 3, SUM allows you to adjust some Selenium Execution Parameters. The following properties are adjustable:

• Web browser to use (Currently are supported Edge Chromium and Chrome which is default choice).
• The webdriver arguments to pass to the runner endpoint.
• Execution speed which corresponds to a delay introduced after the execution of each command.
• Element Wait timeouts – commands with several targets, you can add maximum wait timeout for each of the targets. This does not apply to the commands with an explicit timeout (the wait commands).
• Global Wait timeouts – which is maximum time wait timeout to find the target of a command. This also does not apply to the commands with an explicit timeout.

1. In the step 4 we activate monitoring.

This means we activate the scenario on a runner.

The connection between SUM and selenium script is established by the Selenium Remote WebDriver infrastructure. SUM accesses Selenium Remote WebDriver Infrastructure to pass the Scenario commands to execute.

And now as promised, I’ll show you SUM in action. Our scenario looks like this: We want to monitor if our Focused Build Requirements Management is available on the one hand, but performant on the other hand. To push it a bit further, we also want to check if a very special requirement (status: “Approved” and label “SUM is great!”) appears in the list. Currently the requirement is in status “To be Approved” and our runner gives us an error. So we correct our Requirement and we set it to “Approved”.

Then we go to the SUM app in SAP Cloud ALM and wait for the runner to execute the script again:

As a summary, here again a drill-down in the SUM App into our scenario.

SAP of course has some more features planned for SUM – so stay tuned.

I look forward to your feedback.