Integration of External Execution Results for Automated Tests – Part 1 (SolMan S...

BACKGROUND

SAP Solution Manager 7.2 SPS14 has been made available since January 2022.

This new version has brought several new features, we will see in this blog post how to take advantage of the new Integration of External Execution Results for Automated Tests.

I invite you to take a look at my colleague Evan Stoddard’s blog post that reviews some of the new enhancements added by SAP Solution Manager SPS14.

WHAT’S NEW?

This key feature synchronizes the execution status of Test Packages in SAP Solution Manager with the results of automated tests triggered and controlled by an external test automation tool.

After this synchronization is enabled, the execution status and log IDs of tests executed in an external tool are available in SAP Solution Manager, even if the test case execution was triggered only by the external test automation tool.

The test results are fully accessible from the Test Suite Analytics applications.

The first integrated test tool is SAP Enterprise Continuous Testing by Tricentis (ECT).

HOW TO CONFIGURE THIS NEW INTERFACE?

In this following section, I will describe the steps to follow to implement this new integration between SAP Solution Manager SPS14 and SAP Enterprise Continuous Testing by Tricentis 15.1.

In short, the integration process can be divided into 9 steps:

All steps of this integration can be reviewed in more detail in the official SAP Enterprise Continuous Testing by Tricentis (ECT) manual. We recommend that you always consult the official manual if you encounter errors in certain areas or if there are updates on this this fast-moving topic.

Step 1: Perform standard integration configuration

This new integration is added on top of the standard integration, I invite you to read the blog post of my colleague Lukas Goethel, as well as the SAP Enterprise Continuous Testing by Tricentis (ECT) manual.

Step 2: Review SAP Solution Manager Configuration (solman_setup)

You have to review the SAP Solution Manager Configuration that is specified in the Fiori app “SAP Solution Manager Configuration” (T-Code: solman_setup). And more specifically the:

• Test Suite part
• Test Automation Preparation sub-part
• 4 – Special Configuration tab

Step 3: Create a database for Tricentis Notification Service

In order to take advantage of this new integration, it is necessary to configure Tricentis Notification Service.

To do this, a database must be created. This database will act as a buffer between SAP Enterprise Continuous Testing by Tricentis, and the consuming application (SAP Solution Manager).

It is therefore a temporary database that collects all data, stores it, and then sends it. After sending the data, Tricentis Notification Service deletes the sent data from the database.

This database is not part of SAP Enterprise Continuous Testing by Tricentis installation. You need to prepare and host it on your end. The database you prepare must meet the following requirements:

• Database type
  • MS SQL
• Database user
  • db_owner
• User rights
  • CREATE, ALTER, SELECT, INSERT, UPDATE, and DELETE
• Database size:
  • big enough to handle the data during the transfer
  • Tricentis Notification Service deletes the data after successfully sending it on, so the database won’t grow over time
• Tables:
  • no need to create database tables
  • Tricentis Notification Service creates them when it starts for the first time

Step 4: Activate Tricentis Notification Service on Tosca Server

To activate “Tricentis Notification Service”:

1 – Open the Windows Service app

2 – Right click on “Tricentis.Tosca.NotificationService”

3 – Change the “Startup type” to “Automatic” and start the Service:

Step 5: Configure Tricentis Notification Service

Tricentis Notification Service is the data interface between SAP Enterprise Continuous Testing by Tricentis and other applications.

To configure “Tricentis Notification Service”:

1 – Open your Tricentis Tosca Server on gateway port

2 – Click on “Settings”

3 – Click on “Notification Service” tab, and configure the following properties:

• Port
  • 5025 – Default port used by Notification Service behind the Gateway proxy
  • You can specify a different port if needed.
• Enable Authentication
  • If you enable this feature, Notification Service checks in Tricentis User Administration whether a user has the necessary rights to access the project
• Database Connection String:
  • Enter a valid connection string to your Notification Service database
  • Example: server=xxx;Database=xxx;UId=xxx;Password=xxx
• Log Path
  • Enter the path to the directory where you want to save the log files
• Log Level
  • Define the log level: Error messages, Modifications, or All service communication
• SAP Solution Manager Integration
  • Enable

Step 6: Enroll your project and enable notifications

To activate “Tricentis Notification Service”:

1 – Open your Tricentis Tosca Server on Gateway port

2 – Open Tosca Administration Console

3 – In the Projects Overview, go to the project for which you want to enable Tricentis Notification Service. If you don’t have a project yet, add one.

4 – In the “Notification Service” column of the project, click on “Enable”.

Step 7: Install SAP .Net Connector assemblies to GAC

You should first uninstall any SAP .NET Connector (x86 and x64) from your machine.

Then restart the installation of your SAP .Net Connector (x86 and x64) by selecting the option “Install assemblies to GAC”.

Step 8: Configure Solution Manager Sender

Solution Manager Sender enables you to configure the Notification Service for SAP Solution Manager Integration.

To run and configure Solution Manager Sender, follow the steps below:

1. Go to C:\Program Files (x86)\TRICENTIS\Tosca Server\NotificationService.
2. Run the command line or Windows PowerShell and enter .\Tricentis.Tosca.NotificationService.SolutionManagerSender.exe -config.
3. Press “Enter”.
4. Enter the different parameters, based on the example described below:

Connection Name	[XXX] SAP SOlution Manager
System Number	XX
Client	XXX
Server Host	demo.alm.sap
SNC Mode	0
SSO Mode	0
User	XXXXX
Password	XXXXX
RFC Trace Path	C:\temp\TRACE\
RFC Trace level	15

Step 9: Review the settings of your Tosca Workspace

The Gateway Endpoint of your project in Tosca Commander must be well configured.

Since Tosca 15.0, the gateway includes the Tosca Server Landing Page, allows you to access all server features through a single web interface.

The following setting can be found under “Settings”->”Tricentis Services”.

HOW TO USE THIS NEW INTERFACE?

Step 1: Create a Test Configuration

As mentioned before, this new integration is added on top of the standard integration. The starting point is the creation of a Test Configuration in SAP Solution Manager 7.2.

To do so, I refer to the blog post of my colleague Amanpreet Singh Chahota which details the steps to perform.

Step 2: Create a Test Plan and a Test Package in SAP Solution Manager

Then, you have to create a Test Plan (status “Test in Preparation”), and assign your Test Configurations. Then, you have to create a Test Package and assign your Test Configurations to this Test Package.

Then you have to assign (at least) a Tester to your Test Package.

Step 3: Synchronization with External Test Tool

Then, you have to open tab “External Test Tool”. Select “Tricentis” as External Tool, click on “Refresh” and then “Save”. You can now click on “Edit” and click on “Synchronization can be started”.

Tosca will then open and an ExecutionList (= SAP Test Package) is created. This execution list is composed of the 2 ExecutionFolders (= 2 SAP Test Configurations). You must then perform a “Check-in” of this ExecutionList and “Resume Remote Control”.

You can click on “Refresh” again, and a green square indicates that the configuration has been completed.

Step 4: Execution of the automated Tests

The status of your Test Plan must be changed before any execution. You must use a status that allows the execution of the tests (“Released for Test” for example).

You can now execute your Test Cases directly from Tricentis. The execution status and log IDs of tests executed in an external tool are available in SAP Solution Manager, even if the test case execution was triggered only by the external test automation tool.

The results will be returned automatically to SAP Solution Manager at check-in time!

WHAT’S NEXT : DISTRIBUTED EXECUTION, CI/CD & CHARM INTEGRATION

We have now successfully synchronized the execution status of Test Packages in SAP Solution Manager with the results of automated tests triggered and controlled by an external test automation tool.

Now that you are familiar with this new integration, it is then possible to set up a highly scalable architecture by using Distributed Execution feature (commonly known as DEX), which allows tests to be run across all available computing resources, such as computers on your network, virtual machines or the cloud.

We will also see in the near future that it is possible to move towards a CI/CD Oriented Approach (with an integration with Jenkins for example).

Furthermore, developments are underway for the next releases of SAP Solution Manager. It will be possible to have an integration with Change Request Management (ChaRM), which would allow the execution of automated tests from any ChaRM Document status change.

Follow my profile to be notified of the next blog post. Please feel free to ask any questions in the comments section below.