Testing is a crucial part of the implementation process. It is also traditionally a pain point during implementations. One challenge in test automation is figuring out what exactly to automate because some things can't be automated (a process with a high level of subjective human decision-making, for example). Those tests must continue to be done manually. Another challenge is maintenance of tests specifically in the context of an agile, devops environment. Tests should be stable but also adaptable in case of changing processes, either from software upgrades or business process redesign. Finally, tests should be able to be executed anytime anywhere and test creation should be simple and straightforward with easy data definition (including data variant creation) and fast reliable infrastructure for test execution.
The SAP S/4HANA Cloud Test Automation Tool is SAP's solution which is offered to customers to address testing automation pain points. The SAP S/4HANA Cloud Test Automation Tool enables customers to implement an automated regression suite for their testing needs.
The tool is a cloud native, no code and no installation solution. To kick start customer enablement, more than 300 standard automate scripts are provided based on processes taken from the SAP Best Practices Explorer. Customers are free to modify those scripts based on their business needs. Stability of tests from SAP are guaranteed across upgrade cycles. Test data is maintained with a dedicated app for this purpose, and a wide variety of artifacts (UI elements, APIs, for example) can be tested. Test results are contained in a dedicated app with a user interface that is well designed with easy to read and understand data points in an intuitive and easy to navigate dashboard. Customers also have the ability to create their own test scripts with a dedicated app for this purpose.
Now, let's look a little more deeply at some of the functionality and specifics of The SAP S/4HANA Cloud Test Automation Tool.
First, the tool is provisioned on SAP Business Technology Platform and, therefore, inherits all the advantages SAP BTP provides, such as the availability of worldwide hyperscaler infrastructure that can scale on demand, and the ability to use cloud native applications managed by SAP.
As mentioned previously, SAP S/4HANA Cloud Test Automation Tool contains standard test process automates based on best practices found in the SAP Best Practices Explorer. These automates provide customers with explanations concerning the flow of an application. They can be used as a starting point for custom fields, which enables customers to create custom test automates. SAP will maintain the standard automates and customers, likewise, will maintain their custom ones.
Test data is maintained centrally, and is distinct from an actual test process. This allows the same process to use different test data as well as different processes to use the same data. One additional benefit is that test datasets can be reused across releases. Test Data is organized in three layers. The first is the template repository which refers to the area level the data refers to. Test data templates are delivered based on functional areas (sales, finance, for example) that contain fields that are part of that functional area. Next is the Test Data Container which is created by end users (using the fields from the test data templates area), and which comprises structural entities containing those fields along with some configured characteristics. The Test data container can be standard or custom. SAP delivers Standard Test Data Containers that can be used out of the box and reused as needed. In addition to these, customers can create their own containers.
There are two types of automation testing available for customers to utilize: UI automation and API automation. UI control in this context refers to any controls that are part of a Fiori app, such as buttons, input fields, checkboxes, etc. Any of these standard UI controls can be tested. In addition to the standard controls, actions related to those controls (for example, a user selects a button, or enters values in an input field) are also able to be tested. API automation allows customers to test released APIs that are available on SAP API Business Hub. Both ODATA and SOAP APIs are testable and all possible CRUD operations (Create, Read, Update and Delete) can be tested. In cases where API requires an input payload, a payload structure will be provided to provide the input values for the API. Unlike UI automation, which by design must take account of each step of a business process and each possible action from every step, API automation is relatively easier, requiring only a test of an API call independent of the action which necessitated it. In addition, due to potential changes in the underlying business process as part of the normal upgrade process, UI controls along with actions may need to be adjusted in testing. However, API tests need only be changed if the API itself has changed. As a result, API testing is almost always easier to setup, execute and maintain than UI testing.
One of the unique offerings of SAP S/4HANA Cloud Test Automation Tool is the Post Upgrade Test (PUT) service. This is an optional service from SAP that customers can request on demand. Post Upgrade Tests are tests that are performed by SAP (on customers' behalf in their Q systems) to verify basic standard functionality still works after an SAP S/4HANA Cloud upgrade. For example, after an upgrade, and using sample master data from a customers Q-system, the Create Supplier Invoice application can be tested.
The SAP S/4HANA Cloud Test Automation Tool allows customers to execute standard and custom test processes via test plans. The tool is included as part of the customer's SAP S/4HANA Cloud subscription. It is provisioned in the Q (Quality/Test) systems by default, when the system is handed over to customers. Custom test processes can be created by using the recording functionality. The tool consists of the following four applications: Manage Your Test Processes, Test Your Processes, Analyze Automated Test Results, and Test Data Container.
Customers use the Manage Your Test Processes app to create and manage end-to-end test processes. A process step results from user actions such as UI control input or UI control eventing, such as pressing a button or clicking on a link.
After a test process is created and set to visible in the Manage Your Test Processes app, it can be tested in the Test Your Processes app. With this application, you can create and maintain automated test plans and if applicable view the post upgrade test plans for the current release. A test plan refers to one or more test processes to be executed. Before a process can be tested the Test Data Container app must be used to create test datasets.
Once a test plan is executed, the Analyze Automated Test Results application displays the results. The application enables you to analyze the executed test plan or test process for business process testing. Results are in a tabular format with drill-down capabilities. Customers can quickly understand the reasons for a test failure and take corrective actions. The test results can be saved locally as an Excel file and, in addition, as an executable tile in The SAP Fiori Launchpad to facilitate collaborative analysis and decision making by implementation teams.