Test Strategy
Different tests are conducted at various stages of an implementation project with specific purposes and objectives. Three common tests are implementation tests, user acceptance tests (UAT), and regression tests.
Implementation tests occur in the Realize phase of the SAP Activate methodology. During this time, partner LoB configuration experts are getting each business process fully configured in the customizing tenant of the development system and transporting business configurations and customizations to the test and production systems. Testing the functionality of a piece of a business process is called a unit test. Stringing together multiple unit tests is a string test. Business role customizations need to be tested to ensure the people who should have access to data/functionality do, and those who shouldn't have access, don't. Eventually, you end up with an entire business process test, where you use the test script from SAP Signavio Process Navigator to run the entire test procedures (including the prerequisite steps). If there was a set-up instruction document for the particular business process you are working on, this must be completed first, as they usually cover some type of integration or initial activities that need to occur prior to beginning the prerequisite steps in the actual test script.
Business processes should be tested in the SAP S/4HANA Cloud test system, after all integrations have been set up and data has been migrated for the relevant process, so you can test the process in a situation as close to reality as possible. Unit and string tests are informal, and therefore should happen organically as the configuration and customization is completed. An end-to-end business process test could include one or more individual business processes that occur one after the other, and are formal, which means they need to be documented in the SAP Cloud ALM Test Management apps. Integration and data migration tests are also formal and therefore must be documented.
User acceptance tests (UATs) also occur in the Realize phase, but with a different audience. You may remember from the unit covering Fit-to-Standard workshops, that in an ideal situation, customer LoB experts participating in the workshops would also participate in UAT of the configured business processes. This is because those same customer experts saw business processes demonstrated in the SAP S/4HANA Cloud system and provided feedback about small changes that needed to be made to ensure the system works for their needs. Partner LoB configuration experts worked on addressing these requirements, and the best possible users to confirm the requirements have been successfully handled are the individuals who reported them in the first place. In addition, the customer LoB experts already have some general understanding of how to navigate the SAP S/4HANA Cloud system, and therefore no additional training is required.
For UAT, manual test cases are set up by the partner LoB configuration experts with the SAP Cloud ALM Test Management apps, and customer LoB experts use these apps to document their findings after working through the test procedures in the SAP S/4HANA Cloud test system. UAT is testing is formal, and therefore must be documented. This is a critical time to identify and resolve issues BEFORE the system is live and in productive use. TAKE YOUR TIME to ensure all issues are resolved and the results are documented before moving into the Deploy phase.
Regression tests typically occur in the Run phase after a customer's system is live, to identify whether any changes in the latest release upgrade affect the customer's system configuration and/or customizations. Regression tests after go-live are the responsibility of the customer to handle, and the Test Automation Tool is designed to automate as much of this workload as possible. However, if the customer's implementation is not finished when a release upgrade occurs, the partner implementation team is responsible for pausing the current implementation activities to test the business processes that have been implemented thus-far against the upgraded software and content versions. Another type of test, called post-upgrade tests are available for customer to opt-into if they have not made customization changes to the standard business process. For these tests, consent must be provided in the SAP S/4HANA Cloud test system to allow SAP to run these post-upgrade automated tests after the release occurs. This is another method of attempting to reduce the testing workload for the customers (and partners, if the release occurs during implementation).
To make an upgrade as smooth as possible, partner LoB configuration experts should set up the test process automate (using the Test Automation Tool) as they finish working through the configuration and customization of the actual business process to make sure they align with each other. For example, if you needed to add some custom business logic that triggers something to happen within a standard business process, you need to make sure the standard test automate is also customized to capture this business logic outcome. When an upgrade occurs, you can simply use the Test Automation Tool to run these test automates in the SAP S/4HANA Cloud test system (as opposed to manually clicking through the test script). After go-live, the customers continue to use these test automates for customized processes, and the post-upgrade test automates for standard processes that have not been customized.
NoteUse the Scope and Dependencies spreadsheet to help with planning your testing activities. Find it in SAP Signavio Process Navigator → SAP Best Practices for SAP S/4HANA Cloud Public Edition solution scenario → Accelerators tab → Getting Started tab → Availability and dependencies of solution processes.