Analyzing Testing and Deploy

Agile testing implies running tests frequently and as soon as possible in the development cycle. The cost of fixing an incident grows exponentially with the time that passed since the solution configuration activity was done or the code was written.

In an Agile project, the testing is part of the Sprint activities, and test results are checked before moving a User Story to Done status (see the Definition of Done lesson).

In the Prepare phase, the testing tools are set up, and the customer team is enabled to use the test management and the testing tools.

Test Strategy is finalized in the Explore phase, and the roles associated with testing must be staffed before the Realization phase begins. The recommendation is that testers are even participating in the Fit-to-Standard and solution design workshops so that they have an enhanced understanding of the solution and of the customers' needs.

The test strategy contains, among others, the thorough description of the Test Cycle, meaning the different phases of testing within the test execution:

• Unit Testing (if applicable)

• Business Process (String) Testing

• Integration Testing

• Data Conversion Testing

• Business / User Acceptance Testing

• Performance Testing

• Translation Testing (if applicable)

• Load Testing (different from Performance testing, if applicable).

This figure represents an example of a Test Cycle with the explanation of possible test types and activities related to testing. We must differentiate between the final Business Acceptance Testing that is planned just before the end of the Realize phase and the User Acceptance Testing performed during the Sprints. The difference between the two types of acceptance is that for the final Business Acceptance, extensive, functional, end-to-end testing is performed on migrated business data to confirm the solution, the data, and the organization's go-live readiness.

In this example overview of testing, we see different test activities associated with the project phases and different system environments.

We can see various levels of testing and the preparations for testing.

As we go into the Realize phase, we'll have the build Sprint and integrations Sprints that include the Unit Testing. Some String Tests may take place in the QA environment.

As we complete the solution build and integration developments, Integration testing will take place in the QA environment.

The Test Strategy will clearly identify when and where the project team executes data migration tests and performance tests. The Test Strategy should clearly identify the data, systems, and environments to be used in each test stage and how the testing data / environment will be updated or recreated.

This Test Strategy is delivered in the Explore phase and reviewed by the functional and technical teams.

Cutover activities are planned and prepared from the Realize phase and executed in the Deploy phase.

Cutover plans are adjusted and fine-tuned as a result of the dry run or dress rehearsal go-live that is organized to identify possible planning or technical impediments that could endanger the actual go-live.

Before the go/no-go decision, all issues identified during the dress rehearsal are analyzed and fixed depending on their criticality and the risk level of the go-live is estimated and managed.

For the Run phase, standardized procedures that leverage the experience of the organization and the software supplier are a key prerequisite for the optimization of application management.

SAP supports customers, partners, and consultants with guidelines called SAP standards for solution operations. These standards are published on the SAP Service Marketplace.

With SAP standards for solution operations, SAP provides standards for key operations processes within a company's business and IT units.

These key standards and practices address the needs of business process experts who are responsible for the design and execution of business processes, and of IT departments, and who ensure that the services provided by the SAP solutions are available for business users.

Each standard contains best-practice procedures on how to run the individual tasks, explanations on which tools inside SAP Solution Manager should be used, and available training and services that support the adoption of the standard.

The implementation and optimization of these SAP Standards for solution operations are the key deliverables of the Run phase work packages.

The Feature Traceability app helps you to analyze the readiness of your features based on the progress of its related objects like user stories, project tasks, and assigned transports.

SAP Cloud ALM supports solution testing activities and provides a special view to with the status of realization for the features in the project scope. The result of the testing activities that were performed on each feature is reflected in its status.

In the User Story column, the number of the respective entity with its statuses that are created from or assigned to each listed feature are shown.

Release managers can review the readiness of features for an upcoming release by selecting a release in the filter. Then, by selecting the status Done for projects tasks and a User Story in the extended filters, they can identify if all the related tasks were completed.

In the Deployment Status column, you can check if transports assigned to a feature are already:

• Released in the development systems
• Imported into the quality assurance systems
• Imported into the production systems

Sprints can also be set up to help manage the multiple types of tasks that must be organized for a successful Deploy phase.

These Sprints, most likely with a weekly duration, in which the cutover activities are planned along with testing, training, and OCM tasks, provide transparency on upcoming tasks and alignment between team members.

The daily synchronization between the team members with focus on impediment surfacing and their resolution is just one example of the benefits of continuing to run in Sprint Cycles beyond the Realize phase. The team has greater visibility and understanding on deploy activities, and solutions for removing obstacles are quickly identified. 

In SAP Cloud ALM, you can organize the testing and deployment activities, visualize them in a ScrumBoard, and work in Sprints with Scrum teams also in the Deploy phase.