Introducing automated tests

In any software development cycle, testing is an integral part of the end-to-end process. Although manual tests are easy to perform during development, finding defects over time is hard. If we automate these tests, they produce better results in the long run, compared to labor-intensive manual tests.

Automated tests are small programs with one purpose, to verify a certain behavior of the application. By creating automated tests, we can be confident that code is correct. We can specify the expected behavior in the form of a test, and automatically verify that our application does the right thing. In addition, it is easier to simplify and re-factor code if we have tests that show us how our application should behave. In this way, we can verify that our application still works after we changed the code.

Automated tests are also a good tool to implement common quality standards within a team. It is a good idea to have a rule like: new application code can only be added with tests, in a software development team.

There are multiple kinds of tests available, which differ mainly in the granularity of testing. They all have their advantages and disadvantages. A common visualization is the testing pyramid. Based on the costs of the tests, it shows that you should have much more unit tests than integration tests, and much more integration tests than end-to-end tests. The costs for creating, running, and maintaining the tests increase as you move up the pyramid.

The testing pyramid is a tool to visualize various kinds of tests. The principle is that the lower in the test pyramid a test category is, the more individual tests with a smaller scope should exist, and vice versa.

Unit tests have the smallest granularity. They can be defined directly on the programming level, for example, calling methods of your Java classes. Dependencies to other modules or systems may be mimicked to ensure that they run quickly and only test the portion under consideration. These tests are usually cheap. You should use them to verify that your software modules, such as, classes, behave as expected.

Integration tests have reduced complexity. They skip the UI and work directly on the defined back-end APIs. They test the integration between software modules or systems. In our example, we mainly use them to test the integration between back-end services and the integration to SAP S/4HANA systems. Although they have reduced complexity, they still have medium costs. They still have an overhead, for example, for network communications or spawning a small server to make the back-end APIs available. You should use them to verify that your back-end services can communicate with your SAP S/4HANA system and to test that the services behave as the UI expects it.

For end-to-end tests, the idea is to simulate a typical user workflow. An automated Web browser clicks through the Web interface of our application, and verifies that the expected screens appear. Testing on this level shows that the features of the application are working. You could have, for example, one end-to-end test per feature or user story. However, end-to-end tests are expensive to write and execute. End-to-end tests interact with the browser. This is expensive, because it takes time to start the browser and the browsers loads and renders a lot of resources. In addition, it is usually difficult to specify the interaction with the browser in a way that it can deal with smaller changes in the application UI structure.

Smoke tests are run when the application is deployed to production to verify that the deployment was successful. This is part of the continuous integration and delivery workflow.