Performing Verification

Before you can perform a verification, you must execute your test.

During the test execution, a message extracted from a source system is injected into a target system and is processed by the standard pipeline on the target system. The PIT tool collects the processing results from the target system and stores them locally as execution results. Simultaneously, a message exchange with all available log versions is retrieved from the target system. The result of each execution run is a collection of correlated and comparable source and target messages.

In the target system, a message is always injected into the XI-pipeline after the sender adapter. Therefore, the processing starts after the sender channel module chain.

You can choose whether to skip the processing in the receiver adapter (the message processing ends before the receiver module chain) or if the message is to be delivered to the current receiver

Test messages injected into a target system by the PIT tool are sent via the dedicated test connection PITTEST. Therefore, you can use this connection type as a filter in the PI Message Monitor of the target system in order to find all test messages in this system. The messages are displayed in the Message Status Overview. The column Test Message in the table Selection Details indicates if the processed messages are test messages or not. You can include or exclude test messages from the display list by selecting or deselecting Include test messages in the Custom Settings of the Table Settings Dialog

Test execution is only the first step of a successful test. Even in case message processing was successful in the target system, you don't know whether your expectations are met. Configurations can be different in the two systems. For example, mappings, value mappings, or routing conditions can differ. A verification step determines if a message is processed as expected in the target system.

During the verification process every message exchange from the target system is compared to the corresponding (reference) message exchange from the source system.

The questions are addressed with the verification steps in the test case configuration.

The structure verification is mandatory and is always executed at the beginning of the verification chain. The verification fails, if the structure is different (for example, if three messages were produced in the target system, instead of the expected two), or if messages can't be matched (for example, in the source system message A and message B were produced, but, in the target system, you have message B and message C). In these cases, the verification fails and the subsequent verification steps are skipped.

You can skip verification steps during configuration by disabling the steps in the test case verification.

In some cases, you want to accept two messages as equivalent, even if these messages aren’t the same, for example, if you have a time-stamp field in the payload. Here, the values always differ and you never get a successful verification result. Therefore, the PIT tool offers the possibility to handle such minor deviations through exemptions and replacement rules (exempt the time-stamp field from verification).

You can execute a verification process for a successful execution task either from the job browser or from the action log.

Before triggering an execution of a test based on a run configuration (via direct or scheduled execution), you can decide to run the verification automatically right after the execution (default setting).

It's possible to execute a verification process multiple times for the same execution task. It's best practice during development, to execute, maintain exemptions, and execute again.

The editor provides details for all test messages in the run and their recurrences including the verification status, and, if applicable, verification problems.

In the Overview tab, you get basic information about the verification, such as the test case run configuration (if applicable), the test case, and the test dataset used. It also shows whether the verification is executed as part of a test suite execution. The number displayed in square brackets behind the object name shows the version used during execution. Select the label link to display the content of the corresponding version.

In the Summary section of the overview, you see the number of messages processed during the execution run and the verification run and whether the runs were successful. Select the number link to navigate to the Message Overview to display the details.

The Message Overview tab lists all messages in the run as pairs (incoming source and corresponding incoming target message). For each message pair, you get the execution and verification status. You also get the number of differences found during the verification, if any.

If the execution of a test message failed, you find the corresponding error message in the Test Execution Problems view. If the execution was successful, the target message is verified and the verification status has one of the following values:

The Error Overview tab shows the complete list of all verification problems and the number of occurrences in the target messages. When you select an item in the table, the affected incoming target message identifiers are shown in the following table. By double-clicking a target message identifier, you navigate to the Exchange Structure Overview.

The Exchange Structure Overview enables you to inspect and compare the original source message exchange with the target message exchange. Select Browse, to display a specific pair of source and target message exchange. Choose one of the incoming target message identifiers, and confirm with OK. You can also display a specific message in the Exchange Structure Overview by double-clicking the message identifier in the Message Overview or the Error Overview tab.

In the split window, you see the source and target trees, both showing the respective message exchange with all relevant message versions. You can display the results in the Compare Mode. The compare mode is the default view for non-structure-related differences. In this mode, the selection in both trees is synchronized.

While inspecting the results of a test case verification, you sometimes encounter minor deviations you would like to ignore. For these cases, the verification exemption wizard offers a quick way to add exemptions to the test case.