Alternative work flows are very useful, but you need to design it carefully so that only the data flows that did not finish successfully are rerun.
This can be done by enabling the automatic recovery option for the job.
Watch the following video to learn more about the automatic recovery feature.
Recovery Requirements
In our example, to ensure that the fact tables are loaded with the data that corresponds properly to the data already loaded in the dimension tables:
- The recovery job must use the same extraction criteria as the original job for the recovered data flow.
If the recovery job uses new extraction criteria, such as basing data extraction on the current system date, the data in the tables will not correspond to the data previously extracted into the completed data flow. If the recovery job uses new values, the job execution may follow a completely different path with conditional steps or try/catch blocks.
- The recovery job must follow the exact same execution path as the original job.
Data Services records any external inputs to the original job so that the recovery job can use these stored values and follow the same execution path.
Using Try/Catch Blocks for Recovery
Data Services does not save the result of a try/catch block for re–use during recovery. If an exception is thrown inside a try/catch block, during recovery, Data Services executes the step that threw the exception and subsequent steps.
Since the execution path with the try/catch block might be different in the recovered job, using variables set in the try/catch block could alter the results during automatic recovery.
For example, suppose that you create a job that defines the value of variable $i within a try/catch block. If an exception occurs, set an alternate value for $i. Subsequent steps are based on the new value of $i.
During the first job execution, the first work flow contains an error that generates an exception, which is caught, and then sets the variable to 0. But the job goes on with the next step and executes the workflow for $i>1. However, the job fails in the subsequent work flow as shown in the following figure.
After fixing the error, you run the job in recovery mode. During the recovery execution, the first work flow no longer generates the exception. Thus the value of variable $i is different, and the job selects a different subsequent work flow, producing different results as shown in the following figure.
Caution
To ensure proper results with automatic recovery when a job contains a try/catch block, do not use values set inside the try/catch block or reference output variables from a try/catch block in any subsequent steps.
Recovery Units
In some cases, steps in a work flow depend on each other and are executed together. When there is a dependency, designate the work flow as a recovery unit. This requires the entire work flow to complete successfully. If the work flow does not complete successfully, Data Services executes the entire work flow during recovery, including the steps that executed successfully in prior work flow runs.
Conversely, you may need to specify that a work flow or data flow should only execute once. When this setting is enabled, the job never re–executes the object.
Caution
Summary
In this lesson, you have learned how to configure automatic recovery in SAP Data Services to efficiently rerun only failed data flows after fixing issues.
- Activate automatic recovery to skip successfully completed data flows and rerun only failed components after fixing errors.
- Do not reference variables set inside try/catch blocks in subsequent steps to prevent execution path changes during recovery.
- Designate interdependent workflows as recovery units to ensure all steps complete successfully together, maintaining data consistency.
- Mark workflows or data flows to run only once when re-execution could cause issues like duplicate data.