Maintaining Process Chains

In SAP BW, process chains are organized using a dedicated Display Component hierarchy. For SAP BW/4HANA, the Display Components are migrated to corresponding InfoAreas.

For in-place conversions, the migration will be done automatically during the technical upgrade to SAP BW/4HANA. For Shell or Remote conversion, SAP recommends transporting the display component hierarchy to the target SAP BW/4HANA system first, before transferring any process chains. Without this transport, process chains are temporarily transferred to the NODESNOTCONNECTED InfoArea. When the transport is done, the process chains get re-assigned to the correct InfoArea. To transport the display component hierarchy, add the program id R3TR, object RSDC, and object name RSPC to your transport request.

In SAP BW/4HANA, there are considerably fewer process variants than in earlier releases of SAP BW. This is related to the streamlined availability of objects types in SAP BW/4HANA. Another reason is the characteristics of SAP HANA which does not require additional performance-related activities like index generation or database statistics refresh. There are two new variants as well:

• BW Request Status Management - Housekeeping: Cleanup of Request TSNs in DataStore Objects (advanced), InfoObject Characteristics, and Open Hub Destinations.
• Check for new data in ADSO of successor DTP: Enabling asynchronous Loading of BCS/4HANA InfoProviders. Only relevant for scenarios related to the BCS/4HANA add-on.

When you migrate from SAP BW any DB to SAP BW powered by SAP HANA, the obsolete process variants in existing process chains are skipped during processing and cannot be modeled in new process chains. SAP recommends doing housekeeping in this area, however, to streamline the process chains.

After conversion to SAP BW/4HANA, a number of process variants are no longer available. The process chains as such are still fully supported, but if a variant must be deleted (during an in-place conversion, for example), then the corresponding process chain has to be adjusted accordingly, that is, the variant has to be removed from the chain and the gap between the preceding and subsequent processes must be closed. In some cases, a process variant has to be replaced by the corresponding new process type. Here, the process chain also has to be adjusted accordingly. There is some tool support for these adjustments, but manual effort cannot be avoided.

In the SAP BW/4HANA Cockpit, there are some additional apps regarding process chains in SAP BW/4HANA. The existing SAP GUI transactions remain for the time being, but the sooner you get acquainted with these new capabilities the better.

• Model Process Chains:

  The system displays a list of all process chains in the system. You can create, change, and delete process chains here. To create and edit process chains, you can use the graphical editor in the app. You can insert processes there by selecting process variants, or define processes directly in the editor.

• Model Process Variants:

  The system displays a list of all process variants in the system. You can create process variants for many process types here. You can use these process variants in process chains. However, you do not have to model process variants first to be able to model the process chains. As in the past, it is possible to model process chains and create appropriate process types there.

• Monitor Process Chains — Dashboard:

  The system displays various charts with an overview of important statistical analysis of process chains and process chain runs. From here, you can navigate to the monitoring list of process chains or to the list of process chain runs.

• Monitor Process Chains — View latest status:

  The system displays a list of all process chains with status information. You can schedule, de-schedule, or repair process chains here, and can call the log for the current run.

• Monitor Process Chains Runs:

  The system displays a list of all process chain runs in the system with status information. You can repair a run here or call the log of a run in order to analyze errors.

In 2006, SAP NetWeaver 7.0 introduced a concept called Real-Time Data Acquisition (RDA) to enable very fast uploads of new data at very frequent intervals (every hour to every minute) especially for operational reporting scenarios. RDA is only available for InfoCubes and DSOs (classic), so not for ADSOs, and as a consequence it is not a valid concept in SAP BW/4HANA anymore. As a premium alternative, SAP has implemented the RDA capabilities into the standard framework of Process Chains.

These so-called Streaming Process Chains are the recommended concept since they were released with SAP BW 7.5 SP04. They enable almost real-time data transfer to InfoProviders and present up-to-date data for BW reporting. It is not 100% real-time, because the realistic latency is measured in minutes in the best case. So do not expect latency times of milliseconds to seconds which are common in pure data replication scenarios for example.

There is no automatic migration from the RDA concept to Streaming Process Chains. During the prepare phase of a SAP BW/4HANA conversion, several manual steps have to be performed to remove the RDA components and to set up the same logic based on Process Chains. Please refer to SAP note 2447916 for a detailed step-by-step description.

Streaming processes can be set up for DataSources which are delta-capable and which are based on the SAP BW/4HANA source systems ODP_* or HANA_* . For ODP sources, it is an advantage if the DataSource is streaming-enabled, because in this case the process is transferring data only if new delta data exists in the source. For HANA sources, the same applies if the connection has a property called real-time change data capture. If these preconditions are met, then empty loading requests can be avoided. For this reason, this is the preferred mode for working with streaming process chains.

A streaming process chain can only consist of few process types and should not cover additional housekeeping tasks for example. As the focus is on fast execution and repetition, DTP Execution and ADSO Activation are the main process variants which are supported for these cases.

Streaming is an operation mode of process chains in SAP BW/4HANA. You activate it by choosing Process Chain → Attributes → Streaming Mode from the menu in the SAP GUI or in the Process Chain properties in the SAP BW/4HANA Modeling app.

When a process chain is running, each step takes a certain amount of time; the interesting question is what actually happens, when the chain is started again - maybe even several times - during this period, and the execution reaches the process which is already active from a previous execution:

• The classic approach is to schedule process chains in the background. Then the second execution waits 10 minutes for the previous execution to complete. If the previous is not completed within this period, the second execution is canceled, the process chain is interrupted, and fails with an error.

• A process chain in streaming mode is executed using the new queued task manager framework. This enables a flexible number of worker jobs that get the next process from a queue. If a second, third, and further execution reaches the current active process, these requests to start this process once more are written to a queue only. No more resources are used, and the system does not wait. As soon as the current process is finished, the worker job receives all of these start requests for a given process and restarts the process just once: it takes the most recent chain execution from the queue and executes it. This means that the second and third chain executions are dropped, and only the most recent chain execution is continued.

  Picture this: Several people are waiting outside the airport for their respective cab. They all want to go to the same hotel. They all end up taking the same cab, and the other cab rides do not take place.

The data transfer controlled by streaming process chains can be distinguished in different run modes: A Push and two Pull run modes. The main difference between them is whether the consuming system or the source system is triggering the data transfer:

• In the Push mode, the source system sends a trigger to SAP BW/4HANA which executes the data transfer.
• In the Pull mode, the SAP BW/4HANA as target system is in the lead. This means, it checks for new data in the source and transfers the new data to the target InfoProvider. There are two versions of this mode:
  1. Intelligent Pull: SAP BW/4HANA checks regularly in the source for new data. However, the data transfer is executed only when this check is successful. This means, no empty requests are generated in the target InfoProvider.
  2. Scheduled Pull: The process chain leverages the streaming framework and is scheduled periodically. If required this can happen frequently, in small intervals up to every minute as a maximum. The data transfer is executed for every scheduled run, which means a request is created in the target InfoProvider irrespective of the existence of new source data.

The main characteristic of the PUSH mode is that a trigger is received in SAP BW/4HANA from the source system when a new record is generated. This event starts the immediate execution of the process chain which transfers the new data to the InfoProvider. The process chain is scheduled once in streaming mode, the definition of periodic repetitions is not required. This is the preferred mode for ODP sources, because the process chain runs only when new data is available in the source system.

Preconditions:

1. This mode is only available for ODP DataSources of types SAP Extractors, ABAP CDS Views, BW InfoProviders and SLT. Other ODP types like ODP_HANA or ODP_BYD do not support this push mode.
2. Another precondition is that DataSources are Delta-capable:
   • ODP_SAP or ODP_CDS: The delta-capability is dependent on the implementation.
   • ODP_BW: The delta-capability depends on the source InfoProvider. Supported sources are DSOs (classic), InfoCubes and ADSOs.
   • ODP_SLT: all DataSources are delta-capable in general.
3. Finally, the DataSources need to support Streaming:
   • ODP_CDS, ODP_BW, ODP_SLT: all delta-capable DataSources support this property by default.
   • ODP_SAP: This property depends on the individual implementation of the extractor. For example, LO Cockpit DataSources (2LIS_*) are generally not streaming-enabled due to their complexity (e.g. due to the queued delta), but other SAP delivered DataSources might support this feature. However, most of the SAP delivered DataSources are not streaming-enabled in the delivered version.

     There are some corrections provided (see SAP notes 1692484, 1780297, 1782173), but in most of the cases so-called Realtime-flag has to be set manually in the SAP source system (for example, by a customer report updating ROOSOURCE-REALTIME to value 'X'). However, consider that this only makes sense for DataSources which are able to provide new data in real-time, which means a delta is available at every point of time (see also SAP note 905089, not released currently and so SAP internal only).

     Customer developed DataSources do not have this property initially as well. It has to be enabled explicitly in the DataSource definition (transaction RSO2). The corresponding property is called Real-Time enabled and is available in the customizing settings for the Delta mode.

If these three preconditions are met, then the ODP Queue Daemon is able to trigger the process chain in the target SAP BW/4HANA system to execute the DTP and load the new data immediately. This daemon has to be scheduled in the SAP source system as regular job ODQ_DAEMON_CLIENT_xxx in the ODP Queue (transaction ODQMON, GoTo / Real-Time Daemon...).

The Intelligent Pull mode's main characteristic is that the target SAP BW/4HANA system checks very frequently (each 60 seconds) if a new record is generated in the source. If this check is successful, the process chain is started immediately to transfer the new data to the target InfoProvider. The process chain is scheduled once in streaming mode, the definition of periodic repetitions is not required. This is the preferred mode for HANA sources, because the process chain runs only when new data is available in the source system.

Preconditions:

1. This mode is available for HANA DataSources only.
2. These DataSources need to be Delta-enabled.
3. If the source system type is HANA_LOCAL or HANA_MDC, check additional requirements in SAP note 2372534.
4. If the source system type is HANA_SDA, it is important which type of remote connection is in use:
   1. SAP HANA Smart Data Access (SDA) drivers are not supported for this run mode in general.
   2. SAP HANA Smart Data Integration (SDI) adapters: The intelligent Pull is only available for SDI adapters which support Real-time Change Data Capture as documented in the functional overview of the SAP HANA SDI adapters (source: SAP Product Availability Matrix, aka PAM: https://support.sap.com/content/dam/launchpad/en_us/pam/pam-essentials/TIP/PAM_HANA_SDI_2_0.pdf). As of SAP HANA SDI 2.0 SPS05 those are:
      • ASELTLAdapter, ASELTLECCAdapter
      • DB2ECCAdapter, DB2ECCLogReaderAdapter, DB2LTLAdapter, DB2MainframeLogReaderAdapter
      • MssqlECCAdapter, MssqlLogReaderAdapter
      • OracleECCAdapter, OracleLogReaderAdapter
      • PostgresSQLLogReaderAdapter
      • HanaAdapter
      • TeradataAdapter
      • AbapAdapter
      • CloudDataIntegrationAdapter
      • TwitterAdapter

If these preconditions are met, then the process chain can enable the intelligent PULL mode. In a nutshell, the SDI adapter replicates each change to a generated BW table (/BIC/CMT*) in real-time. You can identify this table in the BW DataSource definition, there is a button called SAP HANA objects. This is the location the streaming process chain checks for new data every 60 seconds. If there is new data in the source, the process chain is started immediately to load it to the target InfoProvider.

Note: For BW DataSources related to source system HANA_* the Streaming property is always off (Streaming cannot be triggered by this DataSource). This property only refers to and is only needed for the PUSH mode ODP sources).

The main characteristic of the Scheduled PULL mode is that a SAP BW/4HANA system executes a process chain very frequently, simply to leverage the new task manager framework. This might be the most preferred mode if the source is updated extremely often, e.g. one record each second. Then the Intelligent PULL order PUSH mode would not be able to transfer the new data in the same speed and a simple scheduling would be the more efficient way to load the data to SAP BW/4HANA.

Preconditions:

1. This mode is available for ODP DataSources which are not streaming-enabled.
2. This mode is also available for HANA DataSources which rely on SAP HANA SDA drivers or SDI adapters not listed in the run mode above.
3. Another precondition is that the DataSource is Delta-capable.

If these preconditions are met, then a process chain in streaming mode has to be scheduled as a periodic job in a high frequency if required (up to each minute). If there are updates available, they will be loaded immediately to SAP BW/4HANA. On the other hand, if there is no new data, then an empty request is created in the target InfoProvider.

The fact that the preconditions the PUSH mode or the Intelligent PULL mode are not met does not mean that you cannot run process chains in streaming mode. You can still leverage the queued task manager framework to run them with high frequency for Delta-capable DataSources. In this case, the process chain needs to be scheduled as a periodic job to pull data from the source, instead of the process chain being triggered only when new source data is available.