Loading Data with ODP_SAP

Now that you've learned about ODP_SAP DataSources, let's see how to load the data to SAP BW/4HANA. Before starting a DTP to load data, it's useful to first test the extraction to check if it provides the correct data.

The extractor-based DataSources are loaded to SAP BW/4HANA using ODP, as the S-API interface is not used anymore. However, the Extractor Checker can still be used.

The Extractor Checker is launched with the transaction code RSA3. This is a good way to look at the data supplied by a traditional SAP DataSource.

The DataSource runs in SAP ECC. Then, you can look at the data records that are selected for extraction. Using this technique, you can prove that the DataSource runs properly in SAP ECC. Before you execute the DataSource, you can also control the number of data packets (calls) and the number of data records per data packet.

When you use Operational Data Provisioning, the transaction ODQMON allows you to monitor the operational delta queue.

The role of a provider is to provide one or more delta queues of a specific type. The BW DataSource is an example of a Provider. The target application of the delta queue is referred to as the Subscriber.

A subscription can be defined as follows: a specific subscriber orders data changes from one or more queues and continues processing the transferred data. Subscribers are categorized by their subscriber type. A subscription occurs when the subscriber requests data. Every subscription has a unique transaction number (for example, 2022-09-02 08:53:26 000000 CET). A subscriber can have more than one subscription. A queue can also be in multiple subscriptions for the same subscriber.

The data, which is highly compressed, is retained in the delta queue for a specified time period, in case the subscriber wants to retrieve the data records again. More than one subscriber can request the data changes to a queue.

Data cannot be changed in the ODQ (a feature that previously did exist with the PSA). Instead, the inbound table data of a DataStore Object (advanced) can be edited before activating the request. If raw data must be persisted in SAP BW/4HANA, a field-based DataStore Object (advanced) can be used to replace this role of the former PSA.

The following calculations are performed in the monitor view:

• Queues:

  For each queue, the system counts the number of units, totals the rows and data volume (before and after compression), and finds the compression rate. It also finds the smallest and largest sequential transaction number (TSN).

• Subscriptions:

  For each subscription, the system counts the number of changed units per queue that are saved (in a retrievable state) in the delta queue. The system totals the changed rows (data changes) and the volume of saved, nonretrieved data (before and after compression), and also finds the compression rate.

• Requests:

  For each request, the system counts the number of units per queue, totals the rows and data volume (before and after compression) and finds the compression rate.

• Units:

  The Unit view displays the data that was extracted when you double-click on the unique time stamp. The preselection made in the monitor or the navigation route to this view (and the associated time stamps and queues) define which units are displayed here. For a unit, the system totals the rows and the volume of saved, nonretrieved data (before and after compression), and also finds the compression rate. Note that the system always performs the calculation at the unit level.

When you calculate the data volume (extended view) in the upper screen area of the monitor, the data volume and related key figures are calculated and displayed.

A request can be defined as a data request from a subscriber. There are two different types of requests, as follows:

• A composite request transfers data from one or more queues that have been grouped together into a subscription.

• An extraction request transfers queue data from the provider to the queue storage.

A composite request can contain several extraction requests. If only data changes that were written to the queue by the source application are requested, the composite request does not contain any extraction requests.

You monitor the requests in the Requests overview in the delta queue monitor view. You can perform various actions for the unconfirmed requests: in other words, requests that the subscriber hasn't transferred.

Use to check all the displayed requests to see if any of the extraction processes were canceled or terminated. The status in the database and in the display is updated (corrected).

You may want to investigate failed extraction requests. To analyze the cause of the error, follow these steps:

1. Use Job Overview to open the job overview. Here, you can display the log of the relevant background job (if it has not been deleted).
2. In the application log (transaction SLG1), you can analyze the data extraction logs, using object ODQ and sub-object Extraction.
3. Use to reschedule failed extraction requests. The extraction process is delayed by 60 seconds. This gives you the chance to activate debugging for the background job - in the process overview (transaction SM50) or in the global work process overview (transaction SM66) - and find the cause of the error.

You may want to close unconfirmed requests.

Use to conclude requests that the subscriber hasn't confirmed or canceled. For the following reasons, you may need to close a request in the monitor:

• The request without subscription has not been retrieved (for example, the connection to target system was deleted).

  In this case, don't assume that the subscriber will close the request themselves.

• The system could not extract the data changes (delta) .

  In this case, none of the subscribers to this queue can continue transferring the new data, because data changes must be transferred seamlessly in the correct sequence. Usually, closing the failed request is not enough to continue transferring the data changes for this queue. You must solve the cause of the problem, otherwise the next extraction attempt will also fail.

If a reorganization job is scheduled, it deletes all requests that have been successfully retrieved by all the downstream targets based on the Recovery period (the default is 24 hours). The following figure shows the normal SAP job log, and also highlights the cleanup job that is normally run automatically.