Use Different Types of DataStore Objects (advanced)

The DataStore Object (advanced) is the SAP BW/4HANA object for storage of data. It can be used for many different modeling scenarios, or for many different EDW layers, for both field-based and InfoObject-based modeling.

The DataStore Object (advanced) consists of three core tables that are generated in the background when the DataStore Object (advanced) is created and activated. The system uses the tables depending on the selected modeling options. Regardless of the use case, these three tables are always generated to later support quick and flexible changes to the data model. The tables are as follows:

• Inbound table: /BIC/A<technical name>1

  Structure: Request ID (REQTSN), Data Package (DATAPAKID), Record number (RECORD), Record mode (RECORDMODE), Key Field 1, Key Field n, Field 1, Field n...

• Table of active data: /BIC/A<technical name>2

  Structure: Key Field 1, Key Field n, Record mode (RECORDMODE), Field 1, Field n...

• Change Log table: /BIC/A<technical name>3

  Structure: Request ID (REQTSN), Data Package (DATAPAKID), Record number (RECORD), Record mode (RECORDMODE), Key Field 1, Key Field n, Field 1, Field n...

Regardless of which instance of the DataStore Object (advanced) you model, the data is always loaded into the inbound table first. Then, the data is either read from this table directly, or it is processed further into the other one or two tables for reading or extraction. This depends on how the data is used and how the model is customized.

In addition to these three core tables, the following three table views are created as well. Which of the three core tables they refer to depends on the modeling scenario (that is, the setting of the Modeling properties):

• View for Extraction from the DataStore Object (advanced): /BIC/A<technical name>6

• View for Reporting for the DataStore Object (advanced): /BIC/A<technical name>7

• View for external SQL Access for the DataStore Object (advanced): /BIC/A<technical name>8

  Note

  Although it is common practice, SAP don't support directly accessing the three tables that store the data of a DataStore Object (advanced) (refer to SAP Note 1682131). For this reason, SAP BW/4HANA provides a database view for external SQL access (also referred to as the External SAP HANA SQL View). It is meant to be a stable interface that allows official and authorized access to the data of a DataStore Object (advanced). It is a technical, low-level API primarily designed to be called in staging scenarios, for example, to perform lookups in routines with ABAP or AMDP/SQLScript in transformations.

Core modeling properties are located in the General tab in the BW Modeling Tools. These properties represent the key settings that define the behavior of the DataStore Object (advanced) and the usage of its three database tables. There are also some additional enhancements called "special properties" regarding inventory key figures, planning functions, and a new write interface, which are available depending on the modeling scenario you choose.

The following applies to the key definition of a DataStore Object (advanced):

• It can contain up to 120 key fields. The key can consist of InfoObjects, or fields, or any combination of both.

  Note

  In a Data Mart DataStore Object, there is no limit to the number of key fields.

• It can contain up to 1000 fields (key fields/InfoObjects plus data fields/InfoObjects).

The following applies to the use of navigation attributes:

• Navigation attributes can’t be set for a DataStore Object (advanced)

  Note

  You can define navigation attributes in a CompositeProvider on top of the DataStore Object (advanced), and then they are fully available for reporting.

• In the Details tab of a DataStore Object (advanced), navigation attributes of InfoObjects can be switched on for Transformations, to avoid setting up manual look ups. In the Details tab, select the InfoObject that must provide its navigation attributes to the source structure of an SAP BW/4HANA transformation rule. Then switch on the property Use Navigation Attributes in Extraction.

Let's start with one of the most popular types of DataStore Object (advanced).

To model a DataStore Object (advanced) as a standard type that uses a Change Log, choose Standard Data Store Object and select the setting Write Change Log. With a Change Log the delta (new, deleted, and changed records) is saved in the Change Log. The Change Log is used to extract the delta. Requests can only be rolled back from the DataStore Object (advanced) if there is a Change Log. That is, the status before the activation of the request can be restored.

If your DataSource only delivers the current dataset as FULL and not delta, you can use the Snapshot Support option. By setting this indicator, deleted data records are identified and updated. Upon activation, the system recognizes records that are in the table of active data but not in the load request. These are written to the Change Log as reverse images.

If you only load unique data records (data records with nonrecurring key combinations) into the DataStore Object (advanced), you can use the option Unique Data Records. During activation of a request, the system checks whether unique records exist. If a record already exists, the activation is canceled with an error. This option is useful to speed up the loading process since only inserts are performed during loading.

Do not request a Change Log. Only the inbound table and the active table are used. Also, in this scenario, you can choose the Unique Data Records setting.

DataStore Objects (advanced) of the type Staging are optimized for fast loading. These DataStore Objects are usually found in the inbound layers of the EDW architecture to ensure that data is transferred from source systems as quickly as possible.

When this type is chosen, there are more settings to fine-tune the model.

For example, when you set the option Inbound Queue Only, no data is saved in the Change Log. The extraction process always reads the data in the inbound table again, for delta extraction or full extraction.

A DataStore Object (advanced) with the property Inbound Queue Only can only be used for reporting to limited extent, since the data in the inbound table is only relevant logically for the object. However, the query doesn't know the activation logic and therefore cannot aggregate over records.

When you set the option Compress Data, the data of a request is written to the inbound table. During the compression process, the data is written to the active table (aggregated in accordance with the semantic key of the active data table) and deleted from the inbound table. To save memory space, the change log is not filled. Therefore, you cannot perform request-based deletion of data from the DataStore Object (advanced). You can only delete data selectively.

For full extraction, the inbound table and the table with the active data are accessed. Delta extraction only uses the inbound table.

A DataStore Object (advanced) with the property Compress Data can only be used for reporting to a limited extent, since the data in the inbound table is only relevant logically for the object. However, the query doesn't know the activation logic and therefore cannot aggregate over records.

When you set the option Reporting-Enabled, request data is written to the Active Data table during activation, but is also kept in the inbound table. Since the data is stored redundantly in the inbound table, the data can be deleted from the active table and rebuilt from the inbound table. The data is only extracted from the inbound table. Reporting only takes place on the table containing the active data, meaning that the data is only visible after activation.

This type of the DataStore Object (advanced) has the following features:

• Key fields: No keys can be specified for this DataStore Object (advanced) type, because all characteristics are included in the key (logical key). Therefore, even if one field changes in a record, a new record is always created (a new key). There is no update or deletion of a characteristic and this ensures that a history of the change is preserved.

• Update: Key figure values can only be aggregated and not overwritten.

• Loading: The data is always loaded into the inbound table.

• Activation: When you activate the data, the data is copied to the table of active data and grouped according to the aggregation behavior. Afterward, the data is deleted from the inbound table. The change log is not filled.

• Rollback: For this type, requests can only be deleted if the request has not yet been activated.

• Extraction: Full and initial extraction read the data for the update into a further data target as a union of the inbound table and the table of active data.

  The data for the delta extraction is read from the inbound table only.

• Reporting: The query reads data as a union from the inbound table and from the table of active data. Therefore, it is not necessary to activate the data to be able to see all the data. The data read is consistent, since data is only included up to the first inconsistent request. The data read is also stable, since the quantity of data included from the DataStore Object (advanced) does not change during the navigation steps in the query.

With the DataStore Object (advanced) for direct update, you can load data directly to the table of active data, including standard consistency checks. For example:

• SID processing.
• Consistency of time characteristics.
• Areas locked by cold store.
• NLS.

You can load data by standard DTP/Transformation or by the following APIs:

• RSDSO_DU_WRITE_API / RSDSO_DU_WRITE_API_RFC: Loads data from an internal table to the table of active data.

• RSDSO_DU_DELETE_API_RFC: Deletes data from the table of active data. This one can be truncated or selectively deleted.

• RSDSO_DU_CLEANUP_API_RFC: Deletes API requests with errors. Red requests block further load requests by DTP/Transformation or by API.

When you use the option Write Interface-Enabled, the data in this DataStore Object (advanced) can be moved to the inbound table using SAP integration solutions by PUSH. Based on this property, it is the successor for the obsolete BW source system types Data Services, Web Service, and External System. The interface can be integrated with the following solutions:

• SAP Data Services.
• SAP Data Intelligence.
• SAP Process Integration (PI).
• SAP Integration Suite as part of the Business Technology Platform (BTP), formerly also known as SAP Cloud Platform Integration (CPI).
• 3rd party ETL Tools.

The Write Interface option can be used for all types of Standard DataStore objects and Staging DataStore objects, but not with Inventory-Enabled or Planning-Enabled.