Loading Data into the Corporate Memory Layer

We now define the first layer in the SAP BW/4HANA data flow, where data is stored. The object we will create is called a DataStore Object (advanced).

A DataStore Object (advanced) stores transaction data in SAP BW/4HANA. A DataStore Object (advanced) can be defined using fields or InfoObjects. A simple DataStore Object (advanced) is defined using fields whereas a more sophisticated DataStore Object (advanced) is defined using InfoObjects. InfoObjects provide additional meta data to the DataStore Object (advanced). For this layer, we do not require the extra meta data.

When you create a DataStore Object (advanced) you choose whether you would like to implement a data activation step. A data activation step is useful when you want to control the 'release' of each data load. This is often implemented to support a quality check of the data before it moves to the next layer, or to hold the data until other related data is loaded so that it can be combined.

Depending on the role of the DataStore Object (advanced) different types of DataStore Object (advanced) can be defined. Here are the most important types:

• Staging DataStore Object: typically used to store a copy of source field data (Raw data / Corporate Memory Layer)
• Standard DataStore Object: typically used to handle delta records (Data Warehouse Layer)
• Data Mart DataStore Object: typically used for reporting and analysis (Data Mart Layer)

In our scenario the DataStore Object (advanced) is defined as Staging DataStore Object with Inbound Queue Only selected. The setting Inbound Queue Only is selected when a data activation step is not needed. In our scenario we have decided that we don't need an activation step. Our DataStore Object (advanced) is built using fields instead of InfoObjects. We have chosen to use fields because in this layer of the data flow we don't need the advanced features that are provided when you use a DataStore Object (advanced) InfoObject based.

Watch the following demo to explore the field-based, staging DataStore Object (advanced):

The next step of the data flow is to define a Transformation between the DataSource and the field-based Staging DataStore Object.

A Transformation is an SAP BW/4HANA object that defines a set of rules that define how to generate target field values using source field values. A Transformation always sits between a source object such as a DataSource, and a target object, such as a DataStore Object (advanced).

When data is loaded, the data passes through a Transformation. A Transformation converts the fields of the source into the format of the target. A Transformation consists of at least one transformation rule. Various rule types allow you to create very simple or highly complex transformation rules. For example, a direct mapping between a source and target field is considered to be a simple transformation rule, while using an ABAP Managed Database Procedure (AMDP) that uses program code for a transformation rule could be very complex.

In our scenario a Transformation is needed between the DataSource and the field-based Staging DataStore Object. As our DataStore Object (advanced) is modeled as Corporate Memory, every field of the DataSource is mapped 1:1 with the corresponding field of the target DataStore Object (advanced). Corporate Memory always preserves the raw data from the source system without making changes. We do this so that we can re-use this data in many other data flows.

Click below to watch a demo where you will explore the Transformation.

The next step of our data flow is to define a Data Transfer Process (DTP) between the DataSource and the field-based Staging DataStore Object.

A Data Transfer Process (DTP) is an object that defines precisely how data is transferred between a source and target object. When you create a DTP you choose settings that determine precisely how data is handled during the transfer.

A DTP is used to load data to an SAP BW/4HANA target object. An important setting in the DTP indicates if you would like to transfer data either in Full extraction mode or in Delta mode. In full mode, the entire data set of the source is transferred to the target; in delta mode, only the data that was posted to the source since the last data transfer is transferred.

Furthermore, it's possible to define a Filter in a DTP to specify which data should be loaded. For example, records for a specific calendar year, or records from a specific sales organization.

To load the data, a DTP has to be executed. For testing purposes execution can be manual, but for production systems, the execution is usually automatic. You will learn later how we automate the execution of multiple DTPs using a Process Chain.

In our scenario, a Data Transfer Process with the Extraction Mode setting Full is defined. This means it cannot be used to manage delta records. We did not define a filter in our DTP.

You can watch the following demo to explore the DTP.