Outlining Integration Options

In SAP Datasphere, the Data Builder provides a CSV file upload function by default.

You can use the Import CSV File option for manual uploads (under 25MB). For this feature, no connection has to be defined.

It automatically creates a table with derived columns based on file structure. You can apply various data wrangling and transformation rules, such as concatenate, split, extract, replace, change or filter.

To load data into an existing table (without transformations), use Upload Data From CSV File in the table editor.

Using connections to sources, there are several approaches of data integration in SAP Datasphere:

• Remote tables

  You can use remote tables to:

  • directly access data in the source (remote access).

  • copy the full set of data (snapshot or scheduled replication).

  • copy data changes in real-time (real-time replication).

• Flows

  Three types of flows are offered:

  • Data flow: Traditional ETL approach: transform data first, then store it, following the established ETL paradigm.

  • Replication flow: For ELT scenarios where data is extracted and loaded first, then transformed.

  • Transformation flow: For post-load transformations on already loaded data.

SAP Datasphere leverages different technologies to setup connections to sources. As a result, each connection provides different functionality, prerequisites and user experience.

Mostly SAP sources are ready to integrate their data as remote tables. For this data integration approach, the SAP HANA Federation Framework is currently mainly based on SAP HANA Smart Data Integration (SDI) and its data provisioning framework (dpServer + dpAgent).

The dpAgent is a lightweight component running outside the SAP Datasphere environment. It hosts data provisioning adapters for connectivity to remote sources, enabling data federation and replication scenarios. The dpAgent acts as a gateway to SAP Datasphere providing secure connectivity between the database of your SAP Datasphere tenant and the adapter-based remote sources.

The dpAgent is managed by the Data Provisioning Server. It is required for all SDI connections in general. Through the Data Provisioning Agent, the pre-installed data provisioning adapters communicate with the Data Provisioning Server for connectivity, metadata browsing, and data access. The Data Provisioning Agent connects to SAP Datasphere using JDBC. It needs to be installed on a local host in your network and needs to be configured for use with SAP Datasphere.

There are some sources which are able to setup a direct connection (for example SAP SuccessFactors and SAP HANA Cloud). For a direct connection to SAP HANA on-premise, SAP HANA Smart Data Access (SDA) with the Cloud Connector is used.

SAP embedded Data Intelligence (DI) provides the functionality of data flow and replication flow. In this scenario, DI connectors are used to reach remote sources. To establish a connection to these sources the Cloud Connector is required to act as link between SAP Datasphere and the source. Before creating the connection, the Cloud Connector requires an appropriate setup.

The Cloud Connector serves as a link between SAP Datasphere and your on-premise sources and is required for connections that you want to use for the following use cases:

• Data flows.

• Replication flows.

• Model import from SAP BW/4HANA model transfer connections (Cloud Connector is required for the live data connection of type tunnel that you need to create the model import connection).

• In rare cases also for remote tables: only for SAP HANA on-premise via SDA.

You can use the Import Entities wizard to load metadata from your SAP S/4HANA Cloud and SAP S/4HANA on-premise connections via semantically-rich objects. The wizard creates Business Builder and Data Builder entities (along with all the objects on which they depend) in SAP Datasphere.

We recommend that, where possible, you use the Import Entities wizard for importing CDS views from these connection types, as it is able to leverage their rich semantics to import higher-level objects and to follow associations to dimensions, hierarchies, and text entities and include them in the import.

You can also use the Import Entities wizard to load metadata from your SAP BW∕4HANA and SAP BW bridge connection via semantically-rich objects. The wizard creates Business Builder and Data Builder entities (along with all the objects on which they depend) in SAP Datasphere.

It is possible to enable SAP SQL data warehousing on your SAP Datasphere tenant to exchange data between your HDI containers and your SAP Datasphere spaces without the need for data movement.

This integrated SAP SQL data warehousing approach allows you to add HDI containers to your space and exchange data between them:

1. Use calculation views and other SAP Business Application Studio objects as sources for your SAP Datasphere local tables and data flows.

2. Use SAP Datasphere views that are exposed for consumption as sources for your SAP Business Application Studio calculation views and flowgraphs.

3. Use SAP Business Application Studio tables as targets for your SAP Datasphere data flows.

In SAP Datasphere, you can also use SAP HANA calculation views, generated from SAP BW/4HANA objects, which are located in schema _SYS_BIC.

An Open SQL Schema in SAP Datasphere provides a flexible SQL endpoint that enables secure data integration and modeling capabilities through database users. It creates a dedicated schema in the underlying SAP HANA database that allows both read and write operations, facilitating seamless data exchange between SAP Datasphere and external systems.

Here are some examples how the Open SQL Schema can be used:

• Create tables, views, and stored procedures using DDL and DML statements.
• Write data to tables.
• Create a table to act as a target to receive data written from a data flow.
• Serve as a bridge for data extracted from third-party systems via middleware.
• Leverage SAP HANA's embedded machine learning capabilities including APL, PAL, spatial engine, text processing, and graph algorithms for advanced analytics and predictive modeling.
• Leverage data anonymization and data masking
• Data science: use tooling like Jupyter notebooks