Using SAP Databricks on SAP BW Data

You want to learn how SAP BW data, which is exposed as data product, can be used in SAP Databricks for AI/ML scenarios, for example to forecast data. You also need to understand how this data can be shared back from SAP Databricks.

To make a data product available for consumption in SAP Databricks, you have the following options:

• Share the data product from the SAP Business Data Cloud catalog.
• Create a catalog from the Delta Share associated with the desired data product in SAP Databricks.

If you share the data product from the SAP Business Data Cloud catalog, you must specify the following information:

• Target System: SAP Databricks.

• Workspace: a SAP Databricks deployment in the cloud that functions as an environment for your team to access SAP Databricks assets (like notebooks, tables, views, SQL queries, and so on). Depending on its needs, an organization can choose to have multiple workspaces or just one. 

• Share Name: only lowercase allowed.

In the following exercise, you will share a data product to SAP Databricks.

SAP Databricks provides advanced data engineering capabilities to SAP Business Data Cloud.

A data scientist can consume, query, process, and transform data products in SAP Databricks, including working with SQL and Python notebooks for various data science and AI/ML projects.

Let’s have a look at the steps which are taken by the data scientist, to perform data analysis on the shared SAP BW data.

1. Locate the shared data product: in SAP Databricks, the data product is accessible as a delta table in the Unity Catalog. The schema and the table (DDS.....) can be retrieved in the folder "Delta Share Received".

2. Perform advanced data engineering using notebooks. For example these tasks:

   • Data preparation
   • Data training
   • Data forecasting
   • Writing forecast results to a delta table

3. Share the delta table via delta sharing: Delta sharing is an open protocol for secure sharing of delta tables. SAP Business Data Cloud supports delta sharing and allows user data sets, represented as shares, to be shared securely. SAP Business Data Cloud SDK is used to create a delta share in SAP Databricks, which includes the delta table and SAP Business Data Cloud as a recipient.

4. Publish the delta share as a data product to SAP Business Data Cloud: To allow the discovery of the shared data in SAP Business Data Cloud, SAP Business Data Cloud SDK is used in SAP Databricks to describe the delta share with the following metadata:

   • ORD: the Open Resource Discovery protocol allows applications and services to self-describe their resources and capabilities. It facilitates cross consumption of resources and capabilities between different systems through a common standard interface.

   • CSN: a standardized format for configuring and describing shares within a network.

   After publishing, the data product is available in the Catalog of SAP Business Data Cloud.

You want to use the data from SAP Databricks in SAP Datasphere to combine it with SAP BW data.

Let’s have a look at the steps to combine SAP Databricks data with SAP BW data in SAP Datasphere:

1. Locate the SAP Databricks data product in SAP Datasphere: You can use the Catalog & Marketplace in SAP Datasphere to find the shared Data Product from SAP Databricks.

2. Install and explore the data product in the target space: You install the data product in the space where SAP BW data will be combined with the data from SAP Databricks. After this, the space holds a Local Table with the data from SAP Databricks.

3. Model using SAP BW data and data from SAP Databricks: you can use the Data Builder in SAP Datasphere to create views and analytic models.

   Note

   Analytic Models can be used as a source in SAP Analytics Cloud to visualize the results in a story using tables, charts and other widgets.

In the following exercise, you use a data product from SAP Databricks in SAP Datasphere.

In this lesson you have learned how SAP Databricks can be used on SAP BW data:

• Expose SAP BW data as a data product and share it with SAP Databricks.
• A data scientist uses the data product in SAP Databricks to perform advanced data engineering.
• A data product from SAP Databricks can be installed and used in SAP Datasphere for modeling scenarios.