The concept of data warehousing was initially defined in the late 1980s. A popular definition originates from Bill Inmon, who described it as "a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management´s decision making process". These four key properties mean the following:
- Data is collected about those entities which have an impact on the future success of your organization (for example vendor, product, customer) and their relationships (for example key figures representing processes like purchasing, production or sales).
- Data is integrated from many different sources and as a result they are available in a consolidated way in a separate system.
- Data is collected for different points of time to enable time series analysis and comparisons, so the time dimension has a crucial significance.
- Data is not changed any more, it is stable and permanent once it is transferred to the data warehouse.
The current offerings are SAP NetWeaver BW 7.5 (in maintenance until end of 2027) and SAP BW/4HANA (at least supported until end of 2040). Both solutions are widely in use at SAP customers due to the high amount of standard content and the easy integration with other SAP solutions.
Both, SAP NetWeaver BW and SAP BW/4HANA are classic on-premise applications, but they are now also enabled to be deployed in private cloud environments. To close the gap towards a full public cloud offering, SAP launched SAP Data Warehouse Cloud in the end of 2019. SAP Data Warehouse Cloud is a pure SaaS (software as a service) approach, sometimes also referred to as DWaaS (Data Warehouse as a service). It is deployed by the well-known hyperscalers in different continents and it receives updates in a biweekly cycle currently.
There are three major use cases for SAP Data Warehouse Cloud:
- Self-Service Data Modeling and Analytics targets key business users and empowers them with self-service, top-down driven analytics in an IT governed environment via so-called "SAP Data Warehouse Cloud Spaces". These key users may be granted instant access to business applications via pre-built adapters to integrate different types of data from any source (SAP and non SAP, on premise or in the cloud).
- Data Democratization describes the requirement to accelerate time-to-insight via rapid onboarding of resources. A lower adoption barrier is achieved with a business semantic layer which abstracts from the underlying physical data sources. An important value added is provided by delivered business content which enables setting up industry-specific data models including core KPIs quickly. End-to-end decision-making is facilitated by collaboration framework based on SAP Data Warehouse Cloud Spaces.
- End-to-End Data Warehousing focuses on reusing existing SQL skills, IDEs, open-source and/or in-built tooling for data integration, governance, harmonization and Big Data historization to provide a consolidated view across the enterprise. SAP try to keep customers' investment from existing on-prem investments. The first step were so-called hybrid scenarios based on deep integration with SAP BW/4HANA. In addition, SAP introduced the so-called "BW Bridge option" in DWC in the end of 2021. This option is a new feature of DWC that provides a path to the public cloud for SAP BW NetWeaver and SAP BW/4HANA customers. Based on a new SAP ABAP stack in the SAP Business Technology Platform, customers can move to the public cloud in their own pace by keeping their past investments. Recommended reading: https://blogs.sap.com/2021/11/17/sap-data-warehouse-cloud-sap-bw-bridge-overview-and-technical-deep-dive/
Data Warehouse Cloud Architecture Level 1
SAP Data Warehouse Cloud is deeply integrated with other SAP cloud solutions and relies on following three components:
- 1. SAP HANA Cloud
- Under the hood, SAP HANA Cloud powers the SAP Data Warehouse Cloud solution with in-memory data processing, advanced analytics, and data integration capabilities. Depending on the type of data source and the business requirements, it offers a variety of ways to collect and connect with data. There are several capabilities for virtualization or replication available. Third-party APIs allow external data movement tools like SAP Data Intelligence and others to connect and bring data out of a variety of sources into DWC A combination of these options is typically used to connect data sources to SAP Data Warehouse Cloud.
- 2. SAP Data Warehouse Cloud
- SAP Data Warehouse Cloud unifies data and analytics in a multi-cloud solution that includes data integration, database, data warehouse, and analytics capabilities for a data-driven enterprise. Built on the SAP HANA Cloud database, this software as a service (SaaS) empowers you to better understand your business data and make confident decisions based on real-time information:
- Connect data across multi-cloud and on-premise repositories in real time while preserving business context.
- Empower users with a virtual workspace and no-code environment to connect, model, visualize, and share data securely.
- Get insights on real-time data and analyze all types data with in-memory speed.
- Accelerate implementation with pre-integrated data and analytics capabilities.
- Reuse your existing BW models, transformations, and customizations with the BW bridge option.
- 3. SAP Analytics Cloud
- The consumption layer in SAP Data Warehouse Cloud is provided by SAP Analytics Cloud . You can create connections to one or more SAP Data Warehouse Cloud systems and then design SAP Analytics Cloud Stories on top of your analytical datasets or perspectives. For SAP Data Warehouse Cloud, SAP Analytics Cloud serves as the overall analytics layer with the prime responsibility to support all kinds of analytics enterprise-wide. In addition it is also possible to leverage 3rd party frontends and expose SAP Data Warehouse Cloud to them via standard SQL interfaces instead of using SAP SAC.
Data Warehouse Cloud Architecture Level 2
In general, SAP Data Warehouse Cloud offers multiple capabilities that address different personas - from Business Analysts with deep business understanding to tech-savvy Developers and Power users.
The Data Layer is the area where data engineers create their models with a technical approach, whereas the Business Layer is the area for business users who create their models using a more semantic approach. This allows business users to work independently from data engineers, while still being able to collaborate and share data with them. Data engineers can focus on the data consolidation and provisioning, while the business users can optimize the business models.
The IT infrastructure on the SAP Data Warehouse Cloud Data Layer is usually more volatile than the needs of the business user. The reasons for changes are multi-fold: system updates, software changes, acquisitions, etc. All this leads to the fact that the Data Layer changes permanently and needs adjustments. Business definitions on the other hand stay relatively stable. They do not change frequently, for example the calculation of the KPI margin. To keep the SAP Data Warehouse Cloud Business Layer more robust, while the Data Layer might be modified continuously, and the Business Layer and Data Layer are loosely coupled with each other only. The layers can be mapped and remapped at any point of time. Without the need of modifying the business models themselves, the data models can be upgraded and remapped to the business models.
The figure above provides a high-level architecture of SAP Data Warehouse Cloud with focus on following five components which are the bases for the modeling framework described above:
- Data Modeling Services
The "Data Builder" is the user interface to acquire and combine data to produce so-called Analytical Datasets, Dimensions and other semantic entities in SAP Data Warehouse Cloud. These can be consumed directly by SAP Analytics Cloud and other analytics clients, or passed to the Business Builder editors for further preparation.
- 2. Business Modeling Services
The "Business Builder" is the designated tool for modeling the objects of the Business Layer. Business users can define business models in the Business Builder, which are separate from the physical Data Layer. They create their models top down, and map them to the Data Layer. The Business Layer can be used to expose business users to the data fields they need while hiding any data fields that might not be relevant. Business users can create different business entities like dimensions and analytical datasets on top of the Data Layer. On top of these models, you can create fact models and consumption models as a basis to consume your data. On top of the consumption model, you can create perspectives, which can be used to analyze your data in stories within SAP Analytics Cloud. Authorization scenarios give the users access to the relevant data according to their role.
- 3. Business Catalog
The Business Catalog is accessed by the Repository Explorer which provides a central access point to all DWC objects in a convenient search-like format. There are sophisticated filter options, and it is possible to create new objects or perform various other maintenance actions on existing ones.
- 4. Space Management
Within SAP Data Warehouse Cloud, Spaces serve as independent work environments for individual departments, LOBs, data domains, project teams, or other user groups and individuals. They represent virtual work environments which enable the reuse of shared data and avoid maintenance of disconnected data sets. Spaces allow users to access core company data and connect own data from files and third parties. Within their space, the users are free to explore, model, analyze and visualize the data assigned to their environment and the results can then be shared with other user groups in their Space(s), finally.
- 5. Administration and Security
SAP Data Warehouse Cloud provides a large variety of standard functions which are required to operate a Data Warehouse in general. Some typical examples are: Space Management, User and Role Management, Security Control, Monitoring and more.
SAP Data Warehouse Cloud Functional View
The components previously described are key functions represented by corresponding UIs and tools. They are used by both IT users and business users who interact with each other:
- Managing Connections (IT)
Connections provide access to data from a wide range of sources, cloud as well as on-premise sources, SAP as well as non-SAP sources and partner tools.
- Working with the Data Builder (IT)
IT users work with the Data Builder to specify the SAP Data Warehouse Cloud Data Layer. This means they acquire and combine data and design Analytical Datasets, Dimensions and other semantic entities as a result.
- Leveraging the SQL Interface (IT)
There is a powerful SQL editor to query on source data where IT experts can choose between writing a standard SQL query using SELECT statements and operators such as JOIN and UNION, or use SQL Script to produce a table function. They can drag sources from the Source Browser, and specify measures and other aspects of your output structure in the side panel.
- Leveraging the Content Network (IT)
Business content for SAP Data Warehouse Cloud is available from the so-called Content Network which delivers preconfigured data models and reporting objects. This means the content consists of SAP Analytics Cloud content and the SAP Data Warehouse Cloud content which rely on each other, but technically can be installed independently. Before the standard SAC stories can be executed, a live connection to SAP Data Warehouse Cloud has to be setup.
- Leveraging the Data Integration Monitor (IT)
From Data Integration Monitor, IT users run and monitor data replication for remote tables, they monitor data flow runs or persisted views and track the queries sent to remote connected source systems.
- Setting up Access (IT)
Security has always been an important element for the complete product life cycle of all SAP products, including product development, planning, and quality assurance. Like the other SAP products, SAP Data Warehouse Cloud is designed to fulfill the highest security standards which guarantee the safety of your data both from web attacks and from attacks in the cloud.
- Exploring the DWC Catalogue (IT and Business)
The Repository Explorer is used by both user groups to search for DWC objects and access them to consume or change them.
- Working in DWC Spaces (IT and Business)
As described above already, SAP Data Warehouse Cloud Spaces are the key virtual work environments for both IT and business stakeholders. Spaces are decoupled, but are open for flexible access, thus allowing all users to collaborate without having to worry about sharing their data. Almost all SAP Data Warehouse Cloud objects are related to a space. During saving a space, it is stored in the SAP Data Warehouse Cloud repository, which contains the design-time definitions of all your spaces. During space deployment the corresponding SAP Data Warehouse Cloud run-time versions are created.
- Working with the Business Builder (Business)
The Business Builder is the designated tool for Business users modeling the objects of the Business Layer. They can define business models, which are separate from the physical Data Layer provided by IT. So Business can create their models top down, and map them to the Data Layer. The Business Layer can be used to expose business users to the data fields they need while hiding any data fields that might not be relevant. On top of these models, fact models and consumption models provide a solid basis to consume the data in SAC.
Accelerate the data-to-insight value chain is a priority for many organizations. Often the mere task of making the necessary technology components work together is daunting. Different interfaces, different security profiles, incompatible data formats, and metadata are just a few of the obstacles data and analytics experts must overcome.
Key takeaways of this unit
SAP BTP and its data-to-value process based solutions are essential to getting maximum use from your gathered data. This unified, cloud-based, data and analytics solutions includes data management, database and modeling, and visualization tools. This allows you to get started quickly and scale to accommodate your changing needs. Under the hood, SAP HANA Cloud powers the solution with in-memory data processing, advanced analytics, and data virtualization capabilities. The powerful data orchestration capabilities of SAP Data Intelligence (Data Flows) are also included and pre-integrated. As are the storytelling and visualization capabilities of SAP Analytics Cloud. Business accelerators further shorten time to outcomes, by providing a growing portfolio of pre-built data models and visualization, to answer pressing business questions. Pre-built transformations and integrations to SAP and non-SAP data sources also help you improve efficiency. Open interfaces and standard support allow interoperability with third-party tools.