Before you begin creating your first data integration job, it's essential to understand the core architecture of SAP Data Services.
The platform is not a single program but a collection of interconnected components that work together to design, store, and execute data workflows. The entire process involves a client-side design tool, a repository for storing metadata, and a server-side engine for execution.
The Data Services Designer
The Data Services Designer (referred to as the Designer) is a Windows-based client application used to create, test, and manually execute jobs. These jobs are designed to transform data and load it into a target database or file system.
Using the Designer, you can develop data management applications that include data mappings, transformations, and control logic.
You can define objects that describe data sources and then use a drag-and-drop interface to configure them within flow diagrams.
The objects you create are stored in a local repository. From the Designer, you can trigger the Job Server to run jobs for initial application testing.
Watch this video to learn more about the Data Services Designer user interface:
The Data Services Repository
There are a few prerequisites to complete before you can use the Data Services Designer to create and execute Extract, Transform, and Load (ETL) jobs. While configuring these components is typically the responsibility of ETL administrators, understanding their function is also beneficial for ETL designers.
One essential component is the local repository, which consists of a set of database tables that store the objects created in the Designer. You can create these repositories during the SAP Data Services installation or at any time using the Repository Manager.
The local repositories support the most widely used Relational Database Management Systems (RDBMS) on the market. Note that local repositories do not need to reside in the same database system as the source or target data. For example, your source data could be in Hadoop while your local repositories are hosted in SQL Anywhere.
The Data Services Job Server
Each repository is linked to at least one Data Services Job Server.
When you execute a job, the Job Server retrieves the job definition from its associated repository. It then initiates the Data Services engine processes to perform data extraction, transformation, and movement. The Data Services engine uses parallel processing and in-memory transformations to ensure high data throughput and scalability.
Summary
In this lesson, you learned about the main SAP Data Services tools.
- SAP Data Services Designer is a Windows-based client for creating, testing, and executing ETL jobs using drag-and-drop interface for data transformations.
- The Local Repository is a database storing Designer objects. It supports multiple RDBMS platforms independent of source or target systems.
- The job server and engine retrieves job definitions from repository and executes parallel processing for high-performance data extraction and transformation.