Name: SAP Data Intelligence ML Scenario Manager (02) - Connections and Metadata Explorer
Uploaded: 2023-9-4
Duration: 496 s

Share feedback

<p>Planned maintenance of Practice Systems</p>

Planned maintenance of Practice Systems

Why create an SAP account?

Enhance your learning experience

SAP Learning

Subscribe now

Learn more

Live Sessions

Hands-on Practice

Certification

My Learning

Ready to start learning?

<div><div><div><div>SAP Data Intelligence helps manage your data across different systems by using the Metadata Explorer. The Metadata Explorer gathers information about the location, attributes, quality, and sensitivity of data. With this information, you can make informed decisions about which datasets to publish and determine who has access to use or view information about the datasets. <br><br>In this second of a nine part video series regarding the SAP Data Intelligence Machine Learning Scenario Manager, Tahir Hussain Babar (Bob) introduces the source data used in the series, by using Connection Management to show the connection to the SAP Semantic Data Lake and using the Metadata Explorer to profile the data. <br><br>0:00 - Introduction<br>0:40 - SAP Data Intelligence Launchpad<br>1:15 - Connection Management<br>2:45 - Metadata Explorer<br>3:20 - Profile data set<br>5:45 - Monitoring<br>6:15 - Result<br><br><br><a href="https://sap.to/6052KrmOr" rel="nofollow noopener noreferrer" target="_blank">https://sap.to/6052KrmOr</a><br><br>Please note:<br>                     This video was originally published on the SAP HANA Academy YouTube Channel                      and therefore may contain references to SAP HANA Academy and YouTube.                      The tech world is always evolving and the SAP HANA Academy YouTube channel                      is set to close on December 31, 2023.                      At that time, references will no longer be valid                      and will gradually be removed when the videos are updated.<br>Also, this video includes automatically generated captions                      and therefore might misrepresent the spoken content or display incorrect product names</div></div></div></div>

SAP Data Intelligence ML Scenario Manager (02) - Connections and Metadata Explorer

Welcome to the SAP HANA Academy. My name's Bob, and in this video series, we'll be looking at SAP Data Intelligence. In this video series, we'll be focusing on using the Machine Learning Scenario Manager. And what we'll cover is how to create predictions within the Machine Learning Scenario Manager and then expose them so that they can be accessed via an API. In this first video of the series, we'll be looking at the data source, and we'll be using connection management and also the Metadata Explorer. The first thing we'll need to do is log into our Data Intelligence Launchpad, and this is the entry point for all your applications. So, we'll need to select a tenant, then when we've selected a tenant you can use one of the users as given to you by your SAP Data Intelligence Administrator or you can use SAP XSU AA. I'm going to log in with one of those users. Now this launch pad is your starting point for when you create your content, and depending on your version of SAP Data Intelligence and your access rights, you might see a different selection of tiles on this entry screen. We'll start off by using the connection management tile, and it's used to connect to a wide range of data sources. So here you can see a list of our connections. We're going to be using a specific connection which is called di underscore data lake. You'll see it's of type sdl which stands for semantic data lake. And this connection is managed by sep and can't be modified. Now what we'll have to do firstly is test this connection. So I'll check the status of the connection. It should work, but we're just verifying that we can access this DI data lake so we can store our CSV files within this connection. In terms of other connection types, I'll just quickly show you the types of things that we can connect to within SAP Data Intelligence Cloud. So you can see there's lots of connections to SAP products, as well as things like Google BigQuery. Today I'm going to talk a little bit about how you can connect to various databases like MySQL, you can use different data types like OData and Kafka, as well as other transportation mechanisms such as SFTP and SMTP as well. You can also connect to various storage mechanisms such as HDFS, Amazon S3 buckets, Alibaba storage systems and Google. Now again, our data is within our semantic data lake, within that DI data lake connection and we can access the data using Metadata Explorer. We can use the Metadata Explorer to publish, profile, prepare and monitor your metadata. Now what you'll see is that the Metadata Explorer homepage contains some cards to help you navigate to your desired area. Whether you want to search for data on a specific connection or create what are called data profiles. So for us, our data is in that DI data lake connection. So I'll select the link, we'll go to the shared folder, we'll select scenarios, and within that we've got a folder called Marathon. Within the folder we've got a file called runningtimes.csv which is going to act as our source data. It's a small CSV file which contains the running times in minutes of around 117 people. So if we look at this data, here we've got an overview of that data. We can see at the moment we know we've got three columns, but we don't know how many rows. We can see that above as well. Now, the reason we don't know how many rows is because we haven't profiled the data. Profiling produces extra metadata about the values within your data set. For example, you can view unique or distinct values, min and maximum lengths, if there's any nulls, so on and so forth. If I do a data preview, we can see that we've got three columns. Now our data again is a small CSV file which contains the running times in minutes of around 117 people who ran two marathons, a half marathon and a full marathon. The first few rows show the fastest runners at the top. In terms of the columns, we've got an ID. The next column is the number of minutes it took to complete a half marathon for this individual. And the third column is the number of minutes it took to complete a full marathon for this individual as well. So we know we've got those three columns, but we don't really have that much metadata apart from knowing the names and the number of these columns. Again, to get that information we need to perform a profile. When we perform this profile, then we'll have extra metadata on each of the columns that we've got. So to start the profiling, there's a little icon on the top right hand side. It'll say start profiling, so we'll select it. The profiling will start and we'll be given a notification that it started. We'll see an alert on the top right hand side that it started because depending of course on the data size this could take quite a long time. Now my data set is only 117 rows so it won't take that much time. But we do have the option of looking at a monitor tool, which is something that an administrator would do. So if I go to my monitor and we look at the monitoring of these tasks, here we can see that we've actually got two tasks running at this moment. You can see that the task is running, the user that started the task and also of course how long it takes and when it's completed. So this just should just take a few seconds to complete and of course what we'll see is that we've got a notification that the profiling task has completed or, for example, if there's any errors. If I go back to my folder or my buckets within the Metadata Explorer where the file is, we'll see a few things. Firstly we'll see that the data has been profiled as you can see here. We know now that we've profiled a number of rows, so you can see that we've profiled those 117 rows. And of course we get the notification on the top right hand side as well for the end user. Now what you'll see here is that we've got some extra metadata, such as minimum and maximum values within each of those columns, as well as a list of distinct values as well. So in our example, in terms of the half marathon minutes, we can see that the fastest person would have taken 73 minutes to run the half marathon, and the slowest person took 141 minutes to run that same half marathon. Again, because we've profiled this data, we can go deeper into each of the columns. So if I select the half marathon minutes and scroll down, we've got a data preview of that specific column as well as various column properties which you can see on the right hand side. We can also see the top 10 distinct values. So this indicates that around 7% of people ran the race in both 104 and in 89 minutes. So now that we've looked at our data set within that bucket within our semantic data lake, and we understand the basics of the three columns in that data set, what we're going to do in the next video is look at a machine learning scenario. Using profiling to look at the metadata is really useful, but if you want to do some serious statistical analysis, these machine learning scenarios really help you better understand your data and gain valuable insights into that data.