Name: Data Intelligence 3 of 21 ; Publishing & Profiling
Uploaded: 2023-9-6
Duration: 395 s

Share feedback

<p>Planned maintenance of Practice Systems</p>

Planned maintenance of Practice Systems

Why create an SAP account?

Enhance your learning experience

SAP Learning

Subscribe now

Learn more

Live Sessions

Hands-on Practice

Certification

My Learning

Ready to start learning?

<div><div>In this video series, we'll introduce some of the applications when using Data Intelligence. In this video, we'll go through using the Metadata Explorer to publish data (which enables you to search the metadata, add comments to the objects, and tag datasets) as well as profile data (which helps you learn more about your data, for example, you can see if there are null or blank values, distinct and unique values, minimum and maximum and average length values).<br><br>0:00 - Introduction<br>0:40 - Dataset metadata<br>1:00 - Publication<br>2:30 - Browse catalog<br>3:00 - Start profiling<br>4:00 - View fact sheet<br><br><br>Please note:<br>                     This video was originally published on the SAP HANA Academy YouTube Channel                      and therefore may contain references to SAP HANA Academy and YouTube.                      The tech world is always evolving and the SAP HANA Academy YouTube channel                      is set to close on December 31, 2023.                      At that time, references will no longer be valid                      and will gradually be removed when the videos are updated.<br>Also, this video includes automatically generated captions                      and therefore might misrepresent the spoken content or display incorrect product names.</div></div>

Data Intelligence 3 of 21 ; Publishing & Profiling

Welcome to the SAP HANA Academy. My name's Bob, and in this series of videos by Digital Partner Engineering and the SAP HANA Academy, we'll be looking at Data Intelligence 3.1. In this video, we'll be looking at both publishing and profiling data within the Metadata Explorer. In the previous video we looked at using the Metadata Explorer to import two files, a parquet file and a CSV file into our semantic data lake. So, once we've imported them, if we select each individual file, on the right hand side you have options to view metadata about those files. So for example, if I select the parquet file, if I select the metadata tab on the right hand side, we can see when the file was created, how big it is, where it's stored, and then any related objects to do with those files. Now we're going to look at publications in this video in terms of profiling. Now the reason we publish our data set is to make a local copy of the metadata. So after the data is published, we're able to search the metadata and then we're also able to add comments to the objects and tag data sets. We can also do things like data lineage which will help us understand where the data originated from. So to set this up, all we need to do is select the file that we want to do the publication on, and drag it to the right hand side. We can perform these publications on multiple data sets at the same time. So I'll also select the CSV, and there we can see we've added both of those files to this publication. Then all we need to do is give the publication a name, so I'm going to call this one S.H.A. We can enter a description, all I'm going to do is enter the description S.H.A. underscore group. And lastly, all we need to do is, if we want to include subfolders, we can. We don't have any, so all I'm going to do is select Publish. We can see that we've started that publishing task, and again we get an alert in the top right hand side that that publishing has started. The process should be relatively quick for us because these are small files, but we also have a monitor tile where you can view the progress of your published datasets. So, to view the results, all we need to do is go to our Metadata Explorer, and within Catalog we can select Browse Catalog. Again, we'll be presented with a list of our connections. So, on the right hand side we can see that we've got an alert, and if I select the alert we can see that the publishing has completed. My lecture is completed. Again, it only took a few seconds. So now all I need to do is of course select my connection. If I scroll down and choose that semantic data lake, we'll select the shared folder, the SHA folder, and here we can see those two files available to us. Our publication makes the metadata available to us, but we've not profiled it yet. To profile the data we need to select the three dots and we've got some options. As well as profiling we can do other things for example, we can view where the file is, we can view the metadata, we can download the data. Of course we're going to profile that data. Here we can see we've started that profiling for the Parquet file. And I'll do the same thing for the CSV file. Of course we get those two alerts that the profiling has started, which I can clear. And then we just need to wait a few seconds for the profiling to be completed. Again, as they're relatively small files, it only takes a few seconds for those two files to be profiled. If I refresh the page, we can see that indeed those two files have been profiled. So now we want to look at the details. So to do that, again, if we click on those three dots, we've got the option of View Fact Sheet. So I'll select View Fact Sheet for the parquet file first, and then what we'll see are the data set properties. So firstly, we'll get an overview of the file, whether it's been published or profiled. We'll see the number of columns and rows, and then if I scroll down, what we'll see are some other analytics, such as column types, profiling trends, and also any glossary metrics which we're going to set up in a few videos' time. If people rate or review the data, or make any comments, you'll also see those comments at the bottom of the screen. We've also got an additional tab which of course looks at the columns within that data set. So if I select columns, as well as the typical information on data type and length of that data type, you have minimum and maximum values, averages and information on distinct values. Of course, we've also got that third tab which enables us to actually view the data. So, if I select data review, we've got information about the data set. Again, there are some data quality issues with the data, and if we look at country, all of the countries are in the United States, but you can see that some of them are US, USA, U.S.A, so on and so forth. Now, there are lots of other issues with this data. If we just look at one example, if we go back and have a look at the zip code column. So if I scroll down we've got all the information of course on whether or not those values are distinct and the actual values. If I go down to zip code and select the arrow on the right hand side and scroll down what we'll do is we'll see a preview of those individual values for that column but if you look in the distinct values you can see the six percent have no value. So, of course, we're going to want to fix that issue. Now, as well as detailed information on each of the columns, we also have a relationships tab, and we'll look at how to use that in the next video. You can find more tutorials on our YouTube channel, and if you'd like to be informed as soon as new video tutorials are published, then please subscribe. Thanks for watching.