Name: Data Intelligence 15 of 21 - Validation Rules
Uploaded: 2023-9-6
Duration: 422 s

Share feedback

<p>Planned maintenance of Practice Systems</p>

Planned maintenance of Practice Systems

Why create an SAP account?

Enhance your learning experience

SAP Learning

Subscribe now

Learn more

Live Sessions

Hands-on Practice

Certification

My Learning

Ready to start learning?

<div><div><div>In this video series, we'll introduce some of the applications when using Data Intelligence. In this video, we'll go through using the Modeler to perform some data validation. Your can create rules and route records that pass through a pass output port, and also have route failed records through a fail output port.<br><br>0:00 - Introduction<br>0:45 - Validation Rule operator<br>4:30 - Save and execute<br>5:15 - View wiretaps for failed rules<br><br><br>Please note:<br>                     This video was originally published on the SAP HANA Academy YouTube Channel                      and therefore may contain references to SAP HANA Academy and YouTube.                      The tech world is always evolving and the SAP HANA Academy YouTube channel                      is set to close on December 31, 2023.                      At that time, references will no longer be valid                      and will gradually be removed when the videos are updated.<br>Also, this video includes automatically generated captions                      and therefore might misrepresent the spoken content or display incorrect product names.</div></div></div>

Data Intelligence 15 of 21 - Validation Rules

Welcome to the SAP HANA Academy, my name is Bob and in this series of videos by Digital Partner Engineering and the SAP HANA Academy, we'll be looking at Data Intelligence 3.1. This video shows how you can create validation rules when using the modeler. In the previous video, we built this graph within Data Intelligence Modeler, and what it does quite simply is it reads from a table within HANA Cloud, and it outputs the data in CSV format. And we can see that data here when we're modifying the graph in design time. Again in this video we're going to be looking at validation rules. So what I'll do is I'll search for that validation operator and I'll put it onto the screen. So the output of the wiretap can now be an input into this validation rules. And again, the wiretap just lets us see the data in CSV format. To configure the operator, we right click and go to open configuration. Of course, we've also got the documentation that's available to us. So if I right click, we can go to open documentation. And as it says, the validation rule creates rules and records that can pass through the pass output port. So you have the concept of pass and fail. So to demonstrate this, if we go back to the configuration, what we'll need to do is look at the input schema. So we can add that schema based on that CSV. So I'll add a column and that first one will be date. The type will be string because remember the data is coming to us from a CSV. Now with the magic of video I'll make adding all the rest of these columns very quick. So when you've added the six columns from that CSV we just need to click on save. To review them, if we go back to each item, item 1 is a string, 2 is the ID, and then the rest of the columns are keys. So I'll click on save. So now that we've set the input schema, what we need to do is define the rules. So we can add the rules here. So the rules are quite simple. We're going to base them on values. So we're going to add a rule which says the first one is that key 1 must be less than 130. And the fail action will be fail. And the second rule will be that key 1 must be greater than 70. And again the fail action is fail. Now of course I can apply multiple rules to multiple columns. I'm only doing it to one column to demonstrate how it works. And it might look a bit counter-intuitive, but the rules actually mean that all values greater than 130 or less than 70 fail. So this doesn't mean if the key is greater than 70, it actually means if it's less than 70, it will fail. It's kind of the reverse. So all I need to do now is select save. So now we can see we've got two rules and we've got six items in our input schema. So now of course what we want to do is see the output. Now what you'll notice is that we've got more output ports coming out of the validation rule. So to see the values we're going to use a wiretap. So again I'll drag wiretaps onto the screen. And what I'll do is if I move my mouse over the validation rules you'll see we've got pass which we're going to put into this wiretap. And if I add another wiretap, we'll put the fail into this wiretap. You've also got information to do with error message for each of those separate ports. Again, by going to the documentation you've also got more information on exactly how this validation rule operator works, including lots of detailed information on the ports right at the bottom. So those last two ports deal with errors and fail information. So of course now what we've done, we've finished, of course we'll want to test this. So what I'll do is I'll save the graph. We've got confirmation below that we've saved the graph, we can actually clear those messages. So I'll clear all the validation messages, and we'll also clear the log. And then of course we'll run the graph. If we go to the status we'll see it's pending. Of course we won't see the open UI link while it's pending, it needs to be in running status. So now we can see it's running so we can access the wiretap by right clicking and selecting open UI. So here we can see all the pass results, all those that match that certain threshold. So here the key one is the first key here, so again this has to be below 130 and above 70, which all of them look like they are. And of course to see the other wiretap we can just select it here in the tab to see our failed rules. So we've got two failed rules. You can see that with the comments failed rule. You know which column it's failed on. And if I go to the right of the cell ID, here we can see the first failed rule has a value of 65, which of course is less than 70. And if we look at the second failed rule, which is the next row, and we go to the right of the cell ID, we can see it's 133, which of course is above 130. And that is why it's the second failed rule, and that is indicated by 1 and 2. So now that we've validated that validation rule operator and we've looked at the pass and fails, we can stop that graph from running. In the next video we're going to look at using a Python 3 operator. So I'm going to demonstrate what we can do to the data now that we've got it in CSV format, with another type of operator where we can add some code. And of course that code will be in Python format. You can find more tutorials on our YouTube channel and if you'd like to be informed as soon as new video tutorials are published, then please subscribe. Thanks for watching.