Name: SAP AI ; Defect Detection ; Inference / Predictions using Jupyter
Uploaded: 2023-9-7
Duration: 658 s

Share feedback

<p><u><strong>Planned maintenance of Practice Systems</strong></u></p>

Planned maintenance of Practice Systems

Why create an SAP account?

Enhance your learning experience

SAP Learning

Choose your subscription

Manage your subscription

Live Sessions

Hands-on Practice

Certification

My Learning

Ready to start learning?

<div><div><div><div><div>In this playlist by Tahir Hussain Babar (Bob), from the SAP HANA Academy, we use SAP Artificial Intelligence to build a Deep Learning Image Segmentation Model which will help us perform automated Defect Detection on images of products. In this video, we look at Inference by using Jupyter to look at a source image with our trained model to examine if there any defects (by using our training model). Base64 conversion is demonstrated, and we also look at analyzing Inference results and how the Endpoint URL works. We finish by testing if the outputted image file from our prediction is correct by decoding it from Base64. Lastly, we look at Input Masks, Ground Truths & Predicted Masks.<br><br>00:00 - Intro<br>00:51 - How to use a Deployed Model<br>03:05 - Analyzing local image files<br>03:46 - Converting images to base64 in Jupyter<br>04:57 - Preparing Model for Inference<br>07:09 - Analyzing Inference result & Endpoint URL<br>07:50 - Testing output image is encoded into Base64<br>09:23 - Analyzing Input Masks, Ground Truth & Predicted Mask in Jupyter Notebook<br><br>SAP AI in BTP: <a href="https://www.sap.com/products/artificial-intelligence.html" rel="nofollow noopener noreferrer" target="_blank">https://www.sap.com/products/artificial-intelligence.html</a><br><br>Please note:<br>                     This video was originally published on the SAP HANA Academy YouTube Channel                      and therefore may contain references to SAP HANA Academy and YouTube.                      The tech world is always evolving and the SAP HANA Academy YouTube channel                      is set to close on December 31, 2023.                      At that time, references will no longer be valid                      and will gradually be removed when the videos are updated.<br>Also, this video includes automatically generated captions                      and therefore might misrepresent the spoken content or display incorrect product names.</div></div></div></div></div>

SAP AI ; Defect Detection ; Inference / Predictions using Jupyter

Welcome to the SAP HANA Academy. My name is Bob, and in this video series, we'll be looking at SAP Artificial Intelligence. In this video playlist, we'll be building a deep learning model on product images for automatic defect detection using SAP AI Core. In this video, we're going to be going through the concept of inference, where we can make predictions using our trained model. Now in the previous video what we did is we executed a deployment of our trained model. Now I finished off the video with stopping my deployment. I've already started it so we've got the deployment re-running. And what we're going to do in this video is use our deployed machine learning model. So I'll create myself a header and then what we'll do is we need to set the scene of what we're going to do. So over the past 15 videos we've built a model which performs some deep learning. As we know, what it does is it makes a prediction on an image about whether the image has a defect on it. Again, the image comes from an assembly plant of light guide plates. And we know if there's a defect, we want to analyze if the product is damaged or if it isn't. And if it's damaged, if it's not damaged too much maybe we can sell it for 50% less as an example. So, what we need to do now that we've looked at a sample data set and we've trained our model, in the previous video we've deployed that model which essentially means we've made the model available on a web server so we can access it via a API and we can supply it with new images and it will tell us whether or not the image of the product has a defect on it or not, and also the amount, the size of the defect on the actual image as well. So to test the model what we're going to do is supply it with a new image. Now I'm going to be lazy, I'm going to get this image from my existing data set. So remember this is our source data set from our very first video. We have a series of images of our products which contain hundreds of little dots carved into the material. And sometimes those products have a defect as you can see within these images. And basically what our model can do is it can extract and enhance the defect. So what should be outputted is an image which looks similar to these which are in our mask folder. Remember these are our ground truth images which an expert previously assigned as a defect. So our expert assigned every pixel to one of two classes. Either 1 being a defect or 0 being no defect. As you'd imagine, if you detect a dot, this is considered normal, so the assigned class would be 0, so it would be no defect. So as you can see with all these images, they don't consist of lots of little dots. So they would be assigned the class 1 for each of these pixels, which would indicate a defect. And as you can imagine, we've used this data to train our model. Again, the other two folders just show products that have no defects on them. So to do some analysis, what I'll do is I'll firstly point to where those local files are using the following command. So they're all on my local machine, I'm not accessing that S3 bucket and I'll run this command. And as you can imagine, in each folder we have the same version of the file. Some of them have no defects, some of them have a defect. So for example here I'm printing out the input image, and then the actual ground truth, which is all of those tiny dots removed, and the defect enhanced. So again as you can imagine what we want to do is do the same but for a new data set, so for a new image. So how do we read images? Well to do this we need to transform one of our images into a binary string and then this will constitute the body of our API call. There's four steps. The first step will read the first file in our folder which will contain different images of our products. We'll then convert these bytes to base64, then we'll decode the base64 to text, and then the last step basically wraps this in JSON format with curly brackets, and the word image preceding this string. So I'll execute the command, and if we scroll down and have a look at the error message, it seems that I don't have the correct library. So I'll just put that library in here and re-execute. So the output from this little command will be a dictionary called rawdata, which will have our image decoded to a base64 string. So now that we've got this string what we can do is create a prediction. So as you can imagine, the input for our predictions, or to do inference, is that base64 conversion. So if we look at the code that actually does the prediction, of course, as we know we'll be accessing this from a web server, which has been deployed on our serving container in Docker. And this is going to be the endpoint that we're going to access. We'll need the endpoint, so I'm printing the endpoint out. And if we look at the format it will be the deployment URL, which is the same as really as SAP AI Core. And what will determine the name used here in the endpoint is how the web application is defined inside the infer.py file. Now we'll also need to include some headers. One of them is authorization. We need to have access to our SAP AI Core. And as you can imagine, we're going to use that new base URL that we created. My first lecture will be on 4-bit programming. And here you can see that base url ai.api.lm. Next we'll need the resource group. So our resource group is image and we'll need to provide the header which is ai.resource group as well as a content type. Which means we'll expect the data in JSON format. We'll also output the inference result which will indicate how much of the image has a defect. Again, we're going to also output the endpoint because we'll need it in Mailman. So if I run the command, this is the result. So the command in blue is the actual endpoint, the full endpoint, that again we're going to use with mailman later on. Then we have the inference result, the defected area within the image, which is a number, and then of course the segmented image, which means that the model inferred which pixels belong to the defect class, and then it assigned one to them and zero to the others. So how do we know it actually works? Well, what we can do is actually decode this string or this Base64 back to an image. There's lots of tools out there on the web for you to be able to do this, but we can use a tool called Base64 Guru. So I'll launch a new tab and I'll do a search for Base64. The one I use, there's lots of different ones, is Base64Guru. And what you'll see here is that we can convert, do many different types of conversion. So we can convert an image to Base64 from our actual file, or of course we can do the reverse. So if I scroll down on the left hand side, what we can do is choose Base64. And then all I have to do is copy the text from the output in Jupiter. So I'll grab all this text. And then I just need to paste this text back into my Base64 converter here, and select decode Base64 to image. And as you can imagine, the image shows us which pixels correspond to a defect in the never seen like guide plate image we provide as input to the model. This script can also be done directly here in Jupyter, so I'll show you how to do that before we move onwards to Mailman. So we're going to use a method called load image, which will be used to convert the prediction that has returned as a binary string. Once the predicted mask is converted, we can then visualize it. So as well as OpenCV, we're using something here called contrast limited AHE, and that basically amplifies things and also reduces image noise. So for us, that noise would be the little dots. The next command is going to do some conversion, because we're dealing with binary data as well as Base64 data. And then of course the last step is to present it to the screen. And that will be our input image, our ground truth and the predicted mask. And this is what the result looks like. So of course the first image is our original input image of the defected device taken from the camera. The second one called ground truth is the mask that tells us exactly which pixels represent a defect. And of course this has been specified by a human who knows the products really well. So we'll know what a defect is and what is not a defect. And lastly we've got our predicted mask, which is what our model predicts. Now we'll include the mask because it's interesting to compare it with what is predicted by the model. Just to see ourselves visually that the model is behaving well on these images. We've presented these three images within a Jupyter notebook but in the next video I'm going to show you how to do something similar using Mailman.