When working with SAP AI Core, you are likely to encounter following questions:
- How does data from my S3 connect to my SAP AI Core?
- Where is the model generated from training process of SAP AI Core stored in my AWS S3?
- How to locate the stored model?
In this blog you will
- Understand artifacts of SAP AI Core with example
- Locate Models in you AWS S3 generated via SAP AI Core
Solve mystery of artifact
Lets understand artifact in relation with your dataset and then we will generalize it later.
The goal in this section is not to create an artifact but to understand what information is mentioned while creating an artifact for a dataset.
To understand how dataset is referenced in SAP AI Core, I will walk you through from the end-result and back to your AWS S3 folder where you data will be located. So, here’s a peek of your end goal.
In the tutorial of SAP AI Core you uploaded data in your AWS S3, following will be the view of your AWS S3 data folder.
- This is a snippet you must have encountered in the tutorial of SAP AI Core Training Step 2 to register your dataset as an artifact.But what information are you writing here?
The url is referring to the folder in you AWS S3, where your data is present.
This argument partially tells the story, as you would naturally raise the question that”my data folder somewhat looks like tutorial/data in my AWS S3″This is answered in our following observation.
- But how does SAP AI Core know which AWS S3 is referred when creating artifact and how is it getting credentials?You must have connected your AWS S3 in the tutorial on Connect AWS S3 Object Store to SAP AI Core Step 5Observe the following snippet you used in the same process.The name: default, is same as what you mentioned in url parameter when creating an artifact. So this answers the question the artifact is referring to which AWS S3 credentials.
The pathPrefix: tutorial is folder preceding your folder named “data” in your AWS S3. And this answers the question of path prefixing the exact folder you want to use as an dataset when creating an artifact.
The following diagram will always help you as a side note on how to register your datasets or models when working with SAP AI Core.
Now that you got hands-on an artifact, that the following definition will solidify your understanding.
Artifact is reference to files stored in hyperscaler data storage (such as AWS S3), including the instruments to access and explore it (such as the path to the AWS S3 Object Store bucket). Each artifact is uniquely identified by an ID.
What makes an artifact identified as dataset ?
It’s the parameter kind when registering your dataset as an artifact.
The value of kind parameter is just a qualifier which implies artifacts whether datasets and models are merely referring to your AWS S3 folder.
This opens up the possibility to skip the training in SAP AI Core and to store your pretrained models (generated locally) to AWS S3 and register as an artifact to use for deployment in SAP AI Core.
Find more options for value of kind in api.sap.com for SAP AI Core
Locate models generated from execution/ training
Let’s put our understanding of artifact to test.
- List all the artifacts for your SAP AI Core.
Use your own value for AI-Resource-Group.
- Find id of the artifact generated as output artifact (model) of execution. The id you will get from the training process. See Training SAP AI Core Step 5
- Locate the url of your generated model.
- Find your object store secret with name “default” because “default” is followed by ai:// (see step 3 above).
- Navigate to folder named tutorial in you AWS S3. Because this is the folder name pointed by pathPrefix in Step 4 above.
- Navigate further as your mentioned in the url of model artifact (Step 3 above).
You will find the file generated out of training process. Here the file format is PKL stands for pickle, a python file serialized object file.
Why is the model folder in AWS S3 named text-model-tutorial?
This is because your YAML using in training mentions that name.
That’s all folks.
Feel free to request me more content on the intricacies of SAP AI Core with an example.