Automatic Payment Reminder with S4/HANA OnPrem and SAP Conversational AI with Telegram

SAP S4/HANA, Cloud Application Programming (CAP) Model, Kyma and SAP Conversational AI (CAI) – these are all popular services and tools used by developers on the SAP Business Technology Platform (BTP). However, have you ever integrated all these components to build a chatbot powered by S4/HANA data?

In this blog post, I will share my recent experience of using these aforementioned technologies to build an Automatic Payment Reminder Chatbot prototype.

Project Purpose

The aim of this project is to create a chatbot using CAI that connects to Telegram and utilizes data from S4/HANA. Upon subscription to the bot, the user can request for outstanding or overdue invoice(s). When there are overdue invoices, the bot will also send a regular reminder to subscribed users to view their invoice details or request for payment details to resolve their open invoices.

Prototype Architecture

The Job scheduler runs at regular intervals and fetches data from the S4/HANA on-premises backend system for relevant open invoice information. This information is stored in the SAP HANA Cloud Database (HANA DB) and acts as an abstraction layer between the chatbot users and the S4/HANA system. The relevant information can then be accessed using the HANA DB directly instead of fetching the information from the S4/HANA each time a user request is made.

CAI interacts with the Telegram user to send automatic reminders and respond to any customer queries related to invoice and billing.

Use Case Description

During the creation of this chatbot prototype, we considered these use cases. The following sequence diagrams describes the communication flow in each scenario.

  1. New customers subscribe to the chatbot and request to view all invoice:
  2. Subscribed customers request to view particular/single invoice:
  3. Send automatic reminder to subscribed customers who have outstanding/overdue payments:

    Additional Considerations

    Customers can unsubscribe from the chatbot to stop receiving automatic reminders:

Implementation Details

The implementation steps to this project can be broken down into 4 major segments:

  1. Extract invoice data from S4/HANA OData services
  2. Use the CAP application to process authentication, domain logic and also store extracted data in HANA DB
  3. Deploy CAP application to the Kyma runtime to expose an API for CAI’s consumption
  4. Configure CAI to build skills and consume APIs

S4/HANA and CAP Application Interaction

The invoice data resides in S4/HANA on-premises. We used the CAP application to connect to the OData service in order to extract this data. Thereafter, we stored this data in our local HANA DB and deployed the application to the Kyma runtime in order to expose an API to be consumed by the Cron job and CAI.

Firstly, we followed Step 1 of this tutorial to create a CAP-Based Application after fulfilling the 2 prerequisites stated in it. Once the CAP application was set up, the next step would be to consume the external OData services from S4/HANA.

This next tutorial helped us to add an OData service to the CAP Application. However, since we were using S4/HANA on-premises instead of S4/HANA Cloud, we did not use the edmx file as demonstrated in the tutorial. Instead we created our own edmx file by simply adding a $metadata tag to our on-premises OData link and then saved the xml data as an .edmx file. Step 2 of this blog gives a clear demonstration to create your own edmx file.

Appending%20the%20%24metadata%20tag%20to%20create%20edmx%20file

Appending the $metadata tag to create edmx file

After adding the edmx file and generating the csn file in the “external” folder within services, we added the OData services into package.json file as shown below (CI and BP).

"cds": { "requires": { "auth": { "kind": "basic-auth" }, "CI": { "kind": "odata-v2", "model": "srv/external/CI", "credentials": { "url": "https://apjiodl.prod.apimanagement.eu10.hana.ondemand.com/v1/payment_reminder_customer_invoices/", "authentication": "BasicAuthentication", "username": "", "password": "" } }, "BP": { "kind": "odata-v2", "model": "srv/external/BP", "credentials": { "url": "https://apjiodl.prod.apimanagement.eu10.hana.ondemand.com/v1/payment_reminder_customer_company/", "authentication": "BasicAuthentication", "username": "", "password": "" } }, "CAI": { "kind": "rest", "credentials": { "url" : "https://apjdl.sapcai.eu10.hana.ondemand.com/public/api", "requestTimeout" : 30000 } }, "db": { "kind": "hana-cloud", "pool": { "max": 1400, "min": 0 } } }, "hana": { "deploy-format": "hdbtable" } }

Next, not all the data in the OData service are relevant therefore we extracted only the fields we wanted from the service. This was be done with a srv_schema.cds file in the db folder.

using { BP as bp } from '../srv/external/BP.csn';
using { CI as ci } from '../srv/external/CI.csn'; entity Customer as projection on bp.A_CustomerCompany{ key Customer as Customer_Code, CompanyCode as Company_Code
}; entity Invoice as projection on ci.Items{ key Customer as Customer_Code, CustomerName as Customer_Name, AmountInCompanyCodeCurrency as Amount, CompanyCodeCurrency as Currency, DueNetSymbol as Overdue_Status, NetDueDate as Due_Date, InvoiceReference as InvoiceRef
};

At this time, we had also connected to our database to get ready to store the retrieved data from S4/HANA in our abstraction layer. During development, we initially connected to SQLite as shown in the tutorial before deploying HANA DB on SAP BTP for production. We followed this documentation to connect the CAP Application to HANA DB.

Entity-Relation%20Diagram%20of%20the%20DB

Entity-Relation Diagram of the DB

We accessed the information from the OData service via the JS file within the srv folder. The following is a function that connects to the service as well as the HANA DB. It then reads the customer information from S4/HANA and updates the “Customer” table in our deployed database.

module.exports = cds.service.impl(async function() { const bp = await cds.connect.to("BP"); const db = await cds.connect.to("db"); const { Customer } = db.entities('sap.capire.payment_reminder'); this.on('Trigger', async req => { const bpResult = await bp.run({ SELECT: { from: {ref:['Pull_Data_Service.Customer']}, orderBy: [ {ref:[ 'Customer' ], sort: 'asc' } ] }}) await DELETE.from(Customer) const bpArr = []; bpResult.forEach(obj=>{ bpArr.push({ Customer: obj.Customer, CompanyCode: obj.CompanyCode, }) }) await INSERT.into(Customer, bpArr) return {msg : 'Updated DB Successfully!'}; })

This function is triggered daily by the Cron job to refresh the data in the HANA DB and also send the regular reminders to our subscribed customers. The completed CAP Application would then be deployed to Kyma as demonstrated here.

Cron Job Integration

Why do we need a Cron Job?

As mentioned above, this payment reminder solution has a feature which sends daily reminder notifications to customers whenever they have any pending payment to made. Therefore, there is a need for a tool to regularly trigger sending of reminder notification to customers. According to this requirement as well as by considering the feasibility with our runtime environment, Kyma, we have decided to implement Cron Job for this solution.

How did we implement the Cron Job?

Since we already had an API exposed from our backend CAP Application to trigger this automatic messaging feature, all we had to do was to implement a Cron Job in our Kyma deployment to call this API regularly.

As shown in the below screenshot of the deployment.yaml file, we added in a new section for the Cron job where the URL of the API request and the UTC time to call this API were indicated. In this configuration, we indicated ‘0 4 * * *’ for schedule which means the time for the Cron job to call the API is at UTC 4 am. Next, we used ‘curl’ as the Linux command to call the implemented API.

Then, this Cron Job configuration was deployed to Kyma and at every 4 am UTC time, a new job is created to execute the curl command which calls the API to trigger the automatic notification.

You may find out more regarding different time setting of Cron Job from here.

apiVersion: batch/v1
kind: CronJob
metadata: name: upl-job labels: app: upl-app
spec: schedule: "0 4 * * *" concurrencyPolicy: Allow successfulJobsHistoryLimit: 1 failedJobsHistoryLimit: 1 jobTemplate: spec: template: spec: containers: - name: upl-job image: curlimages/curl:7.81.0 imagePullPolicy: IfNotPresent args: - /bin/sh - -ec - "curl https://upl.c-290ae5b.kyma.shoot.live.k8s-hana.ondemand.com/service/PullData/Trigger%28%29" restartPolicy: OnFailure

Cron Job being set in the deployment.yaml file

Conversational AI and CAP Application Interaction

Once the data is available in the HANA DB, CAI can use the exposed CAP Application API to request for the invoice information. We used CAI as its high natural language processing (NLP) technology allowed us to focus on building skills and connecting to different APIs for the different use cases. The following are the skills we had configured to fulfill our use-cases.

Skill – “verify-customer-id”

This skill validates the customer by requiring them to provide their customer ID. Upon adding customer ID as a primary requirement, this ID will be sent to the CAP Application as a parameter and handled by one of the services. It returns a boolean value which will be added to the bot’s memory with the key name “validity”.

POST%20ID%20to%20API%20and%20set%20memory%20with%20response

GET method to CAP Application API and set memory with response

In the action for this skill, we check the value for the “validity” key.

If the value is False, an “Invalid ID” message will be sent, the memory of the bot for the user will be erased and the chatbot will be redirected to the start of this skill. The user will have to re-enter a customer ID again.

If the value is True, a POST method would be sent to the CAP Application to save the customer ID and conversation ID to our HANA DB. Subsequently, this customer ID and conversation ID pair will be used for sending the regular reminders.

Skill – “view-invoice” & “view-invoice-by-id”

Once the customer ID has been validated and the customer requests to view invoice, we add an action to use the GET method to retrieve the invoice for that customer ID from HANA DB. When an individual invoice is requested, we will request for the invoice ID and create an “invoiceValidity” key to store the boolean result, similar to that of “verify-customer-id”.same%20GET%20method%20technique%20and%20add%20invoiceValidity

same GET method technique and add invoiceValidity key

Skill – “unsubscribe” & “payment”

These 2 skills are simple action skills. When a user unsubscribes, a POST method is sent to the CAP application to delete the customer ID and conversation ID pair previously saved in HANA DB. In the “payment” skill, the bot’s action is to send the bank details when CAI detects the customer requesting for payment information.

Initiate payment reminder from CAP application

When the Cron job is triggered to refresh the data, a subsequent function is required to send the subscribed users their reminders from the CAP application. In order to initiate a conversation from CAI to the subscribed users, a Bot Connector API (messages) from the SAP API Business Hub was used. Since CAI uses an oauth authentication, we generated the access token using the bot’s client id and secret and provided our developer token to access it from the CAP application. Then we used a POST request to send the message to the user via the API server provided by the API Business Hub.

 this.after('Trigger', async req=>{ const getClientCredentials = oauth.client(axios.create(), { url: 'https://apjdl.authentication.eu10.hana.ondemand.com/oauth/token', grant_type: 'client_credentials', client_id: process.env.clientid, client_secret: process.env.clientsecret, }) const auth = await getClientCredentials() let config = { 'Authorization' : 'Bearer ' + auth.access_token, 'X-Token' : 'Token ' + process.env.xtoken, 'Content-Type' : 'application/json' } const cust = await SELECT.from(Chats) for (let i=0; i<cust.length; i++){ const res = await SELECT.from(Invoice).where ({Customer_ID: {'=': cust[i].Customer_ID}}) if (res.length > 1){ customerName = res[0].CustomerName let data = { "messages": [ { "type": "text", "content": `Dear ${customerName}, your latest invoice records are ready. Please view them.` } ] } let response = await cai.send({ query: 'POST connect/v1/conversations/'+ cust[i].ChatId + '/messages', data, headers:config }) console.log(response) } } })

Connecting the bot to Telegram

In the “Connect” tab of CAI, there are many third-party channels that we can connect the chatbot to. We used Telegram and created a bot on BotFather. Do refer to this blog for more information on connecting the chatbot to third-party applications. Please take note of the different message types being supported by these channels as they differ.

For more information and documentation regarding Conversational AI, please refer to this link.

Conclusion

We have just gone through the purpose and details that went into building this Automatic Payment Reminder chatbot. I hope that it was easy to follow this blog and it has exposed you to the possibility of building a CAI bot that runs on S4/HANA data. Hopefully, the implementation steps are useful to help you build a similar application with this technology stack.If you have any questions, feel free to drop them below!

Lastly, the following is a video demo of our completed prototype. Enjoy!

References

Credits

I would like to thank my co-authors and team who have worked on this project together with me: Aditi Arora , Min Pyae Moe, Gunter Albrecht