Companies are increasingly adopting digital transformation techniques to reduce the amount of time spent in mundane tasks. The success of digital transformation is 100% only when the business strategy is clearly understood. One of my recent Customer PoC was for a large conglomerate company that specializes in offshore & marine, property, infrastructure and asset management businesses.
Owing to their large business, they deal with thousands of vendors on a daily basis. The main issue while dealing with such a large group of vendors is that there is no uniformity in the invoices generated by these vendors. It is hence a tedious manual process to match the invoices to their corresponding purchase orders. This can easily be automated and hence they decided to embark on the Digital transformation process.
The customer processes around 20,000 invoices per month of which 60% of the invoices are directly keyed into their e-invoice portal, 10% of them are populated into SAP systems while the remaining 30% are problematic. They either have handwritten invoices, physical bills that are hard to automate or vendors who are not used to the digital world. These are the invoices that have to be processed as part of our current PoC.
The invoices have 4 different formats : Purchase Order (80% of the invoices) , Non-Purchase Order, Credit Note and Debit Note. If the invoice is identified as a Purchase Order invoice, the extracted line items have to be matched to a purchase order entry. A flow chart of the entire use case is shown below.
The entire process of manually downloading all files, checking if it is an invoice or not and matching each invoice line item to a purchase order entry is time consuming. We decided to use machine learning to tackle this problem
Our solution to the above problem statement is depicted in the flowchart below. We decided to use Intelligent Robotic Process Automation for scanning through the mails and downloading all files from the invoices. These downloaded invoices are then sent as input to a machine learning model that classifies the file as non-invoice, Purchase Order (PO) invoice, Non Purchase Order (Non-PO) invoice, Debit Note or Credit Note. Except for non-invoice, all other files are then passed through the next step that extracts the relevant fields from the invoices and as a last step, for all PO invoices, we will match the line items to a corresponding purchase order entry. The extracted details will then be populated into the existing e-invoice portal that the customer is using.
For the purpose of the PoC, we agreed to work on just 10 vendors for PO invoices, 1 vendor each for Non-PO , Credit note and debit note. The data provided to us by the customer was 10 invoices along with supporting documents for each of the vendor. For Purchase order info, the customer shared with us the csv files containing the relevant info on the purchase order number as well as the entries for each purchase order number.
The solution architecture is shown in the figure below.
We built a custom UI as we chose not to integrate with the customers’ existing portal for the POC. The UI has provision to attach invoices and Purchase order info and once these files are uploaded, the following steps are executed.
- Invoice Classification
We trained a classification model to identify if it is a non invoice or PO invoice or Non PO invoice or Credit note or debit note. If it is not a non-invoice, we then identify the vendor information using another machine learning classification model.
- Invoice Field Extraction
A simple OCR function would not be enough to extract the records from Invoices. We used a combination of 2 results for this purpose:
- Document information extraction API :
This is available as part of AI Business Services in SAP Cloud Platform. It is a pre-trained service leveraging deep-learning algorithms to extract structured semantical information from unstructured documents. It has been trained with several thousands of invoices of different formats so the results will not be very accurate for the invoices we are working on as the model wouldn’t have seen any such invoice during its training. The reason behind using this API despite knowing that the results might not be very satisfactory is that the line items are mostly arranged in a tabular format in all the invoices despite the structure of the invoice. Due to this, the line items extracted have a higher chance of being correct.
- Chargrid Model:
This deep learning model was developed by the SAP SE Machine Learning R&D group in Germany and it was published in Empirical Methods in Natural Language Processing (EMNLP) 2018. This paper proposes a novel document representation approach: character grid or chargrid and makes use of this 2D grid of characters to preserve the 2D layout structure of documents; while simultaneously working on the textual content.
We trained this model on the data given to us by the customer but due to the limited data given to us for training, our results might not be as accurate as expected.
Once these results are generated, we combine both the results using some predefined rules after checking the training results. We use the line items from API and the header items from Chargrid model if there is any mismatch during comparison of the results. For the purchase order number and the reference number, we have additional steps to extract the correct values. As mentioned earlier, the customer provided some csvs which provide info on the reference numbers and purchase order numbers for each vendor. We have added checks to see if there is any match with any of the existing PO numbers and select it only if the confidence percent is above 70%. We also have rules to define how the line items have to be merged.
3) Purchase Order Matching
Purchase Order matching is done only for PO invoices. The line items extracted in the previous step are now processed for PO matching. The PO files provided to us have info on the entries for each PO number. The entries for the PO extracted in the previous step are compared with the line items of the current invoice and the top 3 results and their corresponding confidence score is saved. The confidence score is calculated based on the following values:
- Line item description,
- Unit price
- Unit of measure
- Total price
A fuzzy matching is done between the PO values and the line item values and the confidence score is calculated by giving equal weightage to all these variables.
All these results are then displayed in the UI.
By using SAP Intelligent Robotic Process Automation in combination with SAP Data Intelligence, Invoice Extraction along with Purchase order matching process becomes more reliable, intelligent and automated, thus enabling the finance team to focus on strategic and more valuable tasks.