EDUCAÇÃO E TECNOLOGIA

Cloud Integration with Commerce Azure Blob Storage using REST API – Part 2

This blog post is the continuation of my previous blog post: Part 1

Get files from Commerce Azure Blob storage.

  1. Set up an Integration suite trial. Help Link: Setup.
  2. Azure Storage Explorer. Help Link: Azure Storage Explorer.

Integration%20Flow

Integration Flow

Step 1:

Configure timer to pick up files at a regular interval.

Step 2:

Use Content Modifier to set properties and header for version, as shown below.

Set%20Version%20Header

Set Version Header

Set%20Properties

Set Properties

Step 3:

Use Groovy Script to set up the headers for calling REST API to get the name of files starting with ‘order’.

import com.sap.gateway.ip.core.customdev.util.Message
import com.sap.it.api.ITApiFactory
import com.sap.it.api.securestore.SecureStoreService
import com.sap.it.api.securestore.UserCredential
import com.sap.it.api.securestore.exception.SecureStoreException
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.security.InvalidKeyException
def Message processData(Message message) { //Set Current Time TimeZone.setDefault(TimeZone.getTimeZone('GMT')) def now = new Date().format("EEE, dd MMM yyyy HH:mm:ss") + " GMT" //Set header for datetime message.setHeader("x-ms-date", now) //Get container name String container = message.getProperties().get("container") //Get folder path String folderPath = message.getProperties().get("folderPath") //Get File Name Prefix String filenamePrefix = message.getProperties().get("filenamePrefix") //Set Prefix name String prefix = folderPath + '/' + filenamePrefix + '_' message.setProperty("prefix", prefix) //Get Account Name String account = message.getProperties().get("accountName") // Set canonicalized Resource String canonicalizedResource = '/'+ account + '/'+ container + '\n'+ 'comp:list' + '\n' + 'prefix:' + prefix + '\n'+ 'restype:container' // set verb as requested method String verb = 'GET' //Get version String version = message.getHeaders().get("x-ms-version") //Set Signature String String StringToSign = verb +'\n'+'\n'+'\n'+ '\n' +'\n'+ '\n' + '\n' +'\n'+'\n'+'\n'+'\n'+'\n'+'x-ms-date:'+ now +'\n' +'x-ms-version:' + version + '\n'+ canonicalizedResource //Get Account Key from Secure Parameter String accountKeyAlias = message.getProperties().get("accountKeyAlias") def accountKey = getAccountKey(accountKeyAlias) // Decode Account Key def decodedKey = accountKey.decodeBase64() //Get Hash Value String hash = hmac_sha256(decodedKey, StringToSign) //Set Authorization header String auth = 'SharedKey'+ ' ' + account + ':' + hash message.setHeader("Authorization", auth) return message
} String getAccountKey(String accountKeyAlias)
{ def secureStorageService = ITApiFactory.getService(SecureStoreService.class, null) try { def secureParameter = secureStorageService.getUserCredential(accountKeyAlias) return secureParameter.getPassword().toString() } catch(Exception e) { throw new SecureStoreException("Secure Parameter not available") }
} String hmac_sha256(byte[] secretKey, String data) { try { Mac sha256_HMAC = Mac.getInstance("HmacSHA256") SecretKeySpec secret_key = new SecretKeySpec(secretKey, "HmacSHA256") sha256_HMAC.init(secret_key) byte[] digest = sha256_HMAC.doFinal(data.getBytes()) return digest.encodeBase64() } catch (InvalidKeyException e) { throw new RuntimeException("Invalid key exception while converting to HMac SHA256") }
}

Step 4:

Use Request-Reply step to get the list of files. Configure HTTP receiver channel as below.

HTTP%20Receiver%20Channel

HTTP Receiver Channel

Step 5:

Use XSLT mapping to sort the list of filenames in ascending order based on last modified time.

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" indent="yes"/> <xsl:template match="EnumerationResults/Blobs"> <xsl:copy> <xsl:apply-templates select="Blob"> <xsl:sort select="Properties/Last-Modified"/> </xsl:apply-templates> </xsl:copy> </xsl:template> <xsl:template match="@* | node()"> <xsl:copy> <xsl:apply-templates select="@* | node()"/> </xsl:copy> </xsl:template>
</xsl:stylesheet>

Step 6:

Use General Splitter step to retrieve each file entry as shown below.

General%20Splitter

General Splitter

Step 7:

Use Content Modifier step to save last modified time and filename as properties.

Content%20Modifier%20-%20Properties

Content Modifier – Properties

Step 8:

Use Groovy Script to check whether the file is new based on the comparison between last poll time and last modified time.

import com.sap.gateway.ip.core.customdev.util.Message
import java.text.SimpleDateFormat
def Message processData(Message message) { SimpleDateFormat customFormat = new SimpleDateFormat("E, dd MMM yyyy HH:mm:ss z"); def lastmodified = message.getProperties().get("lastmodified") def lastpoll = message.getProperties().get("lastpoll") def dateTime1 = customFormat.parse(lastmodified) def dateTime2 = customFormat.parse(lastpoll) def getFile = "No" if (dateTime1 > dateTime2) { getFile = "Yes" } message.setProperty("getFile", getFile) return message }

Step 9:

Use Router step as shown below.

Router

Router

Step 10:

Use Groovy Script to set up the headers for calling REST API to get the specific file.

import com.sap.gateway.ip.core.customdev.util.Message
import com.sap.it.api.ITApiFactory
import com.sap.it.api.securestore.SecureStoreService
import com.sap.it.api.securestore.UserCredential
import com.sap.it.api.securestore.exception.SecureStoreException
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.security.InvalidKeyException
def Message processData(Message message) { //Set Current Time TimeZone.setDefault(TimeZone.getTimeZone('GMT')) def now = new Date().format("EEE, dd MMM yyyy HH:mm:ss") + " GMT" //Set header for datetime message.setHeader("x-ms-date", now) // Get file name String filename = message.getProperties().get("fileName") //Get container name String container = message.getProperties().get("container") //Get Account Name String account = message.getProperties().get("accountName") // Set canonicalized Resource String canonicalizedResource = '/'+ account + '/'+ container +'/' + filename // set verb as requested method String verb = 'GET' //Get version String version = message.getHeaders().get("x-ms-version") //Set Signature String String StringToSign = verb +'\n'+'\n'+'\n'+ '\n' +'\n'+ '\n' + '\n' +'\n'+'\n'+'\n'+'\n'+'\n'+'x-ms-date:'+ now +'\n' +'x-ms-version:' + version + '\n'+ canonicalizedResource //Get Account Key from Secure Parameter String accountKeyAlias = message.getProperties().get("accountKeyAlias") def accountKey = getAccountKey(accountKeyAlias) // Decode Account Key def decodedKey = accountKey.decodeBase64() //Get Hash Value String hash = hmac_sha256(decodedKey, StringToSign) //Set Authorization header String auth = 'SharedKey'+ ' ' + account + ':' + hash message.setHeader("Authorization", auth) return message
} String getAccountKey(String accountKeyAlias)
{ def secureStorageService = ITApiFactory.getService(SecureStoreService.class, null) try { def secureParameter = secureStorageService.getUserCredential(accountKeyAlias) return secureParameter.getPassword().toString() } catch(Exception e) { throw new SecureStoreException("Secure Parameter not available") }
} String hmac_sha256(byte[] secretKey, String data) { try { Mac sha256_HMAC = Mac.getInstance("HmacSHA256") SecretKeySpec secret_key = new SecretKeySpec(secretKey, "HmacSHA256") sha256_HMAC.init(secret_key) byte[] digest = sha256_HMAC.doFinal(data.getBytes()) return digest.encodeBase64() } catch (InvalidKeyException e) { throw new RuntimeException("Invalid key exception while converting to HMac SHA256") }
}

Step 11:

Use Request-Reply step to get the file. Configure HTTP receiver channel as below.

HTTP%20Receiver%20Channel

HTTP Receiver Channel

Step 12:

Use Write Variables to save the last poll time.

Write%20Variables

Write Variables

Using the above explained steps, the new files can be retrieved based on last modified datetime. Please note, to avoid getting a large list of filenames during 1st call, an archiving strategy can be devised at Azure Blob storage end to move the old files to archive directory after N days.

Thank you for reading this blog post. Please feel free to share your feedback or thoughts or ask questions in the Q&A tag below.

QA link

Regards,

Priyanka Chakraborti

   Previous – Part 1