Simple Change Request Management Using JIRA Cloud/SAP CPI/ABAP Transports

For a whole SAP migration/implementation or even simple maintenance projects we do need a Change Request Management Tool. The best in the SAP context is to have ChaRM or Focused Build (on Top of SAP Solution Manager/ChaRM) which is natively integrated with SAP CTS/TMS.

But what if you don’t dispose of those tools and you would like to build your in house Change Request Management Tool based on JIRA or Mantis for instance.

The purpose of this blog is to build a simple solution based on JIRA WebHooks, SAP Cloud Process Integration (SAP CPI) and SAP ABAP Transports to achieve an end-to-end change request management.

To be able to follow different steps you will need:

  1. A Cloud Connector configured and exposing endpoints on your on premises SAP System
  2. A JIRA Cloud (You could have a Trial one)
  3. A SAP Cloud Process Integration instance running. Please refer to this tutorials series in Hana Academy Youtube Channel.
  4. You should be familiar with API Management and SAP CPI

As a JIRA User I would be able to release my Transport Workbench/Customizing Task automatically once the JIRA Backlog Item is set to DONE.

In most of the cases, The project Manager or the Functional Consultant do send reminders to Developers to release theirs tasks in order to be able to release the whole Transport. The Developer forget often to release open Tasks nevertheless ithe corresponding JIRA Backlog Item has been closed.

So the best would be to release the Transport Task automatically once the JIRA one is set to DONE (or whatever the wanted status in the workflow).

The below diagram illustrates the different components needed to be integrated together.

Architecture%20Overview

Architecture Overview

  1. We will need a Webhook configured at JIRA ADMIN UI Level: A webhook will act as an event listener that will trigger an action once the configured event occurs. Please refer to this link to read about JIRA Webhooks.
  2. The Webhook will trigger a call to a SAP CPI FLOW that will process the payload transform it according to the expected output. You need to pay attention to this part since the purpose is to build a flow that should make abstraction of the incoming input. Here for simplicity purpose, we will focus on JIRA incoming payload but it could be also a Mantis Payload.
  3. The CPI Flow will trigger a call to an API Proxy passing the transformed Payload
  4. In its turn, the API Proxy will trigger a call to an On Premise Restful Web Service via Cloud Connector. The Restful Web Service will manage the reaming task of releasing the Task

In addition, we will consider a predefined naming convention of the Transport Request Description since we need to find a way on how to map a JIRA Backlog Item and Transport Request Task. For instance, as a Team, we need to agree on putting the JIRA Backlog Item ID at the beginning of the Transport Request Description. Here is an example.

FG-01: Develop ABAP Report for Data Cleanup

In the next section we will go through all components mentioned above but in the reversed order since it is the best way to building blocks.

The purpose of this web service is to provide an endpoint expecting a POST Request having a predefined payload containing all need information to look for a Transport Request Task in the Development System and release it.

We could either build an OData Web Service or just a simple Rest one. I opted for a simple one to keep things simple.

Following are steps to create a Simple ICF Rest Web Service.

Create a class implementing IF_HTTP_EXTENSION interface

Go to SE24 (or SE80) and create a class and set IF_HTTP_EXTENSION as interface. Let’s call it ZCL_PMT_INT_HTTP_HANDLER. The interface IF_HTTP_EXTENSION contains one method IF_HTTP_EXTENSION~HANDLE_REQUEST that will handled all HTTP Requests.

IF_HTTP_EXTENSION%20Interface%20Implementation

IF_HTTP_EXTENSION Interface Implementation

Now we will provide a simple implementation of that method to be able to test it through ICF.

IF lv_http_method EQ 'GET'. server->response->set_status( EXPORTING code = 200 reason = 'Service is Up' ). DATA: conv_out TYPE REF TO cl_abap_conv_out_ce. conv_out = cl_abap_conv_out_ce=>create( encoding = 'UTF-8' ). DATA: lv_content TYPE xstring, lv_text TYPE string. lv_text = 'Service is Up'. conv_out->convert( EXPORTING data = lv_text IMPORTING buffer = lv_content ). server->response->set_data( lv_content ). ENDIF.

Create the ICF Node

Now go to TCODE SICF and create a node under /default_host/sap like below. Let’s name it pmt

Restful%20Web%20Service%20Endpoint%20%28ICF%20Node%29

Restful Web Service Endpoint (ICF Node)

Double click on the pmt ICF Node. Click on Edit, select the Handler List Tab and set the Handler to the previously created class.

ICF%20Node%20Handler

ICF Node Handler

Save and include your object in Transport Request or save it locally in $TMP Package.

At this stage the Service is Ready in it is minimal version and could be tested: in the SICF TCODE, look for the pmt node and right click and select Activate Service. Right Click again and select Test Service.

Test%20Service

Test Service

The corresponding URL should be <protocol>://<host>:<port>/sap/pmt

Provide the Login/Password .

You should received a message in the Browser saying Service is Up

ABAP Code to release a Transport Request Task

Now let’s concentrate on main task: Search for Transport Request Task and Release it.

To achieve this, we need to analyse what will be the payload which is supposed to be attached to the POST Request: According to the JIRA Documentation, the payload should look like below

JIRA%20Webhook%20Payload

JIRA Webhook Payload

So we should be expecting a structure containing a component named issue that contains a component named key. This is most pertinent information for us.

The ABAP Code Below allow to extract that information, and look into E07T/E070 DB Tables for the corresponding TRs.

 METHOD if_http_extension~handle_request. DATA(lv_http_method) = server->request->get_method( ). IF lv_http_method EQ 'POST'. DATA: lv_request TYPE string. DATA(lv_content_type) = server->request->get_content_type( ). DATA lr_data TYPE REF TO data. DATA lv_json TYPE /ui2/cl_json=>json. lv_request = server->request->get_cdata( ). lv_json = lv_request. lr_data = /ui2/cl_json=>generate( lv_request ). FIELD-SYMBOLS: <data> TYPE data, <issue> TYPE data, <key> TYPE data, <s_key> TYPE data. IF lr_data IS BOUND. """""""""""""""""""""""""""""""""""""""""""" """ Extract the KEY of the JIRA Backlog Item """""""""""""""""""""""""""""""""""""""""""" ASSIGN lr_data->* TO <data>. DATA(lo_structdescr) = CAST cl_abap_structdescr( cl_abap_structdescr=>describe_by_data( p_data = <data> ) ). DATA(components) = lo_structdescr->get_components( ). IF line_exists( components[ name = 'ISSUE' ] ). ASSIGN COMPONENT 'ISSUE' OF STRUCTURE <data> TO <issue>. ASSIGN <issue>->* TO FIELD-SYMBOL(<issue_struct>). lo_structdescr = CAST cl_abap_structdescr( cl_abap_structdescr=>describe_by_data( p_data = <issue_struct> ) ). components = lo_structdescr->get_components( ). IF line_exists( components[ name = 'KEY' ] ). ASSIGN COMPONENT 'KEY' OF STRUCTURE <issue_struct> TO <key>. ENDIF. ASSIGN <key>->* TO <s_key>. IF <key> IS BOUND. """""""""""""""""""""""""""""""""""""""""""" """ Search for the corresponding Transport """ Request """""""""""""""""""""""""""""""""""""""""""" DATA(lv_tr_serach) = |{ <s_key> }%|. SELECT FROM e07t LEFT JOIN e070 ON e07t~trkorr = e070~trkorr AND langu = @sy-langu FIELDS DISTINCT e07t~trkorr WHERE as4text LIKE @lv_tr_serach AND trfunction = 'K' INTO TABLE @DATA(lt_trs). IF lt_trs[] IS NOT INITIAL. IF lines( lt_trs[] ) EQ 1. DATA(lv_tr) = CONV trkorr( lt_trs[ 1 ] ). DATA lt_requests TYPE trwbo_request_headers. DATA lt_tasks TYPE trwbo_request_headers. """""""""""""""""""""""""""""""""""""""""""" """ Look for tasks in the found Transport """ Request """""""""""""""""""""""""""""""""""""""""""" CALL FUNCTION 'TR_READ_REQUEST_WITH_TASKS' EXPORTING iv_trkorr = lv_tr IMPORTING et_request_headers = lt_requests
* ET_REQUESTS = EXCEPTIONS invalid_input = 1 OTHERS = 2. IF sy-subrc = 0. """""""""""""""""""""""""""""""""""""""""""" """ Select only Open and classified Tasks """""""""""""""""""""""""""""""""""""""""""" lt_tasks = VALUE #( FOR wa IN lt_requests WHERE ( trfunction = 'S' AND trstatus = 'D' ) ( trkorr = wa-trkorr ) ). IF lt_tasks[] IS NOT INITIAL. DATA lt_return TYPE bapiret2. """""""""""""""""""""""""""""""""""""""""""" """ Release Tasks """""""""""""""""""""""""""""""""""""""""""" LOOP AT lt_tasks ASSIGNING FIELD-SYMBOL(<fs_tr>). CALL FUNCTION 'BAPI_CTREQUEST_RELEASE' EXPORTING requestid = lv_tr taskid = <fs_tr>-trkorr
* COMPLETE =
* BATCH_MODE = IMPORTING return = lt_return. IF lt_return IS INITIAL. "Do Something ENDIF. ENDLOOP. ENDIF. ENDIF. ENDIF. ENDIF. ENDIF. ENDIF. ENDIF. server->response->set_status( EXPORTING code = 201 reason = 'Action Done' ). ENDIF. IF lv_http_method EQ 'GET'. server->response->set_status( EXPORTING code = 200 reason = 'Service is Up' ). DATA: conv_out TYPE REF TO cl_abap_conv_out_ce. conv_out = cl_abap_conv_out_ce=>create( encoding = 'UTF-8' ). DATA: lv_content TYPE xstring, lv_text TYPE string. lv_text = 'Service is Up'. conv_out->convert( EXPORTING data = lv_text IMPORTING buffer = lv_content ). server->response->set_data( lv_content ). ENDIF. ENDMETHOD.

At this stage we are done with backend task. I assume now that your backend system if exposed via Cloud Connector and that the given pattern /sap/pmt is authorized.

In the next section we will wrap the developed Web Service with a SAP API Management Proxy.

First of all we will need to create an API Provider.

Browse to your SAP Integration Suite instance and click on the API Management Tile. On the left hand side menu click on Configure. The list of available API Provider is displayed. Click on create. Fill in all required fields as below:

  • Type: On Premise
  • Host: The Virtual Host configured in your Cloud Connector
  • Port: The Virtual Port configured in your Cloud Connector
  • Location ID: The Location ID that you set when connecting your BTP Account to Cloud Connector

API%20Provider%20Creation%20%281%29

API Provider Creation (1)

API%20Provider%20Creation%20%282%29

API Provider Creation (2)

In The left hand side menu, click on Develop. We will create an API Proxy.

The list of APIs is displayed, click on Create. Fill in the form as below and click Create button.

API%20Proxy%20Creation

API Proxy Creation

Save and deploy. The API is now deployed and exposed via the API Proxy URL below.

API%20Deployment

API Deployment

You could now call your on premises Restful Web Service via that URL by adding the suffix /pmt (remember that the URL on premises is /sap/pmt: Here we need just to add pmt since the /sap was already configured as a prefix in the API Proxy).

The next step is to create the CPI Flow that will transform the payload ans consume this API Proxy.

You need to prepare a flow like below

CPI%20Flow

CPI Flow

1- HTTPS: This is the input HTTP POST Request that will trigger the flow

HTTPS%20Post%20Action

HTTPS Post Action

2- A log Step is added to log the incoming message: You could pick the Groovy Script from here

3- A content modifier to do simple tranformation

4- Remove Non-Supported Attributes: The received JSON payload from JIRA contains some weird fields named as following 24×24 and 48×48 which represents images dimensions. Those fields blocks that JSON to XML Transformation and leads to a failure. Here is the Groovy Script

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import groovy.json.JsonSlurper;
import groovy.json.JsonOutput;
import groovy.json.*
def Message processData(Message message) { //Body def body = message.getBody(String); def jsonSlurper = new JsonSlurper(); def bodyJson = jsonSlurper.parseText(body); bodyJson.user.avatarUrls = {}; bodyJson.issue.fields.project.avatarUrls = {}; bodyJson.issue.fields.assignee.avatarUrls = {}; bodyJson.issue.fields.creator.avatarUrls = {}; bodyJson.issue.fields.reporter.avatarUrls = {}; message.setBody(new JsonBuilder(bodyJson).toString()); return message;
}

5- Convert the JSON to XML

6- Apply XSLT Transformation to Copy only needed fields: No need to pass the whole payload. Here is the Transformation:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:output omit-xml-declaration="yes" indent="yes"/> <xsl:strip-space elements="*"/> <xsl:template match="node()|@*"> <xsl:copy> <xsl:apply-templates select="node()|@*"/> </xsl:copy> </xsl:template> <xsl:template match="node()[not(self::root or ancestor-or-self::user or ancestor-or-self::issue)]"> <xsl:apply-templates/> </xsl:template>
</xsl:stylesheet>	

7- Convert Back the result from XML to JSON. Don’t forget to check Suppress JSON Root Element otherwise you won’t get the expected structure in the backend

XML%20To%20JSON%20Converter

XML To JSON Converter

8- Call the API Proxy prepared earlier: Make sur to put the API Proxy already prepared above.

HTTP%20Outbound%20call%20to%20APIM

HTTP Outbound call to APIM

Pay attention to set credentials in the Manage Security Materials section.

We need to make sure that every thing is working well before proceeding with the WebHook configuration at JIRA ADMIN UI.

I advise to install the Chrome Extension: SAP CPI Helper. It helps a lot for issue tracking during Unit Tests and provides an overview/insights on failed/succeeded calls.

Once the extension is added, you will have a set of buttons that will appear at the Top Right Corner:

CPI%20Helper

CPI Helper

Click on the Info Button in the CPI Helper toolbar and pick the Endpoint URL.

Testing%20the%20Flow%20Endpont

Testing the Flow Endpoint

You need to configure a Basic Authentication using your BTP credentials or ClientId/ClientSecret (usually used for OAuth2).

Now we made sure that our Flow and different blocks are working correctly. It is tile to plug our endpoint into the JIRA WebHook callback.

Now will concentrate on the WebHook configuration. It is nothing then a callback URL with a setting describing when it should be triggered. We will keep it simple and configure the WebHook callback to be triggered if an issue status is set to DONE and if the issue do belong to a specific project.

Go to your JIRA Cloud and click on the Configuration wheel. Select System

JIRA%20Cloud%20Webhook%20configuration%20%281%29

JIRA Cloud Webhook configuration (1)

Scroll to bottom in the left hand side menu and select WebHooks.

Click on Create a WebHook button to Create a new WebHook.

Make sure to:

  • Enter the Flow Endpoint URL
  • Set the Event
  • Not to exclude the body: Exclude body should be set to No since we would like to receive the body.

JIRA%20Cloud%20Webhook%20configuration%20%282%29

JIRA Cloud Webhook configuration (2)

Here the configuration should be finished… But wait !? How JIRA Cloud will be able to consume the FLOW Endpoint URL witout Authentification?

Unfortunatly JIRA Cloud Webhooks do not support Basic Authentication or other Authentication mode right now. There is already an issue which still open since a while requesting this feature. You can check it here.

To overcome this limitation, we will change our Architecture a little bit an introduce a Block for Anonymous Access (Not really Good thing but for POC purpose).

We will add a API Proxy that will handle the Webhook callback and will route the call to the CPI FLOW Endpoint. We will define an Authentication Policy at this API Proxy level which will inject credentials.

Architecture%20Change%3A%20Adding%20API%20Proxy%20Block

Architecture Change: Adding API Proxy Block

Go back to your Integration Suite and Go to Design, Develop, and Manage APIs.

Create a new API pointing to the CPI Flow Endpoint URL. The best is to create a API Provider of type Cloud Integration and to set it when creating the API Proxy.

API%20Provider%20Type%20Cloud%20Integration

API Provider Type Cloud Integration

Anonymous%20API%20Proxy%20%281%29

Anonymous API Proxy (1)

Anonymous%20API%20Proxy%20%282%29

Anonymous API Proxy (2)

After creating the new API, Edit its policies and add below ones:

  • Create a Assign Message Policy at the Target Endpoint PreFlow level. Fill in variables with credentials

Assign%20Message%20Policy

Assign Message Policy

  • Add a new Basic Authentication Policy and point to the header username and password to the already created variables in the Assign Message Policy Step.

Basic%20Authentication%20Policy

Basic Authentication Policy

Save and Deploy.

At this stage we do have an Endpoint URL (API Proxy) providing a Anonymous Access and could be configured at the Webhook Level.

Now go back to JIRA Coud WebHook configuration and set the latest created API Proxy. Click Save and make sur the the WebHook is enabled.

Try to test the whole flow by changing a backlog item status to DONE. Make sure that you are selecting an item belonging to the set project in the Event configuration.

Put a break point at the ABAP side. The breakpoint will be hit and you will be able to analyse the received payload.

Debug

Debug

And here you are done by putting in place your simple in house made Change Request Management Tool.

The achieved work is just a start to build a Change Request Management tool. You could imagine a whole workflow on JIRA Cloud and define your interaction Protocol with ABAP Transports. For instance we could build a workflow:

  • When Creating a Use Story a Transport Request is Created Automatically in ABAP System
  • When Creating a Sub Task in the Story a Task is added Automatically under the corresponding Trasnport Request
  • Release the Transport Request to Test when the US is Completed
  • …. And …..

When building new solution keep in mind to build blocks to facilitate the integration. Without integration capabilities your solution will be an isolated block that will end by being abandoned.