How to integrate the maritime blockchain Tradelens within hours

The world's largest shipping company Maersk, together with its IT subsidiary GTD Solution and technology giant IBM, has launched a blockchain-based maritime track & trace platform in which all partners can participate - TradeLens. TradeLens. TradeLens aims to increase efficiency in container logistics supply chains by automating and digitizing interactions with multiple parties. 

To this end, TradeLens has defined more than 100 events at which data and documents are exchanged in the course of the global supply chain, from the planned shipping time and the bill of lading to customs clearance and insurance policies. These events are continuously filled by the blockchain participants and new information is automatically consumed. A constant give and take.

Source:

TradeLens follows a role-based authorization conceptin which each participating partner is categorized during onboarding to the platform. For example, there are ocean carriers, rail or truck operators, which fall into the group of transport service providers, customs and customs service providers, depots and terminals as well as numerous other participants in the logistics process such as banks and insurance companies. Each role is assigned authorizations for the individual events. For example, container loading and unloading is regularly reported by shipping companies or 3PLs, while rail transport companies report loading at the transshipment station and depot operators provide gate-in/-out reports. All these assignments are summarized in the so-called Data Sharing Specification Sheet. It provides an overview during onboarding of what information is expected and what information can be consumed.

Source:

If there is clarity about the relevant events, a so-called organization is created for the participating partner in TradeLens as well as an account in the IBM Bluemix IAM Cloud (Identity Access Management); this is done during onboarding, which is carried out by GTD Solution. You then have access to TradeLens via the web frontend. However, in order to be able to integrate the APIs into their own IT systems, the participating partner must create a technical user with an API key in IBM Bluemix IAM and make their service ID known in TradeLens. This ensures that the partner can manage their accounts for the use of TradeLens completely independently and TradeLens does not need to know or authenticate any user names and passwords. This concept corresponds to the OAuth 2.0 standard, according to which the resource server (TradeLens) is separated from the authorization server (here the IBM Bluemix IAM account managed by the partner) so that the resource owner (partner) can determine which client (IT applications for the publication or subscription of events) should be allowed to access TradeLens and to what extent - a sophisticated and secure concept for the division of authority and responsibility.

Source:

Once all the organizational matters have been dealt with, it's time to implement the event publications in your own IT systems - the most interesting technical aspect.

Logistics service providers, infrastructure operators, public authorities and financial service providers will regularly ask themselves how they can integrate this process into their existing IT systems with the least amount of effort; ideally in such a way that the entire data exchange is fully automated, secure, scalable and verifiable.

TradeLens offers in its technical documentation provides excellent instructions on how to carry out initial integration tests using Postman. Numerous screenshots are used to explain how the two-step authentication is implemented and how JSON documents are sent to the corresponding API endpoints should be sent to the corresponding API endpoints. No further explanation is therefore necessary. Postman experts will automate these tests using Collection Runner and JavaScript methods so that they can check the relevant API endpoints without manual interaction.

Source:

Once these very well-documented initial integration tests with test data have been successfully completed, the actual task is faced: the automated transfer of data to TradeLens from the company's own core systems. Many companies are calling for the Enterprise Service Bus (ESB) team. In many companies, this team implements and manages all API integrations with peripheral systems. In most cases, a project is set up for a new integration, consisting of IT specialists from different departments as well as specialist departments for the functional requirements - an organizationally and technically complex process that seems to have fallen out of the agile and transactional reality of many logistics companies.

• Intuitive to operate
• Scalable according to current requirements (technical and economic)
• Flexible with regard to the mostly heterogeneous systems to be connected
• trustworthy with regard to the use of data 
• Safe from attacks and misuse
• Stable and robust in terms of availability
• be easy to operate - preferably as a cloud service (SaaS)

Workato fulfills precisely these requirements with its low-code trigger-action principle. Its founders bring decades of experience in the integration and automation of business processes in the enterprise environment and have implemented this knowledge in a native cloud solution with the trend towards empowering the "citizen developer", the technically interested and experienced employee of any organizational unit. Workato's enormous customer growth despite the COVID-19 pandemic shows that they have hit the mark: 

| Accelerate integrations, automate processes

The research giants Forrester and Gartner now rank Workato among the leading integration and automation platforms in the world. According to Deloitte, Workato is the 17th fastest growing company in the Bay Area and the 96th fastest growing company in North America.Workato integrates bothcloud applications and on-premise applications, whether legacy systems or state-of-the-art microservices - and of course at enterprise level. In figures more than 4 billion events per year are processed at over 4,300 customers with an availability of 99.99%.

Source:

Frequent integrations are offered as ready-made recipes, as Workato calls its automation processes. Each recipe consists of a trigger that initiates a sequence of actions. Ingeniously simple, yet extremely powerful! 

Out-of-the-box there are numerous triggers and actions, e.g. for invoicing with TradeShiftthe recording of a customer transaction in Salesforce, updating a customer service ticket in ServiceNow or accessing IoT data in Splunk. But this is just a tiny, exemplary subset. It could easily be expanded to include terms such as AWS, SAP, Oracle, PostgresSQL, MySQL, Snowflake, Slack, MS Teams, Sharepoint, Magento, etc.

In addition, Workato can be functionally expanded to include any other platform thanks to an easy-to-use but universally usable Connector SDK. If, for example, a frequently used EDIfact or X12 connection is required in transport logistics, there is a connector to the SaaS provider Orderful developed with this SDK. If you prefer to integrate a different platform as a ready-made recipe, this is possible in the same way - particularly interesting for recurring automations.

This allows the individual steps of a recipe to be "hidden" within a single trigger, e.g. the TradeLens event "Actual discharge from vessel" was triggered, or encapsulated in one or more actions, e.g. publish TradeLens event "Consignment bill of lading issued". We will go into this option of creating your own connector in more detail in a separate article. Now back to the TradeLens Recipe, which makes the individual event publication steps visible.

The trigger for triggering an event in TradeLens should be a new file on an SFTP server. In the specific use case, a leading European intermodal operator receives status messages from various sources about the transportation process of containers: if a container has been loaded onto a train, if a container has left the transshipment station by truck, if a train has arrived at a port terminal, if a new transport order has been placed, to name just a few examples. As soon as such a status message is stored on an SFTP server in the intermodal operator's private cloud data center, the trigger is activated in the Workato-Recipe. It would also be possible to select specific trigger content such as file name parts, but this is not necessary in this specific case.

Connecting via SFTP is child's play in Workato: Simply add a connection, select SFTP, enter the login parameters such as user name, password, hostkey fingerprint to prevent man-in-the-middle attacks, IP address and port - and click Connect. That's it! The data is stored in encrypted form and protected in accordance with GDPR securely handled.

The same is done with other application connections that are to be used within actions in the recipe, such as Splunk or, in this specific case, Google Sheets. 

Once the trigger has been configured, you can continue with the actions. TradeLens first requires authentication in IBM Cloud IAM with the API key previously created via the interface. If this is correct, you will receive an access and refresh token that you can use to authenticate yourself in the next action in the TradeLens Solution Manager. 

This HTTP request is very easy to configure using drag & drop. Constants such as the IBM Cloud IAM link and the API key should be defined as a property beforehand and then inserted as an element in the corresponding action fields so that this confidential data does not have to be written out in the recipe. Workato makes the values in properties unrecognizable as soon as you use keywords such as "Key" or "Password" in the property name. Helpful, especially when discussing the solution with screen sharing.

The access and refresh token, the response to this HTTP request, is then inserted as the request body into the authentication in the TradeLens Solution Manager, the third step in this recipe. This can also be done using drag & drop by dragging the response "body" of the second step into the request body of the third step - child's play. The request URL is in turn made up of previously defined static properties.

Next, the content of the status message that triggered the trigger is downloaded from the SFTP server(Download File), prepared in terms of content(Parse CSV) and checked using a lookup table(Search Entries) to determine how many events this status message should trigger in TradeLens. The latter results from a content comparison that was carried out during onboarding. Finally, variables are created, as the following actions should be reused in a repeat loop(For-Each) as far as possible by refilling the previously defined variables for the TradeLens event URL and the event content depending on the status message received.

An interesting detail is how easy it is to build a CSV file into a structured object whose elements can be used in other actions. All you have to do is copy the header line of the CSV into the Parse CSV action - and you're done.

This is followed by a For-Each loop in which a JSON message is created for each status message-TradeLens event combination. In this specific case, there are 4 different combinations, which differ primarily in the direction of transport (import, export), the transport medium (train, truck) and the transferred time of the event. In addition, the TradeLens API endpoint must be correctly assigned, which is done by updating the variable created for this purpose.

In enterprise integrations, the mapping of source and target schemas is usually a time-consuming challenge. In Workato, this is efficiently solved via a map action, in which the target schema can either be defined field by field or automatically created by uploading or copying and pasting a JSON target document. Because TradeLens stores the event content in Swagger (OpenAPI 2.0) and the message model can be easily copied from there, the third option was used here. A custom data model (schema) is created via copy&paste, which is used as the target structure within a map action - child's play and done in seconds.

Mapping the source data structure to the target data structure is also very simple. However, Workato also has a built-in formula mode so that more complex calculations can be performed for each field if necessary. TradeLens requires the most sophisticated calculations in the Event Occurrence and Terminal fields. For the former, the timestamp must be specified in ISO 8601 format, i.e. UTC-compliant; for the latter, TradeLens requires the seaport terminal as a combination of UN/LOCODE and SMDG terminal character string. As the corresponding output data fields are not available in this form, the time stamp must be converted to UTC by specifying the relevant time zone, while in the case of the terminal it must first be checked whether the event took place at a seaport terminal at all; if so, the DIUM station number of the International Union of Railways (UIC) is converted into the required character string using a lookup table. The formula mode uses the language Rubylanguage; it is very similar to the well-known formula mode in Microsoft Excel.

Finally, the constructed TradeLens event object is converted into JSON - a one-liner formula - and transferred to the corresponding TradeLens API path resulting from the status message-TradeLens event combination described above.

Even if Workato always logs all jobs including input and output per step, unless you have (optionally) switched this off for data protection reasons, logging to an external source is recommended. Logging solutions such as Splunk or Datadog for which Workato offers ready-made actions. In this case, however, Google Sheets was chosen for reasons of simplicity. In addition to the time stamp of the transmission to TradeLens, the initial values from the transport management system, the transmitted TradeLens event content, the TradeLens API endpoint and the Workato job link were stored in Google Sheets. In addition, the TradeLens event ID was saved, which TradeLens transmits to the sender for each event message so that the TradeLens team can find the transaction again in the event of a problem. All of this is a matter of minutes thanks to the Google Sheets Connector.

Speed, simplicity and reliability. Modern technologies must embody these virtues in order to have a positive impact. Transport logistics in particular, as a driver of global trade and prosperity, must take this triad to heart in order to meet market requirements and successfully master the digitalization demands on its own business. The large number of integrations and automations required to remain relevant and profitable should therefore not be a problem, but rather a welcome opportunity. With the right tools and technologies, this is not a problem but a pleasure, as even global projects on the scale of TradeLens show. We can therefore look forward to the future! 

Business Automatica reduces process costs by automating manual activities, increases the quality of data exchange in complex system architectures and connects on-premise systems with modern cloud and SaaS architectures.