# Integration Framework

# Overview

Oracles are a well known solution to pull data from external sources into smart contracts. With that idea in mind, OpenLaw created the Integration Framework to fetch data from third-party systems into OpenLaw agreements. The framework is Blockchain and service agnostic and all that data pulled from the integrated services (any system that is plugged into the Integration Framework) is validated by OpenLaw VM (opens new window) using Elliptic Curve Digital Signature Algorithm (ECDSA) (opens new window).

The main goal with the Integration Framework is to solve a problem that we often face when we want to securely exchange data between different services. We want to make sure the data matches the OpenLaw Markup Language and was provided by a trusted service, so it can be used in the OpenLaw agreements.

In order to integrate any external service with the Integration Framework, one must provide a service implementation that matches the specification defined in the External Service Proto. The server definition is based on Protocol Buffer (opens new window) which allows code generation in several different languages.

With the implementation in place, one just needs to implement the business logic and make sure to sign the data that will be provided to OpenLaw in the responses.

The Integration Framework currently supports two types of integration:

1. Signature

Any service that provides an e-Signature (opens new window) solution, so it can be used to sign OpenLaw agreements, i.e: DocuSign (opens new window)

2. Computation (Common)

Any service that executes a computation based on the Markup Interface, i.e: Fetch BTC-USD exchange from Coin Market Cap (opens new window)

# Build, integrate and use an External Service

integration_framework_flow

# Building

# Server Definition

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages.

https://developers.google.com/protocol-buffers (opens new window)

# External Service Proto

syntax = "proto3";

package integration.framework.openlaw;

// ExternalService definition parsed by the Protoc Compiler. It must be extended by the server implementation
// so it can respond to gRPC requests from the Integration Framework.
service ExternalService {

    // Gets the Ethereum Public Address from the service which is
    // used to verify events sent from the service to OpenLaw VM.
    rpc GetEthereumAddress (Empty) returns (EthereumAddressResponse) { }

    // Gets the server Markup Interface definition which is used in an OpenLaw
    // Agreement with ExternalCall or ExternalSignature variable types.
    // The expected Markup Interface definition must follow the standard:
    //  - [[Input:Structure(inputField1: <Type>; inputField2: <Type>; inputFieldN: <Type>)]] [[Output:Structure(outputField1: <Type>; outputField2: <Type>; outputFieldN: <Type>)]]
    //  - <Type> - can be replaced by: Text, Number and Date.
    // Basic Markup Interface for the Coin Market Cap service can be defined as
    // a String of value:
    //  - "[[Input:Structure(fromCurrency: Text; toCurrency: Text; amount: Number)]] [[Output:Structure(currency: Text; price: Number; lastUpdate: Text)]]"
    // The standard Markup Interface for any e-Signature service is defined by
    // the following String value:
    //  - "[[Input:Structure(signerEmail: Text; contractContentBase64: Text; contractTitle: Text)]] [[Output:Structure(signerEmail: Text; signature: Text; recordLink: Text)]]"
    // Any e-Signature service must use the exact same Markup Interface as
    // described above, otherwise the e-Signature will not be validated by the
    // OpenLaw VM.
    rpc GetMarkupInterface (Empty) returns (MarkupInterfaceResponse) { }

    // Executes the request from OpenLaw Integrator Framework
    // and waits for the External Service response.
    rpc Execute (ExecuteRequest) returns (ExecuteResponse) { }
}

// The Ethereum Address response message.
message EthereumAddressResponse {
    //The Ethereum public address generated by the External Service.
    string address = 1;
}

// The Markup Interface response message.
message MarkupInterfaceResponse {
    // The Markup Interface definition that matches the
    // OpenLaw Input/Output structures.
    string definition = 1;
}

// The Request message to be processed by the External Service.
message ExecuteRequest {
    // The contractId or flowId string value generated by OpenLaw.
    // e.g: 1a86c0ab-7895-497c-babb-a3c089df1203
    string callerId = 1;
    // The actionId string value which is an arbitrary content that
    // represents the action that happened in OpenLaw VM.
    string actionId = 2;
    // The requestId string value that represents the request in
    // Integrator Framework.
    string requestId = 3;
    // The inputJson string value that matches the Input type defined
    // in the Markup Interface. It must be converted from string to
    // json value in order to access its properties.
    string inputJson = 4;
    // The ethereum public address from the External Service as
    // provided in the EthreumAddressResponse.
    string servicePublicAddress = 5;
}

// The Response message obtained from the executed call.
message ExecuteResponse {
    // The contractId or flowId string value generated by OpenLaw.
    // e.g: 1a86c0ab-7895-497c-babb-a3c089df1203
    string callerId = 1;
    // The actionId string value which is an arbitrary content that
    // represents the action that happened in OpenLaw VM.
    string actionId = 2;
    // The requestId string value that represents the request in
    // Integrator Framework.
    string requestId = 3;
    // The outputJson as string value that matches the Output type
    // defined in the Markup Interface. All values of the json must
    // be String and the json must have no spaces.
    // e.g:
    // - {"currency":"BRL","price":"29.03","update":"2019-09-10T16:31:00.000Z"}
    // - {"param1":"value1","param2":"value2","paramN":"valueN"}
    string outputJson = 4;
    // The output signature string which represents the signature of
    // the outputJson field. The stringified version of the outputJson
    // must be used for signature.
    string outputSignature = 5;
    // The possible statuses of the response after the execution is
    // terminated.
    enum Status {
        FAILURE = 0;
        SUCCESS = 1;
    }
    // The status of the execution. If not provided, the default is FAILURE.
    Status status = 6;
    // The message returned by the External Service to indicate if the
    // execution was completed with success or not. It can be an error
    // message as well.
    string message = 7;
    // The ethereum public address from the External Service as provided
    // in the EthereumAddressResponse.
    string servicePublicAddress = 8;
}

// The Empty request message to indicate no data in the request.
message Empty {}

# Identity

The Integration Framework identifies external services by their Public Ethereum Address. The reason for that is that every response returned by the external service must be signed using Elliptic Curve Digital Signature Algorithm (ECDSA) (opens new window) so OpenLaw VM can verify if the response was provided by the expected server implementation. In addition to that, it gives us the ability to spin up an audit process in which all requests and responses are stored in the Ethereum Blockchain, so anyone is able to verify the communication between OpenLaw and external services.

# Markup Interface

The OpenLaw editor uses the Markup Language to define variables in the agreements. In order to provide data from the agreement to the External Service one must use a Markup Interface. A Markup Interface defines two Structures:

Input - a set of fields and values that are passed to the external service in the gRPC request.
Output - a set of fields and values that are returned from the external service in the gRPC response.

It is important to mention that any e-Signature service must provide the standard Markup Interface for e-signatures as defined below:

[[Input:Structure(signerEmail: Text; contractContentBase64: Text; contractTitle: Text)]] [[Output:Structure(signerEmail: Text; signature: Text; recordLink: Text)]]"

The fields defined for the Input structure are provided by OpenLaw, and the fields defined for the Output structure are required and must be provided in the response of the external service. Please check the e-Signature Response for more details about the required fields. For any other type of responses that are not e-signatures, the fields may change, but the response still needs to be signed as explained in Computation Responses.

# Data Authenticity

In Ethereum Blockchain systems there is a private and a public key. These keys are generated when you create a new blockchain "account". When you create a new external service to integrate with OpenLaw, it is required that you provide an Ethereum Account, so it can be used to sign data and expose the public keys for signature verification.

Any response generated by the External Service must be signed. At the moment, we have two signature methods, one for Computation responses and another for e-Signature responses.

OpenLaw VM implements the signature verification and each response provided to the OpenLaw platform is verified using the external service public ethereum address. If the signature is invalid, which means the derived public key does not match the service public key, the data is discarded. In the next two sections we will see how to build the signed responses.

# Computation Responses

This is an example in Scala of a signed response generated for CoinMarketCap Service which returns the exchange rate for a given currency:


// gRPC message sent by Integration Framework requesting the exchange rate. See
// proto ExternalService.ExecuteRequest for more info
val request = ???
// HTTP response returned by the CoinMktCap API with the exchange rate
val response = ???

private def sign(callerId: String, actionId: String, output: String): String = {
    def sha256(data: String): Array[Byte] =
      EthData.of(data.getBytes(StandardCharsets.UTF_8)).sha3().hash

    val dataToSign = EthData.of(sha256(callerId))
      .merge(EthData.of(sha256(actionId)))
      .merge(EthData.of(sha256(output)))

    serviceEthereumAccount.getAccount.sign(dataToSign).toData.toString
}

// The fields of the json object must match the MarkupInterface Output structure fields
val output = Json.obj(
  "currency" -> response.currency,
  "price" -> response.price,
  "lastUpdate" -> response.lastUpdated)

// Signing the calledId, actionId and json output in string format
val outputSignature = sign(request.callerId, request.actionId, output.toString)

// gRPC message returned to Integration Framework
// 1. Successful response: all fields are required, otherwise the response will fail
ExecuteResponse()
      .withActionId(request.actionId)
      .withRequestId(request.requestId)
      .withCallerId(request.callerId)
      .withServicePublicAddress(request.servicePublicAddress)
      .withOutputJson(output)
      .withOutputSignature(outputSignature)
      .withStatus(Status.SUCCESS)
      .withMessage("Conversion executed with success")

// If the computation failed for some reason, you can return a failure instead
// 2. Failed response: the output and outputSignature are not mandatory for failed responses
ExecuteResponse()
      .withActionId(request.actionId)
      .withRequestId(request.requestId)
      .withCallerId(request.callerId)
      .withServicePublicAddress(request.servicePublicAddress)
      .withStatus(Status.FAILURE)
      .withMessage(s"Conversion failed, error: $error")

# e-Signature Responses

This is an example in Scala of a signed response generated for DocuSign Service which returns the signature that confirms the document was properly signed:


// gRPC message sent by Integration Framework requesting the e-Signature. See
// proto ExternalService.ExecuteRequest for more info
val request = ???
// HTTP response returned by the DocuSign API with the e-Signature
val response = ???

private def sign(email: String, callerId: String): EthereumSignature = {
    val dataToSign = SignatureOutput.prepareDataToSign(Email(email).getOrThrow(), ContractId(callerId), ServerCryptoService)
    EthereumSignature(serviceEthereumAccount.sign(EthData.of(dataToSign.data)).toString).getOrThrow()
}

// Signing the calledId, actionId and json output in string format
val signature = sign(request.signerEmail, request.callerId)

// gRPC message returned to Integration Framework
// 1. Successful response: all fields are required, otherwise the response will fail
val signatureOutput = s"{\"signerEmail\":\"test@openlaw.io\",\"signature\":\"${signature.toString}\",\"recordLink\":\"https://demo.docusign.com/signed/document\"}"
ExecuteResponse()
      .withActionId(request.actionId)
      .withCallerId(request.callerId)
      .withServicePublicAddress(request.servicePublicAddress)
      .withRequestId(request.requestId)
      .withOutputJson(signatureOutput)
      .withStatus(Status.SUCCESS)
      .withMessage("Signature executed with success")

// If the computation failed for some reason, you can return a failure instead
// 2. Failed response: the output and outputSignature are not mandatory for failed responses
// When there is no valid signature you can provide empty values for signature and recordLink fields
// Make sure the json string has no spaces
val emptySignatureOutput = "{\"signerEmail\":\"test@openlaw.io\",\"signature\":\"\",\"recordLink\":\"\"}"
ExecuteResponse()
      .withActionId(request.actionId)
      .withCallerId(request.callerId)
      .withServicePublicAddress(request.servicePublicAddress)
      .withRequestId(request.requestId)
      .withOutputJson(emptySignatureOutput)
      .withStatus(Status.FAILURE)
      .withMessage(s"Signature failed, error: $error")

# Secured Connection

gRPC is designed to work with a variety of authentication mechanisms (opens new window). With that in mind, we have enabled TLS connections between external services and the Integration Framework. All the data exchanged between the client and the server are encrypted. Mutual authentication is disabled for now and you just need to provide your external service X.509 public key (opens new window) during the registration process, so the connection gets validated and your service gets registered. Under the hood, the Integration Framework uses OpenSSL as TLS provider and TLSv1.1, TLSv1.2 and TLSv1.3 protocols are enabled. You can generate a test certificate using the following command:


openssl req -x509 -newkey rsa:4096 -keyout server-private-key.pem -out server-public-key-cert.pem -days 365 -nodes -subj '/CN=<your-service-public-domain>'

Now that you have sample private and public keys generated and considering you have a service written in Scala, here is an example of how you can create a gRPC server with TLS authentication enabled:


val server = NettyServerBuilder
      .forPort(config.getServerPort)
      .addService(ExternalServiceGrpc.bindService(serviceImpl, ExecutionContext.global))
      .sslContext(GrpcSslContexts
        .configure(SslContextBuilder.forServer(config.getServerPublicKey, config.getServerPrivateKey))
        .sslProvider(SslProvider.OPENSSL)
        .protocols("TLSv1.1", "TLSv1.2", "TLSv1.3")
        .build())
      .build()

    server.start()
    server.awaitTermination()

Make sure you have set the OpenSSL as SSL provider and enabled all 3 TLS protocol versions. With that in place, your service is secured and all the connections started by the Integration Framework will be encrypted.

# Computation Service Implementation

package io.openlaw.services

import integration.framework.openlaw.{Empty, EthereumAddressResponse, ExecuteRequest, ExecuteResponse, MarkupInterfaceResponse}
import integration.framework.openlaw.ExternalServiceGrpc.ExternalService
import javax.inject.{Inject, Singleton}

import scala.concurrent.{ExecutionContext, Future}

/**
* ExternalServiceImpl extends the ExternalService trait generated by the Protoc
* Compiler. This is the server implementation that responds to gRPC requests
* from the Integration Framework.
*
* @param priceConverterService - The service implementation which hits the Coin Market Cap API to get the exchange rates.
* @param configService - The configuration service which provide access to server properties and environment variables.
* @param identityService - The service which provides the Ethereum account to get the Public Ethereum Address as service identity.
* @param ec - Scala execution context.
*/
@Singleton
class ExternalServiceImpl @Inject()(priceConverterService: PriceConverterService,
                                    configService: ConfigService,
                                    identityService: IdentityService)(implicit ec: ExecutionContext)
  extends ExternalService {

  /**
    * Gets the Ethereum Public Address from the service which is used to verify
    * events sent from the service to OpenLaw VM.
    */
  override def getEthereumAddress(request: Empty): Future[EthereumAddressResponse] =
    Future.successful(EthereumAddressResponse().withAddress(identityService.getAccount.getAddress.withLeading0x))


  /**
    * Gets the server Markup Interface definition which is used in a OpenLaw
    * Agreement with ExternalCall or ExternalSignature variable types.
    * The expected Markup Interface definition must follow the standard:
    *  - [[Input:Structure(inputField1: <Type>; inputField2: <Type>; inputFieldN: <Type>)]] [[Output:Structure(outputField1: <Type>; outputField2: <Type>; outputFieldN: <Type>)]]
    *  - <Type> - can be replaced by: Text, Number and Date.
    * A basic Markup Interface for the Coin Market Cap service can be defined as
    * a String of value:
    *  - "[[Input:Structure(fromCurrency: Text; toCurrency: Text; amount: Number)]] [[Output:Structure(currency: Text; price: Number; lastUpdate: Text)]]"
    * The standard Markup Interface for any e-Signature service is defined by the
    * following String value:
    *  - "[[Input:Structure(signerEmail: Text; contractContentBase64: Text; contractTitle: Text)]] [[Output:Structure(signerEmail: Text; signature: Text; recordLink: Text)]]"
    * Any e-Signature service must use the exact same Markup Interface as
    * described above, otherwise the e-Signature will not be validated by the
    * OpenLaw VM.
    */
  override def getMarkupInterface(request: Empty): Future[MarkupInterfaceResponse] =
    Future.successful(MarkupInterfaceResponse().withDefinition(configService.getMarkupInterface))

  /**
    * Executes the request from the Integration Framework and waits for the
    * External Service response.
    */
  override def execute(request: ExecuteRequest): Future[ExecuteResponse] =
    priceConverterService.convert(request)

}

# Integrating

# External Service Registration

You can register your external service into an OpenLaw instance by accessing:

Menu -> Admin Tools -> Integrator Registration

The following form should be rendered:

instance-registration

Name
- case-sensitive unique name and it will be used from your agreements to call the registered service.
Type
- type of external service: Common (used for Computation) or Signature.
Version
- version of your external service.
Ethereum Public Address
- public ethereum address generated for your service which must be unique.
Secured endpoint
- gRPC endpoint with one of the TLS protocols enabled: v1.1, v1.2, v1.3.
X.509 public certificate
- public X.509 certificate generated for TLS communication.

# Using

# External Call

In order to use your integrated service from OpenLaw agreements, you need to declare a new variable type called ExternalCall.

Here is an example of a call to the Coin Market Cap integrated service:

<%
[[amount: Number]]

[[Crypto Currency: Choice("BTC", "ETH", "DCR", "ADA")]]
[[From Crypto: Crypto Currency]]

[[FIAT Currency: Choice("USD", "BRL", "EUR")]]
[[To FIAT: FIAT Currency]]

[[startingAt:DateTime]]

[[externalCall:ExternalCall(
serviceName: "Coin Market Cap";
parameters:
fromCurrency -> From Crypto,
toCurrency -> To FIAT,
amount -> amount;
startDate: startingAt)]]
%>

Sign it

[[Signatory Email:Identity]]

---

{{ externalCall.status = 'success' =>
    [[externalCall.result.currency]]
    [[externalCall.result.price]]
    [[externalCall.result.lastUpdate]]
}}

You can access the response values by calling <externalCallVariable>.result.fieldName.

# External Signature

If you want to use your integrated service for e-Signatures from OpenLaw agreements, you need to declare another variable type called ExternalSignature.

Here is an example of a call to the DocuSign integrated service:

This is a test agreement that can be signed using DocuSign e-signature.

[[Signatory: ExternalSignature(serviceName:"DocuSign")]]

# Next Steps

Create a pseudocode for the signature function so it can be implemented in any language.

← Token Forge Overview Relayer Overview →