Flow runtime data: In your own storage now!

  

OVERVIEW

When you run your apps built with Flow, your runtime data is saved in a store in the Boomi Flow Cloud by default. This store is managed by the platform, and is configured the same way for each tenant and every builder. Having the runtime data stored and managed by Flow frees you up from having to worry about storage or provisioning. The downside of this is, you are not able to customize the data store configurations; for example you can not have your runtime data in a location of your choice only. 

Which brings us to the next option, and one on which we are going to be spending a few  minutes now. You can store your runtime data externally; in a storage location of your choice. 

Having an external data store, outside the realms of where the Flow platform stores runtime data, gives you even more granular control over your data.  On-prem? Yes. Heroku or AWS? Of course! 

An external data store can be used for specific data residency and compliance requirements, giving you the freedom to decide where your runtime data ultimately resides, based on your unique business needs.

What exactly happens when you create a store? Well, when you create a store, what you are doing is providing an HTTPS-accessible endpoint; this endpoint adheres to the Flow External Storage API. The Flow engine, in turn, generates a platform key and receiver key pair values; which are used to sign, verify, encrypt and decrypt messages sent to and from the external storage provider.

The platform key is used by the external storage endpoint to verify that incoming requests are indeed coming from the Boomi Flow platform (authenticity and non-repudiation). The platform key is also used to encrypt any responses that are sent back to the platform.

The receiver key is used by the external storage endpoint to decrypt incoming requests from the Flow platform, and to sign any outgoing responses that are sent back to the platform.

[Note: This is a technology preview version and is not a production release. We recommend you wait till we finalize the features and launch the final production release to start using external storage for your production environment.] 

SETTING UP AN EXTERNAL DATA STORE

You can set up an external data store anywhere you like. This could be on-prem, or with third-party providers like AWS or Heroku.

  1. We have detailed instructions on how to deploy an external store to Heroku, using our default implementation here
  2. You can find Kubernetes example configurations, with health checking, and a test-ready database for the default implementation here.
  3. Find everything you want to know on how to use external storage with systemd for the default implementation here.
Want to write your own implementation?

Of course, you can! Here is our External Storage API documentation.

USING THE FLOW DRAWING TOOL

Here is the implementation guide to setting up and running an external storage for your Flow runtime data. You are going to be using the Flow drawing tool to do this.

  1. Adding a data store
  2. Selecting a data store
  3. Enabling replication
  4. Migrating to a data store
  5. Disabling replication
You may also like to see:

USING THE FLOW API

Here is the implementation guide to setting up and running an external storage for your Flow runtime data. You are going to be using the Flow inbuilt API client to do this. 

  1. Setting up storage in Flow
  2. Configuring your storage
  3. Migrating data to an external store
  4. Enabling data replication
You may also like to see:

DICTIONARY

Let’s make sure we are familiar with some key concepts:

API 
Abbreviation for Application Programming Interface. An API is a software library or protocol that facilitates interaction between different software or services. You can find the Boomi Flow API documentation here.

Cloud 
Cloud computing is a way of using a network to access hardware/software based in a remote location, on demand. The Boomi Flow Drawing Tool is a cloud-based low code platform for app development. 

Data storage
The act of using and setting up data stores. 

Data store
Location used to store your runtime data from the Boomi Flow platform externally. Also: The storage service that can store Flow states/data.

Heroku
Heroku is a cloud application platform that lets developers build and run applications in the cloud.

Runtime data
Anything that is used specifically at runtime – states, service invoker logs, user tokens, etc.

Receiver key
Used by the external storage endpoint to decrypt incoming requests from the Flow platform, and to sign any outgoing responses that are sent back to the platform.

Platform key
Used by the external storage endpoint to verify that incoming requests are indeed coming from the Boomi Flow platform (authenticity and non-repudiation). The platform key is also used to encrypt any responses that are sent back to the platform.

SUPPORT

Have a question? Could use some help? We are always available to listen to you. 

Please click the Help button on the bottom right-hand corner of this page to ask.

You may also want to check our Boomi Community for answers.