Data Transformation

In Instalink, process data is passed from a parent action to a child action. Each action returns the data from its process and passes it on to be the input of its children action. Refer to "The Basics" for a more detailed explanation of how process data works.

The primary action that is used to make changes to the process data is the Transform Data action. This action should be used anytime it is necessary to directly modify the contents of the process data.

Transforms

Each Transform Data action consists of a series of instructions on how the data should be manipulated. These instructions are called "Transforms". Each transform contains an input key, transform operation (also called "Transform As"), an output key, and a series of arguments.

• Input Key: Tells the transform where in the process data to find the input value that will be sent to the operator.
• Transform As: This is the operator function that will be used to modify the value found in the process data at the input key.
• Output Key: Tells the transform where in the process data to write the result of the transform operation.
• Arguments: A plurality of additional inputs that the transform uses in the operator function. The number of arguments and their definitions are dependent on the transform operator being used.

For example, consider a situation where the following process data exists in the data flow:

{
    "record": {
        "name": "Red Sneakers",
        "sku": "jc01234us8"
    }
}

In this example, suppose that the caller of the endpoint always sends in a sku with all lower case characters. But, the external service that needs the data requires the sku value to always be in all capital letters. This is a situation where a simple data transformation is required. Create a transform action at the appropriate place in the data flow (after the initial LISTEN action and before the data is needed to be sent the external service). Give the transform action an appropriate description that will help identify the purpose of the transformation. "Set Sku to All Caps" would be a good description of this transform action.

Then add a transform record to the transform action. Instalink uses dot notation for data lookups, so type "record.sku" as the Input Key. Then select Upper Case (TEXT_UPPER_CASE) from the "Transform As" dropdown. Finally select "record.sku" as the Output Key. TEXT_UPPER_CASE has no additional arguments so this is all that is needed to be set for the operation to work. Save the action to commit it to the server. This transform action will now output the following:

{
    "record": {
        "name": "Red Sneakers",
        "sku": "JC01234US8"
    }
}

Now suppose that the requirement is that there is a different property called "altsku" that needs to be present. Simply set the Output Key to be "record.altsku" and the transform would then output the following:

{
    "record": {
        "name": "Red Sneakers",
        "sku": "jc01234us8",
        "altsku": "JC01234US8"
    }
}

It is also possible to perform an operation on an array of data. Consider the following process data:

{
    "records": [
        {
            "name": "Red Sneakers",
            "sku": "JC01234US8"
        },
        {
            "name": "Blue Sneakers",
            "sku": "JD01234US8"
        },
        {
            "name": "Green Sneakers",
            "sku": "JE01234US8"
        }
    ]
}

Include the "[]" syntax in the Input Key to inform the transform to run the operation on every value in the list. In this example setting the input key to "records[].name", the Transform As to TEXT_UPPER_CASE, and the output key to "records[].nameAllCaps" will result in the following:

{
    "records": [
        {
            "name": "Red Sneakers",
            "nameAllCaps": "RED SNEAKERS",
            "sku": "JC01234US8"
        },
        {
            "name": "Blue Sneakers",
            "nameAllCaps": "BLUE SNEAKERS",
            "sku": "JD01234US8"
        },
        {
            "name": "Green Sneakers",
            "nameAllCaps": "GREEN SNEAKERS",
            "sku": "JE01234US8"
        }
    ]
}

Notice that the output key also can make use of the "[]" syntax. It tells the transform to apply the result of each operation to each item in the array found in the process data.

The dot notation syntax can also be used to find an item from a specific index in an array. Using the previous process data example, an input key of "records[1].name" will return the value "Blue Sneakers".

Transform Action Behavior

Transform actions will execute all their transforms as quickly as possible. The processor may run transforms simultaneously. As such, there is no guaranteed order in which the transforms within a transform action will execute when they have the same output key depth. Do not rely upon the order shown in the transform action to be the same as the order that the transforms are processed. If the situation requires that the order in which transforms are processed must be consistent, for example transform A must always execute before transform B, then it is necessary to create separate Transform Actions. Actions in the data flow are always guaranteed to run in order. This makes it possible to pass the results from one transform action to the next transform action and enforce the order in which transformations are processed.

Within an individual transform action, it is important to avoid adding multiple transforms that have the same output key. While this rule is not explicitly enforced by Instalink, having multiple transforms with the same output key could cause unexpected outputs because the order in which transforms are executed is not guaranteed. Whichever transform executes last will override any previous outputs to that key.

While the order of transform execution is not guaranteed, there are some safeguards in place that make creating larger and more complicated sets of transforms possible. The transforms will always execute in the order of the depth of the output key. The less depth the key has, the earlier it will process. This allows the transform action to apply outputs to low and high depth segments of the process data in the same Transform Action.

Consider that the following output keys exist on a single Transform Action:

data.name
data.size
data.subrecords[].id
data.subrecords[].custom.value
data

The transform with the output key "data" will run first as it has a key depth of 1. Then "data.name" and "data.size" will both run simultaneously as they have the same key depth of 2. Next the transform with the output key "data.subrecords[].id" will run. Finally, "data.subrecords[].custom.value" will run as it has the deepest lookup depth. This ensures that the transform action will not override previously created data structures or objects and will instead build upon them.

Transform Operations

Instalink includes many built-in transform operations. These operations can be set directly via the operation selector in the Transform interface, or they may be referenced via a Custom Script. Operations can be found in the operation selector by searching for the operation's name, function signature, or keywords. Documentation for all transform operations can be found online on the Instalink website. The operation selector menu in the admin panel also contains a link to the documentation for each operation.

The argument inputs that appear in the transform menu correspond to the arguments listed in the documentation and are labeled accordingly. Additionally, these same arguments can be set within a custom script.

There will likely be situations where a transform cannot be accomplished by just using the base operations. Use the "Custom Script" operation to enable the transform to utilize an editable script. Please see Data Transform Scripting for more information on custom scripting.