ASL Expressions with Step Functions

Step Functions are a great tool to coordinate multiple AWS services into serverless workflows. Your workflows can be linear OR conditionals OR parallel OR mapped. Each step can be a different service and inputs are passed from each step to another. More on this can be read here.

So, this post is not about how step functions work and what can they do but in this post I would like to focus on how you can pass and process inputs. There are multiple ways to pass inputs between steps and process them before they are passed. By process, I mean validating any primitives OR clean any arrays etc.

Table of Contents

• ASL
• Input Processing
• Operations

ASL

Step functions definition can be defined using Amazon States Language. More about it can be found here. It is a JSON-based structured language which defines various states and tasks performed by those states. An example state machine definition can be:

{
  "Comment": "This is a simple state machine",
  "StartAt": "InitialState",
  "States": {
    "InitialState": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME",
      "Next": "MiddleState"
    },
    "MiddleState": {
      "Type" : "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME2",
      "Next": "EndState"
    },
    "EndState": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME3",
      "End": true
    }
  }
}

Input Processing

Each state can accept InputPath, OutputPath, Parameters and ResultPath parameters. More info on this can be found here. There are 2 types of input here:

• One which is passed to the state machine. Eg,

{
  "comment": "Input comment.",
  "data": {
    "val1": 23,
    "val2": 17
  }
}

• Other which passed to each step. Eg,

"MiddleState": {
  "Type" : "Task",
  "InputPath": "$.data",
  "Resource": "arn:aws:lambda:us-east-1:123456789012:function:FUNCTION_NAME2",
  "Next": "EndState"
}

{
  "val1": 23,
  "val2": 17
}

In the reminder of the post, we will talk about the 2nd type of input. $ is a referential operator to refer to original input. . allows you to access any property in the input. For eg, $.data refers to the data property in your input object.

Operations

As I mentioned earlier, you can pass inputs to each state in multiple ways. You can pass the output from previous step as input to next. You can skip the output from previous step and pass in the same input as previous step to next step. You can combine the output and input from previous step and then pass it entirely to next step. You can create your own input by picking properties from both input and output from previous step.

Won't go into solving that confusion, it can be read here.

For the inputs validation or filtering, you can either handle that in your function, pass in the inputs as is and then in your code apply the validations.

But what if I don't want to do this in any function but want to handle it in the step definition itself. Yes, you can do that. ASL spec uses JSONPath to determine the inputs. Using JSONPath, you can accomplish some of the validations or filtering in the definition itself.

Subset of List

Sample Input

{
  "list": [1, 2, 3, 4, 5]
}

"InputPath": "$.list[1, 2]", will result into [2, 3] to be passed to the step.

Nested Property

{
  "person": {
    "name": "it",
    "email": "email",
    "phone": "phone"
  }
}

"InputPath": "$.person.email", will result into "email" to be passed to the step.

Nested Property for an item in the list

{
  "persons": [{
    "name": "it1",
    "email": "email1",
    "phone": "phone1"
  }, {
    "name": "it2",
    "email": "email2",
    "phone": "phone2"
  }, {
    "name": "it3",
    "email": "email3",
    "phone": "phone3"
  }]
}

"InputPath": "$.persons[0].email", will result into "email1" to be passed to the step.
"InputPath": "$.persons[*].email", will result into ["email1", "email2", "email3"] to be passed to the step.

Logical Filtering

{
  "scores": [
    {
      "math": 10,
      "english": 20
    },
    {
      "math": 15,
      "english": 20
    },
    {
      "math": 5,
      "english": 20
    },
    null,
    {
      "math": 20,
      "english": 20
    },
    null
  ]
}

"InputPath": "$.scores[(@.math > 10)]", will result into [{ "math": 15, "english": 20 }, { "math": 20, "english": 20 }] to be passed to the step.
"InputPath": "$.scores[(@ != null)]", will result into the same list without nulls to be passed to the step.

Existence Check

{
  "result": {
      "item": 12345,
    }
}

"InputPath": "$.[?(@.result)].result.item", will result into [12345]. Basically this checks the existence of property in the root object.

These are few of the examples which you can follow. A full list of supported operators can be found here.