Supercharge your StepFunctions productivity with local file system sync in Workflow Studio

Note: This article is now outdated, but I'll leave it public anyway. As of re:Invent 2023, Workflow Studio with local file system sync is available natively from within Application Composer. I hope that this article served as inspiration to AWS when deciding to build this.

One of my favourite AWS services is Step Functions. It's a serverless visual workflow orchestration service which, once you are up to speed with it, helps you build extremely scalable and resilient applications.

At Mathem, we are heavy users of it as an integral part of our serverless architecture. At the time of writing we have over 70 state machines running in production.

With heavy usage comes ideas on how you want the perfect developer experience to be.

The release of Workflow Studio in 2021 was a huge leap in the right direction. It lets developers quickly and intuitively design visual workflows using drag and drop instead of writing ASL in YAML, JSON or CDK.

This post assumes that you have some pre-existing experience of building state machines in Step Functions, preferably using Workflow Studio.

The problem

Despite how great Workflow Studio is, there is a gap in the full end-to-end developer experience. Once the workflow is designed, the developer has to go through the following manual steps:

1. Export the workflow as YAML or JSON
2. Import the file in their project
3. Replace placeholders in the YAML with DefinitionSubstitutions. In the vast majority of development scenarios, the workflow isn't perfect after the first iteration - it normally needs several iterations. If iterating requires manual steps, then it'll get very costly, not to mention boring, for the developer. For each change, the developer needs to:
4. Convert their ASL to JSON (if it isn't already)
5. Import it back into Workflow Studio
6. Make changes
7. GOTO 1

Below is a state machine describing the above steps. The steps with a sad smiley are the ones we want to eliminate.

The solution

My ideal solution, alike my #1 AWS wishlist item, would be an AWS supported version of Workflow Studio as a VSCode extension with a bidirectional sync between the designer and the ASL definition. Until we have that I will support this workaround.

Getting started

First, you need to install a Chrome extension from github.com/ljacobsson/sfn-workflow-studio-sync. The instructions are in the readme.

Link Workflow Studio with your local ASL file

Next, you'll need to navigate to Workflow Studio. You can also open an existing state machine and choose Edit and click through to Workflow Studio.

If the extension was successfully installed, you'll see a new button at the top right corner:

Clicking that will bring up a file picker dialog. This is using the File System Access API to create a temporary read/write access between the webpage and the selected file for the duration of the page's lifetime.

Keep your ASL file in one window and Workflow Studio in the other and watch the time saving as it happens:

This solves the export/import manual tasks in the developer experience flow from above, but we want to automate the linking of resources in the SAM template and the ASL definition. This is of course only applicable if you are a SAM user.

Link your SAM template with Workflow Studio

After the ASL file sync has been established, you'll notice two new buttons in the top right corner of the studio:

The 'Force sync'-button forces the sync to happen. In some cases it's needed, for example after changing Choice conditions.

The other button lets us link a SAM template to Workflow Studio in order to do the definition substitution mapping.

Some resource types, for example Lambda, offer a drop down in the GUI to select a hard-coded reference. Using a hard coded reference is normally a bad idea. In order to create a substitution, select 'Enter' at the top of the dropdown list.

This will bring up a dropdown representing the resource types in your SAM template along with their available attributes:

Selecting an option will update both your SAM and ASL simultaneously:

Most resource types, however, require you to complete JSON to set up your integrations. Here you simply type in the substitution variable you want to use:

You can add more than one substitution variable per JSON configuration:

Summary

This extension is heavily dependent on the DOM of the Workflow Studion, which I have no control over. Chances are that AWS will release a breaking change at any time. If that happens, please submit an issue and I'll do my best to fix it.

AWS prioritises new features based on customer feedback, so if you like this, please submit a feature request to AWS for official support <3

Follow me on Twitter (@lajacobsson) for updates on this and other serverless developer experience tools I maintain.