DEV Community

Eric
Eric

Posted on • Edited on

Why are we moving away from the Transform connector (Liquid Templates) in Azure Logic Apps?

TLDR

We are working on a project that requires us to regularly transform data from one shape to another for our integrations. We decided liquid templates in the Transform connector in Logic Apps/integration accounts seemed like a reasonable solution. We quickly found out the developer experience is terrible.

Be careful, Liquid template syntax in Azure Logic Apps Transform connector varies between the tooling in VS Code. While your transform may work in VS Code, you will often find it doesn't work as expected in Azure.

What did we run into?

The extensions available in VS Code are built on a ruby flavor of liquid templates while the Transform connector is using DotLiquid version 2.0.254 (as of March 11, 2019). There are differences in syntax and capabilities.

  • https://github.com/dotliquid/dotliquid/wiki/DotLiquid-for-Designers

  • You'll find that there are more features available in the ruby flavor that don't exist in DotLiquid

  • Where did I get that 2.0.254 version? Straight from a response from the Microsoft support team. The problem is that is does not correlate to any release on DotLiquid's Github repo.

What did our development/deployment process look like?

  • Download VSCode and install Shopify Liquid Preview Alt Text
  • Create a repo
  • Branch
  • Create a transform.liquid and transform.liquid.json and start writing transforms! A great developer experience so far.
  • CTL+SHIFT+P to preview your transform against your sample data in your .liquid.json
  • Commit the template and and create a PR
  • Once your PR makes it to master, upload to development azure environment, run your logic app and start feeling the pain
  • If you run into problems, dig into DotLiquid source code and find the feature doesn't exist or the syntax is different
  • Create another "azure" version of your template and try to keep the two in sync
  • Cry on the inside

Questions for you

  • Is anyone else feeling the pain of the transform connector in Logic Apps? If so, what kind of tools have you discovered to improve developer productivity?

Proposal for Microsoft

  • Create a DotLiquid Preview extension that aligns to the version of DotLiquid used in the transform connector
  • Deploy a new Map type to integration accounts (i.e. Liquid-Ruby) and change the configuration property Liquid.UseRubyDateFormat = true;
  • Open source the Transform connector so we can make modifications and extend the capabilities

Side Notes

This is my first blog post ever. I'm interested to hear your feedback on how I can improve. Any and all feedback is welcome.

Top comments (3)

Collapse
 
david_burg profile image
David Burg

You can submit and vote for feedback and ideas for Azure Logic Apps at aka.ms/logicapps-wish

Lately Logic App has been the main contributor to DotLiquid on GitHub.

Yes we do take the DotLiquid NuGet build in Logic App. I'm not certain what was happening back when this blog was written, we do now work with DotLiquid NuGet packages. We even fixed the package built in the DotLiquid repo as it broke when old built tech certificate expired in June 2020.

There are multiple deviations and gaps between DotLiquid implementation and Shopify specification for Liquid. We have contributed more tests and GitHub issues to the open source project for that. DotLiquid could benefit of additional contributors to fix the deviations and fill the gaps - and in turn your open source contribution would make it to Azure Logic Apps.

Collapse
 
akscse profile image
Ashwini Kumar Singh

Liquid gets so complicated when you have to debug. It was still easier to debug with Shopify, but not with Azure Liquid.
Please do share your alternate solution to this.

Collapse
 
eromanowski profile image
Eric

Im not at that company anymore, but they ended up just building their own mapper in c#/azure functions