A Guide to Durable Function Chaining with Node.js

I recently got the opportunity to work on a project where I found myself confronted with the task of meticulously partitioning operations into two distinct steps, each intricately linked and demanding sequential execution. Adding an extra layer of complexity, these steps needed to be initiated daily at designated times. While doing the research on which architecture to follow I stumbled upon the ingenious concept of Durable Function Chaining. In this blog I am excited to delve into the intricacies of crafting a meticulously orchestrated Function App, where the seamless chaining of functions takes center stage.

Introduction

In the world of serverless computing, orchestrating complex workflows efficiently is crucial. Azure Durable Functions provide a powerful solution for building such orchestrations. Function orchestration refers to the process of coordinating and managing the execution of multiple functions or tasks in a specific sequence to achieve a desired outcome or complete a complex workflow. It involves defining the order of execution, managing dependencies between tasks, handling errors, and ensuring that the overall workflow is executed correctly and efficiently. This blog post will walk you through the process of creating durable function chains using Node.js.

Prerequisites

Before diving into this tutorial, ensure you have the following:

• Node.js installed on your machine.
• Azure Functions Core Tools for local development
• Basic understanding of Azure Functions and serverless concepts

Setting Up the Development Environment

Start by creating a new Azure Functions project. Open your terminal and run the following commands:

npm install -g azure-functions-core-tools@3 --unsafe-perm true
func init DurableFunctionChaining --language javascript
cd DurableFunctionChaining

Next, install the Durable Functions extension:
npm install durable-functions

Components involved in Durable Function Chaining

Durable Function chain contains three major components. Let’s dive deeper into each of these,

• Durable Functions Http Starter- The Durable Functions HTTP Starter serves as the entry point for initiating the entire durable workflow. It's an HTTP-triggered Azure Function that is typically executed only once per workflow instance. When a client sends an HTTP request to this starter function, it starts an instance of the defined orchestrator function and returns an instance ID that uniquely identifies the workflow instance. It accepts input parameters or data from the HTTP request payload. Returns an instance ID that can be used to track and manage the workflow's progress.

• Durable Functions Orchestrator- The Durable Functions Orchestrator is responsible for controlling the flow of the entire workflow. It defines the sequence of activities to be executed, manages dependencies between tasks, and handles error scenarios and retries. The orchestrator function Coordinates the logic that chains together the activities and manages their execution. Manages state across the workflow, storing and passing data between activities. Can pause execution, waiting for certain conditions or external inputs.

• Durable Functions Activity- Durable Functions Activities Represent individual units of work that are executed as part of the workflow. These functions are designed perform specific tasks, such as data processing, validation, integration with external services, and more. Activities are invoked by the orchestrator using the callActivity method, and their outputs can be used as inputs for subsequent activities. Can be implemented as independent functions with a single responsibility. Can be reused across different orchestrations and workflows.

Creating Your First Durable Function

Now that you have understood that we basically need three components for implementing durable function chain. Let’s now go through the functions and their configurations.
Before creating the functions, you need environment ready. In VS Code install the extension Azure Functions. After it is installed, you can view the icon at the left-hand side of the VS code as,

Next step is to sign into Azure. Then the subscriptions can be seen under the RESOURCES.
Now add functions to the function app by clicking on the Azure Function app button on the WORKSPACE tab. You can create new function app or you can add functions to your existing function app.

Let's begin by creating an HTTP-triggered function. Click on the Create function option to add Durable Functions HTTP Starter

When you create a function in your project in Node.JS, it will ask you to select the folder where you want to create the functions. Language in which you are building the app, select TypeScript, Then Typescript programming model would be Model V4. Now select Durable Functions Http Starter and name it. A folder gets added to your root folder with the name of your function and this folder will contain two files,

HttpStart is the Function name and function.json will have configurations where as index.ts will have the code. Same structure can be seen for all the function that are added to the project but with different configurations.

function.json would look like

{
  "bindings": [
    {
      "authLevel": "function",
      "name": "req",
      "type": "httpTrigger",
      "direction": "in",
      "route": "orchestrators/{functionName}",
      "methods": [
        "post",
        "get"
      ]
    },
    {
      "name": "$return",
      "type": "http",
      "direction": "out"
    },
    {
      "name": "starter",
      "type": "orchestrationClient",
      "direction": "in"
    }
  ],
  "scriptFile": "../dist/HttpStart/index.js"
}

And index.ts

import * as df from "durable-functions"
import { AzureFunction, Context, HttpRequest } from "@azure/functions"

const httpStart: AzureFunction = async function (context: Context, req: HttpRequest): Promise<any> {
    const client = df.getClient(context);
    const instanceId = await client.startNew(req.params.functionName, undefined, req.body);

    context.log(`Started orchestration with ID = '${instanceId}'.`);

    return client.createCheckStatusResponse(context.bindingData.req, instanceId);
};

export default httpStart;

The client.startNew() creates the instance to call the Orchestrator as in the function.json we have specified route to point to orchestrator.

Implementing Function Chaining

Consider a scenario where we process an order. We'll create three activity functions: validateOrder, processPayment, and sendConfirmation.
Create an activities folder in your project directory.
Inside the activities folder, create three files: validateOrder.js, processPayment.js, and sendConfirmation.js. Or same can be added from Azure Function App icon in the local workspace.

In each of these files, create an activity function like the following:

// validateOrder.js
module.exports = async function (context, order) {
    // Validation logic
    return isValid;
};

// processPayment.js
module.exports = async function (context, order) {
    // Payment processing logic
    return paymentStatus;
};

// sendConfirmation.js
module.exports = async function (context, order, paymentStatus) {
    // Send confirmation logic
    return confirmationSent;
};

Now, create an orchestration function to chain these activities. In your main project folder, create a file named orchestration.js:

const df = require("durable-functions");

module.exports = df.orchestrator(function* (context) {
    const order = yield context.df.callActivity("validateOrder", context.bindingData.order);
    const paymentStatus = yield context.df.callActivity("processPayment", order);
    yield context.df.callActivity("sendConfirmation", order, paymentStatus);
});

The callActivity method of orchestrator takes the name of the activity method as first parameter, make sure you have given the same name to your activity.

Error Handling and Retries

In the orchestration function, you can implement error handling and retries using standard JavaScript try-catch blocks and delay logic.

Error handling and retries are crucial aspects of building resilient orchestrations using Durable Functions. Durable Function chaining offers built-in mechanisms for handling errors and retrying activities or orchestrations. Let's explore these concepts with an example of an order processing workflow.

We'll implement error handling and retries to ensure that the workflow can handle transient failures gracefully.

const df = require("durable-functions");

module.exports = df.orchestrator(function* (context) {
    const order = context.bindingData.order;

    try {
        // Step 1: Validate Order
        const isValid = yield context.df.callActivity("validateOrder", order);

        if (!isValid) {
            throw new Error("Invalid order");
        }

        // Step 2: Process Payment
        const paymentStatus = yield context.df.callActivityWithRetry(
            "processPayment",
            new df.RetryOptions(3, TimeSpan.FromSeconds(10)),
            order
        );

        // Step 3: Send Order Confirmation
        const confirmationSent = yield context.df.callActivity("sendConfirmation", order, paymentStatus);

        return confirmationSent;
    } catch (error) {
        // Handle errors
        context.log.error(`An error occurred: ${error}`);
        throw new Error("Failed to process order");
    }
});

It's important to note that Durable Functions provide several options for controlling retries, such as specifying the maximum number of retries, the delay between retries, and handling specific types of exceptions. You can tailor these options to your specific requirements.

Monitoring and Logging

Durable Functions offer built-in logging capabilities that you can use to monitor the progress of your orchestrations. You can also integrate with Azure Monitor for more advanced monitoring.

Testing and Debugging

You can test individual activity functions by invoking them directly. For the orchestration function, use the Durable Functions emulator or deploy it to Azure for testing.

When to use Durable Function Chains

For simpler workflows like order processing as in the above example, it is better to opt for the durable function chaining to keep the architecture streamlined. It can handle sequencing, retries, error handling, and state management effectively. For more complex workflows with multiple integration points, Azure Service Bus might be a better fit as it offers decoupled architecture. It's important to evaluate your application's requirements and choose the approach that aligns best with your goals.

Conclusion

Durable Function chaining with Node.js provides an efficient way to manage complex workflows in a serverless environment. By following the steps in this guide, you've learned how to create, implement, and manage durable function chains. Experiment with different orchestrations and explore more advanced features to build robust, scalable applications.

Additional Resources

Azure Durable Functions Documentation