DEV Community

Cover image for AWS SAM Multi-Destination Connectors
Allan Chua for AWS Community Builders

Posted on • Edited on

AWS SAM Multi-Destination Connectors

The AWS SAM CLI team has published a new feature for SAM connectors called multi-destination connector. This makes it easier to define multiple integrations to several AWS services on the same invoking resource (Source). Its primary goal is to reduce the complexities associated with maintaining and writing IAM permissions inside SAM templates.

In this article, we will be exploring:

  1. What challenges do SAM connectors try to address.
  2. How to define SAM multi-destination connectors.
  3. How they improve developer experience compared to policy templates and the original variant of SAM connectors.
  4. Some opportunities that SAM team have to improve this current version.

A little bit of context

A few months ago, SAM connectors were introduced with the following goals in mind:

  • Simplifying the definition of IAM permissions: With SAM connectors, developers can define IAM permissions more easily and in a more granular way.
  • Reducing complexity in writing and maintaining serverless applications: SAM connectors provide an abstraction layer that makes it easier to write and maintain serverless applications. They help to simplify the process of integrating multiple AWS services and reduce the need for writing complex IAM code.
  • Improving productivity: By simplifying the definition of IAM permissions and reducing the complexity of writing custom IAM code. This directly results to improved productivity.

However, some improvement opportunities were spotted by the developer community which included:

One connector for each source to destination pair.

The first version of SAM connector requires a single connector definition between a source and destination pair. Take the code example below which includes three dynamo db definitions, a lambda function and three SAM connectors:

Transform: AWS::Serverless-2016-10-31

Resources:
  UsersTable:
    Type: AWS::Serverless::SimpleTable

  GroupsTable:
    Type: AWS::Serverless::SimpleTable

  PoliciesTable:
    Type: AWS::Serverless::SimpleTable

  #
  # PUT Function
  #
  PutFunction:
    Type: AWS::Serverless::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      CodeUri: put-endpoint/
      Environment:
        Variables:
          USERS_TABLE_NAME: !Ref UsersTable
          GROUPS_TABLE_NAME: !Ref GroupsTable
          POLICIES_TABLE_NAME: !Ref PoliciesTable
      Events:
        PutResource:
          Type: "Api"
          Properties:
            Path: "/user/put"
            Method: "PUT"

  # First connector seems tolerable
  PutUserToUserDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: UsersTable
      Permissions:
        - Write

  # Until you need another one! 
  # (Shout out to multi-table design users)
  PutUserToPolicyDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: PoliciesTable
      Permissions:
        - Write

  # And yet another one
  PutUserToGroupDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: GroupsTable
      Permissions:
        - Write

Enter fullscreen mode Exit fullscreen mode

With SAM multi-destination connectors, all connectors can now be grouped within their respective main resource (Original source / lambda function):

  PutFunction:
    Type: AWS::Serverless::Function
    Connectors:
     RWAccessConnector:
        Properties:
          Destination:
            - Id: UsersTable
            - Id: PoliciesTable
            - Id: GroupsTable
          Permissions:
            - Write
            - Read
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      CodeUri: put-endpoint/
      Environment:
        Variables:
          USERS_TABLE_NAME: !Ref UsersTable
          GROUPS_TABLE_NAME: !Ref GroupsTable
          POLICIES_TABLE_NAME: !Ref PoliciesTable
      Events:
        PutResource:
          Type: "Api"
          Properties:
            Path: "/user/put"
            Method: "PUT"
Enter fullscreen mode Exit fullscreen mode

More lines of code compared to Sam Policy Templates

When the first version was released, it required us to write more lines of code (9 lines for read or write connector, 10 for read and write connector) than what SAM policy templates require us to write (3 lines).

  PutUserToGroupDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: GroupsTable
      Permissions:
        - Write
        - Read

  # What it was trying to beat prior to multi-destination connector
  PutUserToGroupDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref GroupsTable
Enter fullscreen mode Exit fullscreen mode

Thanks to AWS SAM connectors, the overhead code of the function scaffold can now be seen as a valuable investment. As you link more resources to the main resource, the code line savings you get from using SAM connectors scales up, resulting in increased efficiency and reduced development time.

    Connectors:
     RWAccessConnector:
        Properties:
          Destination:
            - Id: UsersTable
            - Id: PoliciesTable
            - Id: GroupsTable
            - Id: UserIdentificationCardBucket # S3
            - Id: PaymentSettlementQueue # SQS
            - Id: InvoicingTopic # SNS (Diverse resource definition on unified syntax)
          Permissions:
            - Write
            - Read
Enter fullscreen mode Exit fullscreen mode

Another important advantage of AWS SAM connectors is their ability to support multiple types of destination resources using a single definition tree. This feature makes SAM connectors much more user-friendly compared to SAM policy templates.

With SAM connectors, there is no need to remember the specific property names (TableName, BucketName, QueueName, etc.) needed to link different resources, as the new syntax allows for a simplified and intuitive approach to defining resource connections

When multiple source-destination connectors are defined in separate areas of a large SAM file, it can be difficult to maintain and use them effectively.

Prior to the release of multi-destination connectors, if engineers incrementally link several resources to a function using SAM connectors, chances that they'd do it on different sections of a large SAM file are high.

Imagine engineers merging conflicting changes on a large SAM file? Sound familiar eh?

PutUserToUserDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: UsersTable
      Permissions:
        - Write

  # Imagine 200 lines in-between here

  PutUserToPolicyDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: PoliciesTable
      Permissions:
        - Write

  # Imagine 1,532 lines in-between here

  PutUserToGroupDBConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: PutFunction
      Destination:
        Id: GroupsTable
      Permissions:
        - Write

  # At this point, we can't remember 
  # where they link and what is their purpose don't we?
Enter fullscreen mode Exit fullscreen mode

Unwanted incidents resulting from scattered resource definitions can lead to challenges in code reading for team members over the long run. To avoid these issues, it is important to establish proper standards and quality gates in the development process.

Multi-destination connectors provide a convenient solution for managing destination-specific integration code by allowing developers to place all the code inside a unified YAML node (Source Resource).

This not only streamlines the development process, but also allows for greater ease of use and more efficient resource management, making it easier to be kind to our future selves.

  PutFunction:
    Type: AWS::Serverless::Function
    Connectors:
     RWAccessConnector:
        Properties:
          Destination:
            - Id: UsersTable
            - Id: PoliciesTable
            - Id: GroupsTable
            - Id: UserIdentificationCardBucket
            - Id: PaymentSettlementQueue
            - Id: InvoicingTopic 
          Permissions:
            - Write
            - Read
   # Code trimmed for brevity
Enter fullscreen mode Exit fullscreen mode

What features can SAM CLI team still improve?

  • Although SAM connectors offer support for a wide range of resources, they still do not cover all the resources that are available in SAM Policy Templates. It would be particularly beneficial to have connectors that support AWS Secrets Manager and AWS SSM Parameter Store, as these are two of the most commonly used features when building integrations with non-AWS resources in serverless stacks.

  • In my humble opinion, there is still room for improvement in SAM connectors. By implementing a more concise syntax, we can reduce the amount of boilerplate code required. For example, the following syntax could be used to define multiple connectors for a single function

  PutFunction:
    Type: AWS::Serverless::Function
    WriteConnectors:
      - Id: SampleTableA
      - Id: SampleTableB
      - Arn: arn:aws:secretsmanager:::secret:sample-credential-Lw92LOwe
Enter fullscreen mode Exit fullscreen mode

This would make it easier to manage multiple connectors and simplify the development process even further.

  • It would be really cool if the SAM CLI team could introduce a linting rule that enforces the usage of multi-destination connectors over 1-1 destination source pairs. This would encourage developers to adopt the more efficient and streamlined approach offered by multi-destination connectors, leading to more consistent code and better resource management.

Summary

  • Using SAM Multi-destination connectors can improve the readability and maintainability of large SAM templates, making it easier to manage integrations between resources.
  • SAM Multi-destination connectors reduce integration code length, leading to more efficient code and faster development times.
  • SAM Multi-destination connectors group connectors by the source, leading to a more organized integration code.
  • We are no longer constrained to use one-to-one source to destination pair connector syntax.
  • There are still some opportunities to improve SAM connector syntax after this.

Wrapping Up!

I'm eager to hear your thoughts on AWS SAM Multi-destination connectors. What do you think could be improved, and how would you improve it if you had the chance to provide feedback to the SAM CLI team? Share your comments below!

Related Literature

Top comments (0)