Trigger Lambda Functions with event filtering

AWS Lambda functions recently announced an enhancement with event-triggers for DynamoDB, Amazon SQS, Amazon Kinesis as event sources which makes it easier for event based Lambda function triggers to get invoked only based on the filter expression.
You can read about the official announcement from AWS Blog post.

As it is known, AWS Lambda function supports for various event-sources for triggering the Lambda fn such as - DynamoDB Triggers, SQS Triggers, SNS Triggers and many more! For anyone who has worked with Lambda fn events, you would have to handle a lot of cases on your Lambda function code making it a case or execution handled by the developer.
Eg. If you are working with DynamoDB Triggers/Streams, and your business logic needs the Lambda function to execute only when there is a new record put into the DynamoDB table then you would have to check for the eventName which has to be INSERT and then proceed further otherwise, end the Lambda function execution.
Now with the new announcement it makes it easier to ensure the Lambda function gets invoked only based on the event source's filter which is configured. This not only enriches the developer's journey but also saves the milliseconds of Lambda function's execution making it a cost saving in the longer run.

Setting up event-filtering on Lambda functions

The Lambda Fn triggers with event filtering can be added or updated whenever the Lambda Fn is created or updated via console, CLI, CloudFormation.
When trying to add event-filtering from console, navigate to your Lambda function -> Configuration -> Triggers and add new. In the additional settings option, Filter criteria where you would have to enter a valid JSON object of your filter pattern/criteria.

In this demo, we will add a DynamoDB trigger for filtering only "eventName":"INSERT". And to do so, we will have to add a valid JSON for the matching filter criteria.

{ "eventName" : ["INSERT"] }

Note, for adding DynamoDB Triggers, your Lambda fn should have the respective IAM policy which grants your Lambda fn access to DynamoDB.
The same from CLI can be performed with the bash command

aws lambda create-event-source-mapping \
--function-name DynamoDBEventFiltering \
--batch-size 100 \
--starting-position LATEST \
--event-source-arn arn:aws:dynamodb:us-east-1:XXXXXXXXXX:table/cars-demo \
--filter-criteria '{\"Filters\": [{\"Pattern\": \"{\"eventName\": [\"INSERT\"]}}]}'

The equivalent YAML template for CloudFormation or SAM is

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An Amazon DynamoDB trigger that logs the updates made to a table.
Resources:
  DynamoDBEventFiltering:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: app.handler
      Runtime: nodejs14.x
      CodeUri: src/
      MemorySize: 128
      Timeout: 3
      Events:
        cars-demo:
          Type: DynamoDB
          Properties:
            Stream: !GetAtt CarsDemoDynamoDB.StreamArn
            StartingPosition: TRIM_HORIZON
            BatchSize: 100
            Filters: 
              - Pattern: {\"Filters\": [{\"Pattern\": \"{\"eventName\": [\"INSERT\"]}}]}

The event filtering should follow a strict JSON format defined. You can find the rules here.
To test the Lambda function, you would need to do a DynamoDB operation such as - put, delete or update only for the DynamoDB Streams to trigger the Lambda function. If the same event JSON is used as Lambda Fn invoke JSON it would execute as the event-filtering is supported with the integration of DynamoDB, SQS, Kinesis. When performing actions such as creating an item on DynamoDB, updating it and deleting the same item, the DynamoDB Streams has triggered the Lambda fn only for the INSERT/put operation as defined by event-trigger filter criteria.

The other operations has not even invoked the Lambda function.

Pricing

AWS Lambda functions confirms on the announcement AWS Blog post that there is no additional cost. The takeaway cookie here is with this event-filtering it saves the millsesonds of execution as the Lambda Fn is not even invoked which is a good save when the Lambda Fn is invoked millions of times on a production environment.

Conclusion

The event-filtering for sources - DynamoDB, SQS and Kinesis helps Lambda Fn developers to integrate well with event-drivern architectures. This helps the developer experience in terms of integrating with DynamoDB, SQS and Kinesis but also it helps in saving cost as the Lambda Fn would not get invoked if the filter criteria is failed.
You can also refer to the AWS Blog about the same with Kinesis event filtering implementation by Ben Smith.

For a deep-dive about how DynamoDB Streams based event-filtering works -

Deep dive into Lambda event-filters for DyanmoDB

Jones Zachariah Noel for AWS Community Builders ・ Dec 12 '21

#aws #dynamodb #serverless #eventdriven