In this post, we will build a serverless email service that can send the latest 24 hours AWS Announcements to your mailbox using AWS Application Composer and AWS Serverless Application Model (SAM) like this.
Follow me for more
Table of Contents
- Prerequisites
- AWS Application Composer
- Amazon Event Bridge Schedule
- AWS Step Functions
- AWS Lambda Function
- AWS DynamoDB
- Amazon SES
- The Code
- Deployment
- Test
Prerequisites
AWS Application Composer
Recently at AWS re:Invent 2022, AWS is launching a preview of AWS Application Composer, a visual designer that you can use to build your serverless applications from multiple AWS services.
We will use AWS Application Composer to design our serverless application service.
- First, launch AWS Application Composer from your AWS Management console.
- Then create new project with configurations like this.
- Drag and drop the resources from the left side into the canvas until you get this workflow diagram
As you can see from the above diagram, we will be using several AWS services to build our serverless service.
We will be using
- Amazon Event Bridge Schedule
- AWS Step Functions
- AWS Lambda
- DynamoDB
- AWS Simple Email Service
Amazon Event Bridge Schedule
Amazon EventBridge allows you to create, run, and manage scheduled tasks at scale.
We will use Amazon Event Bridge Schedule to invoke our step functions with these configurations
In this configuration, we will set cron expression to invoke our step functions every day at 7 AM with Asia/Jakarta timezone.
AWS Step Functions
AWS Step Functions provides serverless orchestration for modern applications. Orchestration centrally manages a workflow by breaking it into multiple steps, adding flow logic, and tracking the inputs and outputs between the steps.
We will use AWS Step Functions to orchestrate our lambda functions with these configurations
for the state machine definitions, we will configure it like this so that we can invoke parseFeeds
Lambda function and then sendMail
Lambda function.
StartAt: Start
States:
Start:
Type: Pass
Next: parseFeeds
parseFeeds:
Type: Task
Next: sendMail
Resource: arn:aws:states:::lambda:invoke
Parameters:
FunctionName: ${parseFeedsArn}
Payload.$: $
sendMail:
Type: Task
Next: Done
Resource: arn:aws:states:::lambda:invoke
Parameters:
FunctionName: ${sendMailArn}
Payload.$: $
Done:
Type: Pass
End: true
AWS Lambda Function
AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources for you.
We will create two lambda functions, one for parsing AWS announcements RSS feeds and saving them to DynamoDB (parseFeeds
) and the other one for sending last 24 hours' AWS announcements from DynamoDB and sending it to your mailbox (sendMail
).
for parseFeeds
lambda function, we will configure it this way
then for sendMail
lambda function, we configure it this way
AWS DynamodDB
Fast, flexible NoSQL database service for single-digit millisecond performance at any scale
We will use DynamoDB to save the last 24 hours AWS Announcement from parseFeed lambda function and then get those items and send them to your email from sendMail lambda function.
Our DynamoDB will be configured this way
We will configure guid as partition key, then isodate as sort key, then we set expiration key so that DynamoDB will delete the record with expiration TTL above 30 minutes (because we don't need the data anymore after successfully send to your email to save Dynamodb cost)
After finishing configuring your resources, you can save your AWS Application Composer project into the project folder on your local computer as template.yml
from Menu > Save changes
Your template.yml should look like this
Transform: AWS::Serverless-2016-10-31
Resources:
StateMachine:
Type: AWS::Serverless::StateMachine
Properties:
Definition:
StartAt: Start
States:
Start:
Type: Pass
Next: parseFeeds
parseFeeds:
Type: Task
Next: sendMail
Resource: arn:aws:states:::lambda:invoke
Parameters:
FunctionName: ${parseFeedsArn}
Payload.$: $
sendMail:
Type: Task
Next: Done
Resource: arn:aws:states:::lambda:invoke
Parameters:
FunctionName: ${sendMailArn}
Payload.$: $
Done:
Type: Pass
End: true
Logging:
Level: ALL
IncludeExecutionData: true
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt StateMachineLogGroup.Arn
Policies:
- AWSXrayWriteOnlyAccess
- Statement:
- Effect: Allow
Action:
- logs:CreateLogDelivery
- logs:GetLogDelivery
- logs:UpdateLogDelivery
- logs:DeleteLogDelivery
- logs:ListLogDeliveries
- logs:PutResourcePolicy
- logs:DescribeResourcePolicies
- logs:DescribeLogGroups
Resource: '*'
- LambdaInvokePolicy:
FunctionName: !Ref parseFeeds
- LambdaInvokePolicy:
FunctionName: !Ref sendMail
Tracing:
Enabled: true
Type: STANDARD
DefinitionSubstitutions:
parseFeedsArn: !GetAtt parseFeeds.Arn
sendMailArn: !GetAtt sendMail.Arn
StateMachineLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub
- /aws/vendedlogs/states/${AWS::StackName}-${ResourceId}-Logs
- ResourceId: StateMachine
parseFeeds:
Type: AWS::Serverless::Function
Properties:
Description: !Sub
- Stack ${AWS::StackName} Function ${ResourceName}
- ResourceName: parseFeeds
CodeUri: src/Function
Handler: index.handler
Runtime: nodejs16.x
MemorySize: 128
Timeout: 30
Tracing: Active
Environment:
Variables:
TABLE_NAME: !Ref Feeds
TABLE_ARN: !GetAtt Feeds.Arn
Policies:
- DynamoDBWritePolicy:
TableName: !Ref Feeds
parseFeedsLogGroup:
Type: AWS::Logs::LogGroup
DeletionPolicy: Retain
Properties:
LogGroupName: !Sub /aws/lambda/${parseFeeds}
sendMail:
Type: AWS::Serverless::Function
Properties:
Description: !Sub
- Stack ${AWS::StackName} Function ${ResourceName}
- ResourceName: sendMail
CodeUri: src/Function2
Handler: index.handler
Runtime: nodejs16.x
MemorySize: 128
Timeout: 60
Tracing: Active
Environment:
Variables:
TABLE_NAME: !Ref Feeds
TABLE_ARN: !GetAtt Feeds.Arn
Policies:
- DynamoDBReadPolicy:
TableName: !Ref Feeds
- SESCrudPolicy:
IdentityName: example.com //Domain you will use to send email
- SESCrudPolicy:
IdentityName: example@email.com //email account you will use to receive the email in sandbox SES
sendMailLogGroup:
Type: AWS::Logs::LogGroup
DeletionPolicy: Retain
Properties:
LogGroupName: !Sub /aws/lambda/${sendMail}
Feeds:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: guid
AttributeType: S
- AttributeName: isodate
AttributeType: S
BillingMode: PAY_PER_REQUEST
KeySchema:
- AttributeName: guid
KeyType: HASH
- AttributeName: isodate
KeyType: RANGE
TimeToLiveSpecification:
AttributeName: expiration
Enabled: true
SendMailEventSchedule:
Type: AWS::Scheduler::Schedule
Properties:
ScheduleExpression: cron(0 7 * * ? *)
FlexibleTimeWindow:
Mode: 'OFF'
ScheduleExpressionTimezone: Asia/Jakarta
Target:
Arn: !Ref StateMachine
RoleArn: !GetAtt SendMailEventScheduleToStateMachineRole.Arn
SendMailEventScheduleToStateMachineRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
Effect: Allow
Principal:
Service: !Sub scheduler.${AWS::URLSuffix}
Action: sts:AssumeRole
Condition:
ArnLike:
aws:SourceArn: !Sub
- arn:${AWS::Partition}:scheduler:${AWS::Region}:${AWS::AccountId}:schedule/*/${AWS::StackName}-${ResourceId}-*
- ResourceId: SendMailEventSchedule
Policies:
- PolicyName: StartExecutionPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action: states:StartExecution
Resource: !Ref StateMachine
Amazon Simple Email Service
Amazon SES allow you to start sending email in minutes.
Notice in sendMail
function resource above that we add several policies so that our function can send email through Amazon SES
- SESCrudPolicy:
IdentityName: example.com //Domain you will use to send email
- SESCrudPolicy:
IdentityName: example@email.com //email account you will use to receive the email in sandbox SES
Makes sure you already create verified identities in Amazon SES for the domain to send email and the email account that you are going to use to receive email from Amazon SES (for sandbox SES) like this
The Code
parseFeeds functions
in your root project folder where you save your project template.yml
, run this command (make sure you already install nodejs and npm before)
mkdir src src/Function
cd src/Function
npm init -y
npm install rss-parser
touch index.js
then copy and paste this code below into index.js
// import aws sdk
const AWS = require('aws-sdk');
AWS.config.update({region: 'ap-southeast-1'});
const docClient = new AWS.DynamoDB.DocumentClient({apiVersion: '2012-08-10'});
const parse = new (require('rss-parser'))();
exports.handler = async (event) => {
const {items: feeds} = await parse.parseURL("https://aws.amazon.com/about-aws/whats-new/recent/feed/");
let parseCount = 0;
try {
await Promise.all(feeds
.filter((feed) => new Date(feed.isoDate).getTime() >= Date.now() - 24 * 60 * 60 * 1000) //filter feeds with isodate last 24 hours
.map(async (feed) => {
const params = {
TableName: process.env.TABLE_NAME,
Item: {
"guid": feed.guid,
"isodate": feed.isoDate,
"title": feed.title,
"link": feed.link,
"description": feed.contentSnippet,
"expiration": Math.round(Date.now() / 1000) + 1800 //delete feeds after 30 minutes
}
};
await docClient.put(params).promise();
parseCount++;
}));
return {
statusCode: 200,
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({parseCount})
};
} catch (error) {
console.log(error.message);
return {
statusCode: 500,
body: JSON.stringify({message: "Error parse feeds"})
};
}
}
sendMail function
in your root project folder again run this command
mkdir src/Function2
cd src/Function2
touch index.js
then copy and paste this code below into index.js
const AWS = require('aws-sdk');
AWS.config.update({region: 'ap-southeast-1'});
const dynamodb = new AWS.DynamoDB({apiVersion: "2012-08-10"});
const ses = new AWS.SES({ apiVersion: "2010-12-01" });
exports.handler = async (event) => {
try {
const {Items} = await dynamodb.scan({TableName: process.env.TABLE_NAME}).promise();
if(Items.length > 0) {
const params = {
Destination: {
ToAddresses: ["example@email.com"],
},
Message: {
Body: {
Html: {
Charset: "UTF-8",
Data: "<h1>Today's AWSome Announcements</h1>" + Items.map(
(item) =>
`<a href='${item.link.S}'>${item.title.S}</a><p>${item.description.S}</p>`
).join(""),
},
},
Subject: {
Charset: "UTF-8",
Data: "AWS Announcements",
},
},
Source: "hello@example.com",
};
const {MessageId} = await ses.sendEmail(params).promise();
return {
statusCode: 200,
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify({MessageId}),
};
}else{
throw new Error("No items found");
}
} catch (error) {
console.log(error.message);
return {
statusCode: 500,
body: JSON.stringify({message: "Error send email"})
}
}
}
Deployment
We will use SAM CLI to provision and deploy our serverless service, makes sure you already install AWS CLI and SAM CLI on your local computer and then configure AWS CLI before (check prerequisites).
First, we need to build our project with this command
sam build
you will get output like this
Building codeuri: runtime: nodejs16.x metadata: {} architecture: x86_64 functions: parseFeeds
Running NodejsNpmBuilder:NpmPack
Running NodejsNpmBuilder:CopyNpmrcAndLockfile
Running NodejsNpmBuilder:CopySource
Running NodejsNpmBuilder:NpmInstall
Running NodejsNpmBuilder:CleanUpNpmrc
Running NodejsNpmBuilder:LockfileCleanUp
Building codeuri: runtime: nodejs16.x metadata: {} architecture: x86_64 functions: sendMail
package.json file not found. Continuing the build without dependencies.
Running NodejsNpmBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam\build
Built Template : .aws-sam\build\template.yaml
Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {stack-name} --watch
[*] Deploy: sam deploy --guided
then you can start for first time deploy your serverless service with this command
sam deploy --guided
fill in the configuration below
Configuring SAM deploy
======================
Setting default arguments for 'sam deploy'
=========================================
Stack Name [mail-broadcast]: mail-broadcast
AWS Region [ap-southeast-1]: ap-southeast-1
#Shows you resources changes to be deployed and require a 'Y' to initiate deploy
Confirm changes before deploy [Y/n]: Y
#SAM needs permission to be able to create roles to connect to the resources in your template
Allow SAM CLI IAM role creation [Y/n]: Y
#Preserves the state of previously provisioned resources when an operation fails
Disable rollback [y/N]: N
Save arguments to configuration file [Y/n]: Y
SAM configuration file [samconfig.toml]:
SAM configuration environment [default]:
then your serverless application will start deploying into your AWS account.
for the next deployment you just need to use this command
sam deploy
After successful deployment, you can check into your AWS management console and find out that we successfully provision all the AWS services and deploy our lambda functions.
Test
To test our serverless service without have to wait Amazon EventBridge schedule to invoke your step functions, you can invoke your step function by going to step functions menu in your AWS management console and start execution in your step function until it shows like this
If the execution is successful like above, you will receive an email like this in your email account that you already configure into Amazon SES(email will only be sent when there is AWS Announcement in the last 24 hours).
Top comments (1)
This is a useful breakdown of a relatively common need, I think. I actually want to create a simple email send-out system for an app that I am running (moving away from a deprecated email system that is using exim4 embedded on the server), so I'll try using AWS Application composer.