SMS Campaigns on Amazon Pinpoint - One Time Send

Introduction

Amazon Pinpoint is a service that allows SMS's to be sent programmatically via the AWS-SDK or Boto3 libraries. (Psst - think of it like AWS's own Twilio)
I've been using Pinpoint for a year at work and wanted to share some approaches, tips, and best practices that I've learned in building applications with it. This article will cover the approach I call the "One Time Send". In this approach, we'll send out a single SMS at a time to a customer and collect the SMS delivery result.

Basics

This article assumes that you'll know the basics of AWS and building serverless applications using Lambda functions.

I like to use Serverless Framework to deploy my infrastructure and Lambda function code, but any of the big IaC frameworks can be used.

For drawing architecture diagrams, I enjoy using draw.io.

One Time Send Architecture

Campaign Data Sources

Campaign Data Sources are inputs to your SMS campaign. Here are a few examples of data sources that can be used to trigger SMS Campaigns:

External Integrations:

• CRM Systems: Customer accounts and cases in CRM systems can receive targeted SMS messages based on events / values in the CRM system.

• Web Form Submissions: Customers can fill out web forms and receive an SMS confirming their submission.

AWS Services:

• DynamoDB: Updates made to a DynamoDB table as part of a workflow in a new or existing application can be brought in as a data source for SMS campaigns.

• Manual File Uploads: Users can upload lists of SMS recipients in CSV Format to an S3 Bucket.

Required Fields

• Phone Number

Optional Fields

• Case / Account ID (For CRM Systems)
• Message Type

Validate Input and Load Task Database

This step is meant to validate the input from our Campaign Data Source and load a new "task" into the Database. Each task is another SMS we need to send out.

External Integrations

Most External Integrations will require us to set up an API Gateway endpoint to connect with the rest of our application. API Gateway can then pass the data to a Lambda function to perform the validation and load steps.

AWS Services

We can take advantage of Lambda triggers to natively integrate our AWS resources acting as Data Sources. Keep in mind that SQS can be used to add retries / reliability in the workflow.

Possible Validations:

• De-Duplication Check: If you only need to send out an SMS one time in a given day or through the duration of the whole campaign, the Lambda function here can query the Task Table to check if an SMS has already been sent.
• Data Integrity: If the Campaign Data Source has invalid numbers - think 999-999-9999 or 000-000-0000 these kinds of entries can be filtered out from the campaign.
• Active Hours: If the campaign is only supposed to run at certain hours of the day, a time check could be added to see if it's an appropriate time to send out an SMS.

Possible Transformations:

• The Pinpoint API requires numbers to be specified in the following format: (Country Code Phone Number: +11110001111) You can transform the input phone number values to match that format. You may find yourself adding a country code or taking out parentheses and dashes. Regex is your friend.

Task Database

Why insert tasks into a database? Can't we just do everything in the Validation Lambda function?

Here's what we can do with the tasks stored in a database:

• We can set up de-duplication logic in the validation step to ensure we're not sending multiple SMS to a single customer when we're not supposed to.
• We have a record of how many SMS were entered into our campaign, sent an SMS, and successfully received an SMS. We can use these metrics for some light data analysis.
• If Pinpoint has an outage, we'll be able to re-trigger the SMS sending once it comes back online.

I've been using DynamoDB as my Task Database as it pairs very nicely with Lambda functions. Enabling DynamoDB Streams will the contents of new items to be streamed to the Send SMS Lambda function.

A few other DynamoDB features that I've found very useful are time to live, on-demand scaling, and Cloudwatch metrics for Read and Write Consumed Units.

Possible Partition Keys:

• If you need to send out 1 SMS per day to a number, you could have the phone number as primary key and the date as sort key.
• If you need to send out multiple SMS to the same number and each SMS is independent of the others, you could have the phone number as primary key and a UUID as sort key.
• If you need to send out 1 SMS to a number for duration of the whole campaign, you could have the phone number as primary key without any sort key.

Possible Fields:

• Phone Number
• Date
• Delivery Result
• Time to Live (Automated Deletion)

Send SMS

This step processes the contents of the DynamoDB Streams. As new items are inserted into the database, we'll send out a new SMS for each one.

Tips for your Send SMS Function:

• Watch your Function Timeout setting, if you're inserting large amounts of data at once, each DynamoDB stream event will be large, and it'll take time to send out each SMS. As a rule of thumb, it would take about 20 seconds to send out 50 SMS.
• Increase Function Memory to 512 MB or greater to reduce the duration of the invocation.
• Send out 1 SMS per Pinpoint API call. This will give you a unique message ID per SMS sent out.
• A "successful" Pinpoint API response status doesn't mean the customer received the SMS. You'll have to enable the Pinpoint Events stream and read Delivery Statuses from the events logged to an S3 bucket. #### SMS Delivery Result

As mentioned in the previous step, we'll need to stream events from our Pinpoint application to an S3 bucket. It can take anywhere from 30 minutes to 8 hours to receive the SMS delivery result.

I like to use Kinesis Firehose to stream events directly into the S3 bucket. If you have an analytics tool that you like to use, you ingest these files into a data warehouse to gain insights into your SMS campaign performance.

Update Delivery Result

Once we see that the SMS was successfully delivered or not delivered to the customer, we can update the entries in our task database and if applicable, the original Campaign Data Source to include the delivery result. Thanks to Lambda and SQS we can listen for new files inserted into the Delivery Result S3 bucket and update those databases as soon as we know about the delivery result.

Tips for your Update Delivery Result Function:

• Set up an SQS Queue in between your S3 bucket and Lambda function for resiliency!
• Set up an SQS DLQ for that SQS queue in order to analyze failed records in production.
• Increase Timeout / Memory: Just like I mentioned earlier, this will be helpful for campaigns sending lots of SMS's.

Thanks for reading about my approach to SMS campaigns. I'm planning on posting approaches for other kinds of SMS campaigns in the coming days - stay tuned. Happy to answer any questions in the comments about the architecture above or Pinpoint.