Mocking Amazon Web Services with LocalStack

In this article i will discuss about LocalStack for local copy of AWS and in this tutorial we will use LocalStack as Message Broker we will use SQS , SNS and Terraform for that.

Prerequisites

We’ll need:

Node.js installed
Docker installed and running
AWS and TerraForm CLI

What Is LocalStack ?

It spins up a testing environment on your local machine that provides the same functionality and APIs as the real AWS cloud environment.

We don’t need AWS in development/test environment so that’s why LocalStack is here so that we can test our Amazon services locally.

How To Run LocalStack ?

Clone LocalStack from GitHub and run docker-compose up command.

After cloning LocalStack and running docker-compose up command. you will see that it open each service on different ports.

localstack_main | Starting mock IAM (http port 4593)…
localstack_main | Starting mock Kinesis (http port 4568)…
localstack_main | Starting mock KMS (http port 4599)…
localstack_main | Starting mock Lambda service (http port 4574)…
localstack_main | Starting mock CloudWatch Logs (http port 4586)…
localstack_main | Starting mock Redshift (http port 4577)…
localstack_main | Starting mock Route53 (http port 4580)…
localstack_main | Starting mock S3 (http port 4572)…
localstack_main | Starting mock Secrets Manager (http port 4584)…
localstack_main | Starting mock SES (http port 4579)…
localstack_main | Starting mock SNS (http port 4575)…
localstack_main | Starting mock SQS (http port 4576)…
localstack_main | Starting mock SSM (http port 4583)…
localstack_main | Starting mock STS (http port 4592)…
localstack_main | Starting mock StepFunctions (http port 4585)…

Create S3 Bucket in LocalStack

Write this commands in terminal to create and get all buckets relax! we are doing this for testing LocalStack is running properly or not .

# create the bucket
$ aws --endpoint-url=http://localhost:4572 s3 mb s3://local-aws-bucket
# list all buckets
$ aws --endpoint-url=http://localhost:4572 s3 ls

Let’s Mock SQS , SNS and TerraForm

First create a separate folder for this project and write this command.

# npm initialization
$ npm init

Now we need to create queue by using sqs and a topic by using sns and we will subscribe queue with our created topic by using sns.

# creating the queue
$ aws \
sqs create-queue \
--queue-name local-queue \
--endpoint-url [http://localhost:4576](http://localhost:4576/)
--region us-east-1 \

# should return
{
  "QueueUrl": "http://localhost:4576/queue/local-queue"
}

# creating the topic
$ aws \
sns create-topic \
--name local-topic \
--endpoint-url [http://localhost:4575](http://localhost:4575/) \
--region us-east-1

# should return
{
  "TopicArn": "arn:aws:sns:us-east-1:123456789012:local-topic"
}

# create subscription
$ aws \
sns subscribe \
--notification-endpoint [http://localhost:4576/queue/local-queue](http://localhost:4576/queue/local-queue) \
--topic-arn arn:aws:sns:us-east-1:123456789012:local-topic \
--protocol sqs \
--endpoint-url=[http://localhost:4575](http://localhost:4575/) \
--region us-east-1

Now create three files consumer.js and publisher.js and main.tf with these commands.

$ touch consumer.js
$ touch publisher.js
$ touch main.tf

Now paste this code in publisher.js

// publisher.js

const AWS = require('aws-sdk');

const { promisify } = require('util');

// LocalStack uses the 'us-east-1' region by default

AWS.config.update({ region: 'us-east-1' });

// the endpoint param is important!

// if it wasn't defined AWS would request the production endpoint

const sns = new AWS.SNS({ endpoint: 'http://localhost:4575' });

// I prefer working w/ promises

// rather than w/ callbacks

// therefore I'm making "sns.publish" return promise

sns.publish = promisify(sns.publish);

const TopicArn = "arn:aws:sns:us-east-1:000000000000:local-topic"; // leave this one blank for now!

async function publish(msg) {

const publishParams = {

TopicArn,

Message: msg

};

let topicRes;

try {

topicRes = await sns.publish(publishParams);

} catch (e) {

topicRes = e;

}

console.log('TOPIC Response: ', topicRes);

}

for (let i = 0; i < 5; i++) {

publish('message #' + i);

}

Now paste this code in consumer.js

// consumer.js

const AWS = require('aws-sdk');

const { promisify } = require('util');

// again, specify default Localstack region

AWS.config.update({ region: 'us-east-1' });

// also, the same as with the publisher

const sqs = new AWS.SQS({ endpoint: 'http://localhost:4576' });

// as i said, i like promises

sqs.receiveMessage = promisify(sqs.receiveMessage);

const QueueUrl = "http://localhost:4576/queue/local-queue"; // leave this one blank for now!

const receiveParams = {

QueueUrl,

MaxNumberOfMessages: 1

};

async function receive() {

try {

const queueData = await sqs.receiveMessage(receiveParams);

if (

queueData &&

queueData.Messages &&

queueData.Messages.length > 0

) {

const [firstMessage] = queueData.Messages;

console.log('RECEIVED: ', firstMessage);

const deleteParams = {

QueueUrl,

ReceiptHandle: firstMessage.ReceiptHandle

};

sqs.deleteMessage(deleteParams);

} else {

console.log('waiting...');

}

} catch (e) {

console.log('ERROR: ', e);

}

}

// we poll every 500ms and act accordingly

setInterval(receive, 500);

publisher.js is responsible for publish message to a queue subscribe to a topic by using SNS service and consumer.js is responsible for consume that message which is on that queue.

Now install aws-sdk and run publisher.js and consumer.js by using these commands.

$ npm i aws-sdk --save
$ node publisher.js
$ node consumer.js

You will see that when running publisher.js it publish message to queue which was subscribe by a specific topic and when you run consumer.js it consume those messages and return that message to server.

Now we will try to use TerraForm for setup cloud infrastructure in local we don’t need to write those aws-cli commands again and again .

Let’s start TerraForm :

Remember main.tf file which we created previously ? yes! that file is Terraform file.

Put all this code in main.tf file

// main.tf

provider "aws" {

skip_credentials_validation = true

skip_metadata_api_check     = true

skip_requesting_account_id = true

access_key                  = "foo"

secret_key                  = "bar"

region                      = "us-east-1"

endpoints {

sns     = "http://localhost:4575"

sqs     = "http://localhost:4576"

}

}

resource "aws_sqs_queue" "local_queue" {

name = "local-queue" # same as in code!

}

resource "aws_sns_topic" "local_topic" {

name = "local-topic" # same as in code!

}

resource "aws_sns_topic_subscription" "local_sub" {

topic_arn = "${aws_sns_topic.local_topic.arn}"

endpoint  = "${aws_sqs_queue.local_queue.arn}"

protocol  = "sqs"

}

Now write these commands we are doing this because it will automatically subscribe queue to topic we don’t need to do it manually.

$ terraform init
# build infra with apply command
$ terraform apply --auto-approve

Run your consumer & publisher again! See that it’s still working

# when you're done, clean up after yourself
$ terraform destroy

I have uploaded whole code on GitHub with bonus code.

ATTENTION

I STARTED A NEW INITIATIVE FOR DEV COMMUNITY ON GITHUB NAMELY : Learn With Geeks , SO YOU WANNA JOIN IT ?
send me an email on sharma_vivek62@yahoo.com or comment on this article i will add you as member of this organization.

Thank you for reading this article!