How to securely connect your Go applications to Amazon MemoryDB (or ElastiCache) for Redis using IAM
Amazon MemoryDB for Redis has supported username/password based authentication using Access Control Lists since the very beginning. But you can also use IAM based authentication that allows you to associate IAM users and roles with MemoryDB users so that applications can use IAM credentials to authenticate to the MemoryDB cluster. With this authentication method, you don't need to use a (long-lived) user password. Instead, you use an authentication token generated using AWS Signature Version 4.
There are many benefits to this approach. Instead of managing username and password based credentials, you can use IAM to centrally manage access to MemoryDB clusters. For client applications running on Amazon EC2, Amazon EKS, AWS Lambda, AWS App Runner etc., you can inject these credentials (depending on the platform e.g. profile credentials in EC2 and instance role in App Runner) - this provides greater security.
MemoryDB documentation has an example for a Java application with the Lettuce client. The process is similar for other languages, but you still need to implement it. So, let's learn how to do it for a Go application with the widely used go-redis client.
As a bonus, this is also applicable to ElastiCache for Redis, which also supports IAM authentication. There are minor differences which I will list at the end of the blog.
Let's get started....
1. Configure the MemoryDB cluster along with user and IAM role
Start by creating a MemoryDB user, Access Control list (ACL) and add the user to it. Make sure to use IAM as the authentication type.
Now, create the MemoryDB cluster. Make sure to choose version 7.0 or above and choose the ACL you created before.
While the cluster is being provisioned, create an IAM role. Use the following Trust Relationship to allow connectivity from an EC2 instance:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
}
}
]
}
Use the following permissions - replace the MemoryDB cluster, IAM username, AWS account ID and region:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
memorydb:Connect"
],
"Resource": [
"arn:aws:memorydb:<enter aws region>:<enter aws account ID>:cluster/<enter memorydb cluster name>",
"arn:aws:memorydb:<enter aws region>:<enter aws account ID>:user/<enter memorydb iam user name>"
]
}
]
}
2. Configure AWS Cloud9 environment
Create an AWS Cloud9 environment to execute the client application that will connect to MemoryDB.
- Make sure to use same VPC and subnet as the MemoryDB cluster
- Update the MemoryDB subnet group to add inbound rule for the client application connectivity.
Refer to the Getting started with Redis on AWS - the easy way! blog post that already has this covered.
3. Run the client app
Once the MemoryDB cluster and Cloud9 instance are ready, go ahead and execute the client application.
Open the Cloud9 instance terminal, clone the following GitHub repository:
git clone https://github.com/build-on-aws/aws-redis-iam-auth-golang
cd aws-redis-iam-auth-golang
Update the run.sh file to enter the MemoryDB cluster endpoint, and username. For example:
export SERVICE_NAME=memorydb
export CLUSTER_NAME=demo-iam-cluster
export CLUSTER_ENDPOINT=clustercfg.demo-iam-cluster.xyzxy4.memorydb.us-west-1.amazonaws.com:6379
export USERNAME=demo-iam-user
Once the app is up and running, simply invoke it's HTTP endpoints to verify that it works with IAM authentication.
curl -i -X POST -d '{"key":"foo1", "value":"bar2"}' localhost:8080
curl -i -X POST -d '{"key":"foo2", "value":"bar2"}' localhost:8080
curl -i localhost:8080/foo1
curl -i localhost:8080/not_there
The application super simple. That's on purpose. Let's move on to the important bits, which is about the IAM authentication part.
IAM authentication logic - Behind the scenes
You can refer to the authentication code in the GitHub repository
1) We first create a HTTP GET request that contains the cluster name as part of the URL along with username, the action (connect) and token expiry (900 seconds) as query parameters:
//...
queryParams := url.Values{
"Action": {connectAction},
"User": {userName},
"X-Amz-Expires": {strconv.FormatInt(int64(tokenValiditySeconds), 10)},
}
authURL := url.URL{
Host: clusterName,
Scheme: "http",
Path: "/",
RawQuery: queryParams.Encode(),
}
req, err := http.NewRequest(http.MethodGet, authURL.String(), nil)
//...
2) The request is then signed using PresignHTTP:
func (atg AuthTokenGenerator) Generate() (string, error) {
signedURL, _, err := atg.signer.PresignHTTP(
context.Background(),
atg.credentials,
atg.req,
hexEncodedSHA256EmptyString,
atg.serviceName,
atg.region,
time.Now().UTC(),
)
//...
}
3) The token generator is invoked in a CredentialsProvider during client creation:
//...
generator, err := auth.New(serviceName, clusterName, username, region)
client = redis.NewClusterClient(
&redis.ClusterOptions{
Username: username,
Addrs: []string{clusterEndpoint},
NewClient: func(opt *redis.Options) *redis.Client {
return redis.NewClient(&redis.Options{
Addr: opt.Addr,
CredentialsProvider: func() (username string, password string) {
token, err := generator.Generate()
return opt.Username, token
},
TLSConfig: &tls.Config{InsecureSkipVerify: true},
})
},
})
//...
This can also be used for ElastiCache!
The same approach applies - with a few changes. First, the IAM policy has to be updated to reflect ElastiCache resources (obviously!)
For example:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect" : "Allow",
"Action" : [
"elasticache:connect"
],
"Resource" : [
"arn:aws:elasticache:<enter aws region>:<enter aws account id>:<enter type - replicationgroup or serverlesscache>:<enter replicationgroup or serverlesscache name>",
"arn:aws:elasticache:<enter aws region>:<enter aws account id>:user:<enter username>"
]
}
]
}
Before you run the application, update the SERVICE_NAME environment variable to elasticache as well as the endpoint URL, cluster name and IAM username for the ElastiCache instance.
This example assumes you are using a Redis Cluster connection mode (which is the only option in case of MemoryDB). But in case of ElastiCache be mindful of whether you are using cluster-mode enabled configuration. If not, you will have to tweak the code to use redis.NewClient (instead of redis.NewClusterClient). The CredentialsProvider option will be available nonetheless.
Conclusion
Using IAM authentication has its benefits. Instead of managing username and passwords in multiple locations/applications, delegate the heavy lifting to IAM. All you need is to provide configure appropriate permissions (principle of "least privilege").
But you also need to be aware of the limitations. For e.g. IAM authentication is not supported in MULTI EXEC commands. For a complete list, refer to the documentation.
Have you tried using IAM authentication with MemoryDB or ElastiCache for Redis in other programming languages? Let me know.
Happy building!
Top comments (0)