AWS Copilot is a tool that allows you to create and manage your databases, ECS applications, and other resources. All from the command line.
Very often, the success of your product is dependent on the number of users your product is able to reach. AWS Services like ECS enables you to achieve high scale while keeping costs low. ECS dynamically spawns up new instances of your application to keep up with high-load demands. It also automatically scales down the number of instances when the demand reduces.
In order to make your application available to your users, ECS must integrate with and leverage a lot of other AWS infrastructure services like the Elastic Container Registry, Target Groups, VPC, etc. This process of creating and mapping the services must be repeated for each of the environments that your application is being deployed in. And if you’re truly looking to handle scale and manage costs, you’re looking at building micro-services which entails repeating these steps for each of the environments for every service!!! Phew! I’m getting tired just thinking about it.
AWS Copilot removes the overhead of creating and mapping each of these AWS services manually, and allows us to spend time solving real world business problems. With some bash sorcery we’re able to setup and deploy the following infrastructure components using a single command
Elastic Container Registry
ECS Cluster
ECS Service
ECS Task definition
Aurora serverless cluster
Security groups, VPCs, and the works.
This tutorial assumes a good understanding of
In this tutorial, I will walk you through how to publish and update a node application using 4 simple steps
- Install and configure AWS CLI
- Install Copilot CLI
- Creating the infrastructure and deploying your application on AWS using copilot.
- Deploy an update to your application
Step 1 - Install and configure AWS
Install the AWS CLI
To install the CLI follow the steps mentioned here: AWS CLI installation instructions
Configure the AWS CLI
To configure the CLI please follow these steps:
Login to your AWS console
- Create a User Group named Administrators. Set the AdministratorAccess permission policy.
- Navigate to the Users tab and add a new user with username Administrator.
- Select the AWS Credential Type as Access Key - Programmatic access.
- Add the user to the Administrators group.
- Save the access token ID and the secret access key. Here is a video showing these steps.
Open up the terminal and enter the following command.
aws configure
Step 2 - Install Copilot CLI
Copilot CLI Installation instructions
Copilot CLI will use the credentials configured for AWS CLI.
Step 3 - Install & Start Docker
If you haven’t already, install docker. Please follow the steps mentioned here. Next is to make sure docker is installed and is running. Copilot will use docker to create images of your application.
Step 4 - Starter Project
Open the terminal and clone the starter project using this command.
git clone https://github.com/wednesday-solutions/aws-copilot-node-demo-starter.git
This repository contains a containerized application. Peek around the repo to know the files and folders.
Step 5 - Setup ECS
Open the terminal, navigate to the cloned repository and run setup-ecs.sh with the app and the environment name.
Once you run the script you’ll be asked if you’d like to deploy to a test environment, say no. We’ll get to that in a bit.
Copilot will use the default profile for credentials and configuration defined in ~/.aws/credentials and ~/.aws/config
Let’s go through the setup-esc.sh file in detail.
echo "name of the service is: $1 $2"
# 1
copilot init -a "$1" -t "Load Balanced Web Service" -n "$1-$2-svc" -d ./Dockerfile
# 2
copilot env init --name $2 --profile default --default-config
# 3
copilot storage init -n "$1-$2-cluster" -t Aurora -w
"$1-$2-svc" --engine PostgreSQL --initial-db "$1_$2_db"
# 4
copilot deploy --name "$1-$2-svc" -e "$2"
- This line creates the
manifest.yml
that contains the instructions to setup a Load Balanced Web Service. It points to the./Dockerfile
that we have in the repo
2.
This line creates the Elastic Container Repository, KMS Key, necessary IAM Roles, public & private subnets, a VPC, etc.
3.
Creates a cloudformation template to provision an Aurora serverless cluster with a PostgreSQL engine.
- Builds and deploys the docker container.
And just like that your application has been deployed! On successful execution of the the script it will give out the public DNS of your application.
Use the public DNS logged at the end to check out your deployed application. You can obtain the public URL from the associated load balancer.
Step 6 - Copilot
Open the project in your favorite editor. You’ll notice a new folder called copilot is created. The setup-ecs script also runs a command to initialize copilot.
This folder contains the ECS Services manifests and their add-ons. Here is a high level description of each file.
- manifest.yml: This file consists of configurations for: Docker Task definition Docs for the manifest can be found here
- Addons: Add-ons are CloudFormation templates. To add more resources for your application, make changes in the template. CloudFormation template reference can be found here.
- Database configuration: This folder contains the RDS database configuration. For the database the following resources were created.
Outputs
The database template injects the environment variables defined in Outputs.
Database connection details will be available in PLAYGROUNDDEVELOPCLUSTER_SECRET
Connecting to the database
Let us deploy an update after connecting to the database using the above environment variable.
....
try {
// using the database credentials from the injected output variable
const { username, host, dbname, password, port } = JSON.parse(
process.env.PLAYGROUNDDEVELOPCLUSTER_SECRET
);
console.log({ username, host, dbname, password, port });
pg({
database: dbname,
user: username,
password,
port,
host
})
.then(console.log)
.catch(console.error);
} catch (err) {
console.error('err', JSON.stringify(err));
}
....
Deploying an update
After making changes, run the update-ecs
script with the app name and the environment. An updated image of the container will be built and pushed to ECR. Once the new revision of the task definition is stable, the previous version will be brought down.
# deploy an update after the changes to the service
copilot deploy --name "$1-$2-svc" -e "$2"
Post Install
Once the service is up and running, the following might be handy.
Connect to the database
Security groups attached to the database resource will not allow for connections outside the service. Follow this guide to use an ec2 instance to connect to your database.
Obtain the database credentials
Obtain the cluster secret created by Copilot from the Secrets manager console.
Troubleshooting
In case of deployment failures, take a look at CloudFormation event logs. Copilot uses CloudFormation to manage resources.
Where to go from here
Congratulations! You have now automated the creation and updation of all of the ECS infrastructure. Take a look at the final output here.
This tedious and cumbersome process can now be done friction-free by running a single command. Use it to build your next side project, or amaze your coworkers with these newly acquired DevOps chops!
I hope you enjoyed this tutorial as much as I enjoyed writing it. If you think this is helpful please share and leave comments. Alternatively you can tweet your comments at us here.
Author:
Christin Itty
Top comments (0)