Mastering AWS ECS with CloudFormation: A Comprehensive Guide

Introduction

In the dynamic landscape of cloud computing, efficient and secure infrastructure deployment is a cornerstone of success. AWS CloudFormation represents a powerful tool in this realm, offering automation and precision. This comprehensive guide dives into a CloudFormation template designed for establishing an Elastic Container Service (ECS) cluster across public and private subnets—a configuration that balances accessibility, security, and optimal performance.

Understanding CloudFormation and ECS

Before we delve into the template, let's understand the key concepts:

AWS CloudFormation

CloudFormation is an AWS service that helps you model and set up your Amazon Web Services resources. It allows you to use a simple text file to automate and manage the deployment of resources, avoiding the time-consuming and error-prone manual process.

AWS Elastic Container Service (ECS)

ECS is a highly scalable, high-performance container orchestration service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances or AWS Fargate.

The Architecture Overview

The proposed architecture uses AWS CloudFormation to automate the deployment of an ECS cluster. This architecture is designed to provide:

Scalability to handle varying loads.
Enhanced security through isolated network environments.
High availability across multiple availability zones.

Detailed Breakdown of the CloudFormation Template

This section will cover each component of the CloudFormation template in detail.

VPC and Subnet Configuration

VPC Setup
CIDR Block: The VPC is configured with a 10.0.0.0/16 CIDR block, offering a large range of IP addresses.
DNS Support: DNS support and hostnames are enabled for better network management and resolution.

Subnet Planning
Public Subnets: Two public subnets are designed for resources that need to be connected to the internet.
Private Subnets: Two private subnets are used for backend systems that don't require direct internet access.

Internet Gateway and Routing

Internet Gateway
The Internet Gateway serves as a bridge between the VPC and the internet, facilitating communication for resources in the public subnet.

Route Tables
Public Route Table: Routes traffic from the public subnet to the Internet Gateway.
Private Route Table: Manages internal traffic within the VPC.

NAT Gateways for Private Subnet Internet Access

The NAT Gateways are crucial for allowing resources in the private subnets to access the internet for updates and patches while keeping them secure from inbound internet traffic.

Elastic Load Balancing

Application Load Balancer (ALB)
Placement: Located within the public subnets to balance incoming internet traffic.
Functionality: Distributes traffic to different targets within the ECS cluster based on predefined rules, enhancing performance and fault tolerance.

ECS Cluster Setup

Fargate Integration
Serverless Approach: Utilizing AWS Fargate removes the need to provision and manage servers for containerized applications.
Cluster Configuration: The template includes the setup for an ECS cluster, ready to host containerized applications.

Security Groups and Network Access

Security Group for ALB
Regulates the traffic to and from the load balancer, ensuring that only legitimate requests reach the application services.
Container Security Group
Controls the communication between the load balancer and the ECS services, crucial for maintaining the integrity and security of the applications.

Setting Up an ECS Service and Task Definition

After discussing the theoretical aspects of the architecture and CloudFormation, it's time to show a practical implementation. Once the infrastructure is in place, the next critical steps involve creating the ECS service and task definitions.

ECS Service and Task Definition

An ECS service is a configuration that enables you to run and maintain a specified number of instances of a task definition simultaneously. A task definition is a blueprint for your application that describes one or more containers.

Creating an ECS Service

In our setup, we've created an ECS service that utilizes a task definition to deploy a react-app. Here's a breakdown of the components involved:

Launch Type: Utilizes Fargate for serverless deployment.
Load Balancer: An Application Load Balancer is configured to distribute traffic to the containers.

Network and Security Configuration

Network Configuration: The service is associated with the VPC and subnets created by our CloudFormation template.
Security Groups: Security groups are assigned to the service to control the traffic according to the predefined rules.

Testing the Load Balancer

With the ECS service deployed, the final step is to verify that the Application Load Balancer correctly distributes incoming traffic to our react-app service.

DNS Record Testing: By accessing the DNS name provided by the load balancer, you can confirm that the service is reachable and that the load balancing is functioning as expected.

CloudFormation Template and Deployment

Here, I provide the entire CloudFormation template code. This template is the blueprint for the entire infrastructure setup described above.

AWSTemplateFormatVersion: '2010-09-09'

Description: AWS core resources to create an ECS cluster spanning public and private subnets. Supports

             public facing load balancers.

Mappings:

  SubnetConfig:

    VPC:

      CIDR: '10.0.0.0/16'

    PublicOne:

      CIDR: '10.0.0.0/24'

    PublicTwo:

      CIDR: '10.0.1.0/24'

    PrivateOne:

      CIDR: '10.0.2.0/24'

    PrivateTwo:

      CIDR: '10.0.3.0/24'

Resources:

  VPC:

    Type: AWS::EC2::VPC

    Properties:

      EnableDnsSupport: true

      EnableDnsHostnames: true

      CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR']

PublicSubnetOne:

    Type: AWS::EC2::Subnet

    Properties:

      AvailabilityZone: !Select

        - 0

        - Fn::GetAZs: !Ref 'AWS::Region'

      VpcId: !Ref 'VPC'

      CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR']

      MapPublicIpOnLaunch: true

  PublicSubnetTwo:

    Type: AWS::EC2::Subnet

    Properties:

      AvailabilityZone: !Select

        - 1

        - Fn::GetAZs: !Ref 'AWS::Region'

      VpcId: !Ref 'VPC'

      CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR']

      MapPublicIpOnLaunch: true

PrivateSubnetOne:

    Type: AWS::EC2::Subnet

    Properties:

      AvailabilityZone: !Select

        - 0

        - Fn::GetAZs: !Ref 'AWS::Region'

      VpcId: !Ref 'VPC'

      CidrBlock: !FindInMap ['SubnetConfig', 'PrivateOne', 'CIDR']

  PrivateSubnetTwo:

    Type: AWS::EC2::Subnet

    Properties:

      AvailabilityZone: !Select

        - 1

        - Fn::GetAZs: !Ref 'AWS::Region'

      VpcId: !Ref 'VPC'

      CidrBlock: !FindInMap ['SubnetConfig', 'PrivateTwo', 'CIDR']

InternetGateway:

    Type: AWS::EC2::InternetGateway

  GatewayAttachement:

    Type: AWS::EC2::VPCGatewayAttachment

    Properties:

      VpcId: !Ref 'VPC'

      InternetGatewayId: !Ref 'InternetGateway'

  PublicRouteTable:

    Type: AWS::EC2::RouteTable

    Properties:

      VpcId: !Ref 'VPC'

  PublicRoute:

    Type: AWS::EC2::Route

    DependsOn: GatewayAttachement

    Properties:

      RouteTableId: !Ref 'PublicRouteTable'

      DestinationCidrBlock: '0.0.0.0/0'

      GatewayId: !Ref 'InternetGateway'

  PublicSubnetOneRouteTableAssociation:

    Type: AWS::EC2::SubnetRouteTableAssociation

    Properties:

      SubnetId: !Ref PublicSubnetOne

      RouteTableId: !Ref PublicRouteTable

  PublicSubnetTwoRouteTableAssociation:

    Type: AWS::EC2::SubnetRouteTableAssociation

    Properties:

      SubnetId: !Ref PublicSubnetTwo

      RouteTableId: !Ref PublicRouteTable

NatGatewayOneAttachment:

    Type: AWS::EC2::EIP

    DependsOn: GatewayAttachement

    Properties:

        Domain: vpc

  NatGatewayTwoAttachment:

    Type: AWS::EC2::EIP

    DependsOn: GatewayAttachement

    Properties:

        Domain: vpc

  NatGatewayOne:

    Type: AWS::EC2::NatGateway

    Properties:

      AllocationId: !GetAtt NatGatewayOneAttachment.AllocationId

      SubnetId: !Ref PublicSubnetOne

  NatGatewayTwo:

    Type: AWS::EC2::NatGateway

    Properties:

      AllocationId: !GetAtt NatGatewayTwoAttachment.AllocationId

      SubnetId: !Ref PublicSubnetTwo

  PrivateRouteTableOne:

    Type: AWS::EC2::RouteTable

    Properties:

      VpcId: !Ref 'VPC'

  PrivateRouteOne:

    Type: AWS::EC2::Route

    Properties:

      RouteTableId: !Ref PrivateRouteTableOne

      DestinationCidrBlock: 0.0.0.0/0

      NatGatewayId: !Ref NatGatewayOne

  PrivateRouteTableOneAssociation:

    Type: AWS::EC2::SubnetRouteTableAssociation

    Properties:

      RouteTableId: !Ref PrivateRouteTableOne

      SubnetId: !Ref PrivateSubnetOne

  PrivateRouteTableTwo:

    Type: AWS::EC2::RouteTable

    Properties:

      VpcId: !Ref 'VPC'

  PrivateRouteTwo:

    Type: AWS::EC2::Route

    Properties:

      RouteTableId: !Ref PrivateRouteTableTwo

      DestinationCidrBlock: 0.0.0.0/0

      NatGatewayId: !Ref NatGatewayTwo

  PrivateRouteTableTwoAssociation:

    Type: AWS::EC2::SubnetRouteTableAssociation

    Properties:

      RouteTableId: !Ref PrivateRouteTableTwo

      SubnetId: !Ref PrivateSubnetTwo

####

  # ALB related resources

  ####

  PublicLoadBalancerSG:

    Type: AWS::EC2::SecurityGroup

    Properties:

      GroupDescription: Access to the public facing load balancer

      VpcId: !Ref 'VPC'

      SecurityGroupIngress:

          - CidrIp: 0.0.0.0/0

            IpProtocol: -1

  PublicLoadBalancer:

    Type: AWS::ElasticLoadBalancingV2::LoadBalancer

    Properties:

      Scheme: internet-facing

      LoadBalancerAttributes:

      - Key: idle_timeout.timeout_seconds

        Value: '30'

      Subnets:

        - !Ref 'PublicSubnetOne'

        - !Ref 'PublicSubnetTwo'

      SecurityGroups: [!Ref 'PublicLoadBalancerSG']

  # A dummy target group is used to setup the ALB to just drop traffic

  # initially, before any real service target groups have been added.

  DummyTargetGroupPublic:

    Type: AWS::ElasticLoadBalancingV2::TargetGroup

    Properties:

      HealthCheckIntervalSeconds: 6

      HealthCheckPath: /

      HealthCheckProtocol: HTTP

      HealthCheckTimeoutSeconds: 5

      HealthyThresholdCount: 2

      Port: 80

      Protocol: HTTP

      UnhealthyThresholdCount: 2

      VpcId: !Ref 'VPC'

  PublicLoadBalancerListener:

    Type: AWS::ElasticLoadBalancingV2::Listener

    DependsOn:

      - PublicLoadBalancer

    Properties:

      DefaultActions:

        - TargetGroupArn: !Ref 'DummyTargetGroupPublic'

          Type: 'forward'

      LoadBalancerArn: !Ref 'PublicLoadBalancer'

      Port: 80

      Protocol: HTTP

  ####

  # ECS related resources

  ####

  ECSCluster:

    Type: AWS::ECS::Cluster

    Properties:

      ClusterName: 'enhanced-architecture-fargate'

  ContainerSecurityGroup:

    Type: AWS::EC2::SecurityGroup

    Properties:

      GroupDescription: Access to the containers

      VpcId: !Ref 'VPC'

      SecurityGroupIngress:

        - IpProtocol: tcp

          FromPort: 0

          ToPort: 65535

          SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG'

######################################

Outputs:

  VpcId:

    Description: The ID of the VPC that this stack is deployed in

    Value: !Ref 'VPC'

  PublicSubnetOne:

    Description: Public subnet one

    Value: !Ref 'PublicSubnetOne'

  PublicSubnetTwo:

    Description: Public subnet two

    Value: !Ref 'PublicSubnetTwo'

  PrivateSubnetOne:

    Description: Private subnet one

    Value: !Ref 'PrivateSubnetOne'

  PrivateSubnetTwo:

    Description: Private subnet two

    Value: !Ref 'PrivateSubnetTwo'

  ExternalUrl:

    Description: The url of the external load balancer

    Value: !Sub http://${PublicLoadBalancer.DNSName}

Deploying the Stack with AWS CLI

To deploy this CloudFormation stack, you can use the following AWS CLI command. This command initializes the creation of the resources as defined in the template.

aws cloudformation create-stack --capabilities CAPABILITY_IAM --stack-name enhanced-architecture --template-body file://./path_to_template_file.yaml

Benefits of This Architecture

Scalability

The setup is inherently scalable, easily adapting to increased traffic and workloads without significant manual intervention.

Security

The segregation of public and private subnets ensures a secure environment for backend services, protecting sensitive data and operations.

High Availability

The multi-AZ deployment ensures that the system remains operational and robust, even in the event of an individual zone's failure.

Cost-Efficiency

With AWS Fargate, you only pay for the resources your containers use, leading to optimized costs especially beneficial for varying workloads.

Conclusion

This guide presents a detailed view of deploying a scalable, secure, and highly available ECS cluster using AWS CloudFormation. The provided template is a robust foundation for any organization looking to leverage the power of AWS for their containerized applications. By automating infrastructure deployment, this approach not only saves time but also significantly reduces the potential for error, ensuring a reliable and efficient cloud environment.

References and Further Reading

The journey through AWS ECS and CloudFormation doesn't stop here. For those eager to delve deeper and expand their knowledge, the following resources offer a wealth of information:

1. AWS CloudFormation User Guide
   
   A comprehensive guide to AWS CloudFormation, detailing concepts, template references, and best practices.
   
   AWS CloudFormation User Guide

2. AWS Elastic Container Service Documentation
   
   Everything you need to know about ECS, from basics to advanced topics, is covered in the official AWS documentation.
   
   AWS ECS Documentation

3. AWS Fargate Documentation
   
   Learn more about the serverless compute engine for containers and how to use it with ECS.
   
   AWS Fargate Documentation

4. Best Practices for Security in Amazon ECS
   
   Understand how to secure your containerized applications and infrastructure in ECS.
   
   Security Best Practices in Amazon ECS

Peace!✌️