🚀 Streamlining Terraform PR Automation with Atlantis on ECS Fargate

Have you ever experienced these Terraform headaches?

😱 Team members running Terraform commands from their laptops with different versions
😱 Your college is on vacation and you need to take over his/her work. You ask yourself, "what was deployed?"
😱 No visibility into what changes are being applied
😱 Manual processes for reviewing and applying Terraform changes
😱 Forgetting to apply changes after they're approved

Here comes Atlantis. Atlantis solves all these problems by providing a centralized, automated workflow that integrates directly with your Git repositories. It creates a standardized process for Terraform changes that everyone follows.

🤔 What is Atlantis?

Atlantis is an awesome tool that automates your Infrastructure as Code (IaC) processes by integrating directly with your Git workflow. When someone creates a pull request with Terraform changes, Atlantis automatically runs terraform plan and comments the results right on the PR. Once approved, it can run terraform apply too!

In my opinion, one of the greatest benefits of Atlantis is that it doesn't introduce a new user interface - instead, it integrates seamlessly with your existing version control system, allowing teams to perform code reviews and Terraform operations through the same familiar interface.

🚀 Why Atlantis can benefits you

✔️ Enhanced Collaboration with GitOps Approach
Atlantis brings collaboration to the forefront of infrastructure management by integrating with version control systems. Teams can review, comment, and approve Terraform changes directly within pull requests, creating a centralized platform for feedback and reviews

✔️ Increased Efficiency
By automating Terraform commands, Atlantis saves time and reduces the risk of human error. The tool handles the execution of terraform plan and terraform apply, allowing teams to focus on higher-level tasks.

✔️ Infrastructure State Management
Atlantis expertly handles the isolation and management of Terraform state files, preventing conflicts that can occur when multiple developers are changing infrastructure simultaneously using locking.

🔍 Atlantis Workflow for Terraform Automation

This diagram illustrates the Terraform Plan phase of the Atlantis workflow:

1. A developer creates a pull request with Terraform code changes
2. GitHub sends a webhook notification to Atlantis
3. Atlantis automatically runs terraform plan on the proposed changes
4. The plan output is returned to Atlantis
5. Atlantis comments the plan results directly in the PR
6. An approver reviews the PR and the Terraform plan output

This automation eliminates the need for developers to run plans locally and manually share the results, creating a standardized review process.

This diagram illustrates the Terraform Apply phase of the Atlantis workflow that happens after approval:

1. After approval, a developer comments atlantis apply on the PR
2. Atlantis executes terraform apply to implement the changes
3. The apply output is returned to Atlantis
4. Atlantis comments the results in the PR
5. The PR can now be merged, completing the workflow

In this blog, I want to demonstrate you how to deploy Atlantis on AWS ECS Fargate using Terraform and setup webhook for Atlantis!

You may ask, why AWS ECS Fargate? In my opinion running Atlantis on AWS ECS Fargate offers several advantages:

• Serverless: No need to manage EC2 instances
• Scalable: Automatically scales based on demand
• Isolated: Runs in its own container for better security
• Cost-effective: Pay only for what you use

📋 Prerequisites

To begin this, you will need:

• AWS Account with appropriate permissions
• Terraform installed
• AWS VPC with public and private subnets
• GitHub repository with Terraform code
• GitHub App ID and GitHub App Installation ID (To create these resources, you can refer to this links GitHub App, Install GitHub App)
• GitHub App private key
• Webhook secret for GitHub repository
• Route53 domain (optional but recommended for HTTPS)

🚶‍♂️ Implementation

For this I created a public repository to give you an example how I deployed Atlantis in AWS ECS Fargate.

atlantis-deployment/
├── main.tf
├── github_webhook.tf
├── secrets.tf
├── variables.tf
├── outputs.tf
├── provider.tf
├── terraform.example-tfvars.
└── tf-modules/
    └── github-repository-webhook/
        ├── main.tf
        ├── variables.tf
        ├── outputs.tf
        ├── version.tf
        └── README.md

The main.tf file contains the core infrastructure for deploying Atlantis on AWS ECS Fargate. It creates CloudWatch log group for the atlantis server. I used this terraform module to deploy Atlantis into ECS Fargate.

data "aws_caller_identity" "current" {}
data "aws_region" "current" {}

################################################################################
# Atlantis Module to run Atlantis on AWS Fargate
################################################################################

### ECS Atlantis CloudWatch Group
resource "aws_cloudwatch_log_group" "atlantis" {
  name              = "/ecs/atlantis"
  retention_in_days = 7
}

### Atlantis Server
module "atlantis" {
  depends_on = [aws_cloudwatch_log_group.atlantis]
  source     = "terraform-aws-modules/atlantis/aws"
  version    = "4.4.0"

  vpc_id          = var.vpc_id
  service_subnets = var.private_subnets_id
  name            = "atlantis"


  ## Atlantis ECS Container Definition
  atlantis = {
    environment = [
      {
        name  = "ATLANTIS_REPO_ALLOWLIST"
        value = join(",", var.atlantis_repo_allowlist)
      },
      {
        name  = "ATLANTIS_GH_APP_ID"
        value = var.github_app_id
      },
      {
        name  = "ATLANTIS_WEB_BASIC_AUTH"
        value = var.web_basic_auth
      },
      {
        name  = "ATLANTIS_WEB_USERNAME"
        value = var.web_username
      },
      {
        name  = "ATLANTIS_WEB_PASSWORD"
        value = var.web_password
      },
      {
        name  = "ATLANTIS_LOG_LEVEL"
        value = "debug"
      },
      {
        name  = "ATLANTIS_WRITE_GIT_CREDS"
        value = true
      },
      {
        name  = "ATLANTIS_ATLANTIS_URL"
        value = join("", ["https://", var.route53_record_name])
      },
      {
        name  = "ATLANTIS_GH_APP_INSTALLATION_ID"
        value = var.github_app_installation_id
      }
    ]
    secrets = [
      {
        name      = "ATLANTIS_GH_WEBHOOK_SECRET"
        valueFrom = aws_secretsmanager_secret_version.github_webhook_secret.arn
      },
      {
        name      = "ATLANTIS_GH_APP_KEY"
        valueFrom = aws_secretsmanager_secret_version.github_app_private_key.arn
      }
    ]
    log_configuration = {
      logDriver = "awslogs"
      options = {
        awslogs-group         = aws_cloudwatch_log_group.atlantis.name
        awslogs-region        = data.aws_region.current.name
        awslogs-stream-prefix = "atlantis"
      }
    }
  }

  ## ECS Service
  service = {
    enable_execute_command  = true
    task_exec_iam_role_name = "atlantis-task-execution-role"
    task_exec_secret_arns   = [aws_secretsmanager_secret_version.github_app_private_key.arn, aws_secretsmanager_secret_version.github_webhook_secret.arn]
    # Provide Atlantis permission necessary to create/destroy resources
    tasks_iam_role_name     = "atlantis-tasks-role"
    tasks_iam_role_policies = var.tasks_iam_role_policies

  }
  ## ALB
  alb_subnets             = var.public_subnets_id
  certificate_domain_name = var.certificate_domain_name
  route53_record_name     = var.route53_record_name
  route53_zone_id         = var.route53_zone_id
  create_alb              = var.create_alb
}

The github_webhook.tf file create the GitHub webhook integration for Atlantis. It uses a local module from ./tf-modules/github-repository-webhook. This configuration ensures that GitHub can send notifications to Atlantis when pull requests are created or updated.

################################################################################
# GitHub repository webhook
################################################################################

module "github_repository_webhooks" {
  depends_on = [module.atlantis]
  source     = "./tf-modules/github-repository-webhook"

  repositories = var.github_repositories

  webhook_url    = "${module.atlantis.url}/events"
  webhook_secret = aws_secretsmanager_secret_version.github_webhook_secret.secret_string
}

Optionally you can update the repository webhook manually from GitHub. Follow this guidance to set this up.

The secrets.tf file create AWS Secrets Manager and SSM Parameter Store resources to store sensitive information.

################################################################################
# Atlantis Server configuration and secrets
################################################################################
resource "aws_ssm_parameter" "atlantis_web_password" {
  name  = "/atlantis/web_password"
  type  = "SecureString"
  value = var.web_password
}
resource "aws_secretsmanager_secret" "github_app_private_key" {
  name        = "github_app_private_key"
  description = "GitHub App private key"
}
resource "aws_secretsmanager_secret_version" "github_app_private_key" {
  secret_id     = aws_secretsmanager_secret.github_app_private_key.id
  secret_string = var.github_app_private_key
}
resource "aws_secretsmanager_secret" "github_webhook_secret" {
  name        = "github_webhook_secret"
  description = "GitHub Webhook Secret"
}
resource "aws_secretsmanager_secret_version" "github_webhook_secret" {
  secret_id     = aws_secretsmanager_secret.github_webhook_secret.id
  secret_string = var.github_webhook_secret
}

Optionally you can configure Atlantis Server setting in repo side. Make sure if your use case required this additional setup.

‼️ Please adjust your own terraform variable. The variable provided in the repo is just a dummy to gives you guidance to use the project.

Run the following commands to initialize and apply your Terraform configuration:

terraform init
terraform plan
terraform apply

🎉 Congratulations that you make this far, then I need to remind you to think about your Atlantis security. In this link Atlantis gives you information what should be done or let's say to improve your security for Atlantis.

Once the deployment is complete, you can verify that Atlantis is running by accessing the URL for Atlantis or (if you not using Route53 domain) DNS name of the Application Load Balancer. Below is an example of Atlantis UI.

🏗️ Test Atlantis using Pull Request in GitHub repository

To test Atlantis in a GitHub pull request, start by creating a new branch in your repository (dev) and adding a simple Terraform configuration. Push this branch and open a pull request. Atlantis will automatically detect the PR and run terraform plan, posting the results as a comment. Review the plan output in the PR comments.

If you're satisfied with the changes, comment atlantis apply on the PR. Atlantis will then execute terraform apply and post the results. Once the changes are applied successfully, you can merge the PR. This process allows you to verify that Atlantis is correctly integrated with your GitHub repository and automates your Terraform workflow through pull requests

🎯 Conclusion – Less Headache managing Terraform!

In my opinion, Atlantis provides a powerful solution for automating Terraform workflows through pull requests. By integrating with version control systems, it enhances collaboration, improves security, and ensures consistency in infrastructure changes 🤖. It remains a valuable tool for teams looking to streamline their infrastructure as code processes. It is for me a tool that reduce the risk of human error while improving the team collaboration.

🔔 Stay tuned for my next blog.

Next topic I will be writing is about to run Atlantis manage Terraform infrastructure in multiple AWS Accounts. See you soon!