Terraform Workflow with GitLab Runner Shell Executor based on AWS EC2 Auto Scaling Group

Introduction:

GitLab Runner is a powerful tool that allows you to run continuous integration and delivery pipelines on your own infrastructure. By using a GitLab Runner with a shell executor, you can easily execute your own custom scripts and commands on your infrastructure.

Having your own preinstalled and preconfigured GitLab runner allows you to have more control over the resources, used to execute your jobs, as well as the ability to install and configure the necessary software and dependencies. In this project, we assign an IAM role with the necessary permissions to an EC2 instance with a preconfigured GitLab runner.

It has several advantages over using Access Keys and shared runners:

1. Enhanced security: When you assign an IAM role to an EC2 instance, the EC2 instance assumes the role and can access the necessary resources without the need for access keys. This eliminates the need to store sensitive access keys on the instance, reducing the risk of unauthorized access and potential security breaches.

2. Simplified management: IAM roles provide a centralized and simplified approach to managing permissions for AWS resources. With an IAM role, you can easily manage permissions for multiple instances from a single location, rather than managing individual access keys for each instance.

3. Reduced maintenance: IAM roles can be assigned to an EC2 instance at launch time and can be easily modified later as needed. This eliminates the need to manually update access keys on the instance, reducing maintenance overhead and simplifying the management of resources.

4. Better auditing: When you assign an IAM role to an EC2 instance, all API calls made by the instance are associated with the role, making it easier to audit and track actions taken by the instance. This helps with compliance requirements and improves visibility into your AWS environment.

The disadvantage of this solution can be the cost of running EC2 instance.

Here is AWS best practice for usage IAM role instead of Access Keys: https://docs.aws.amazon.com/accounts/latest/reference/credentials-access-keys-best-practices.html

*DISCLAIMER: *
Registering runners is in the change process and depends on the Gitlab version. More information here: https://docs.gitlab.com/ee/architecture/blueprints/runner_tokens/
and here: https://docs.gitlab.com/runner/register/

About the project:

In this project, we have 2 folders with Terraform configurations:
gitlab-runner-configuration for creation and configuration of the Gitlab Runner shell executor based on AWS ASG. In this configuration, we use the default VPC with default subnets. This infrastructure will have: a SSH key pair, an auto-scaling group with a launch template, a network security group, and an IAM role with assigned policies.

Here is the Gitlab runner installation and configuration bash script:

#!/bin/bash

# make update
sudo apt update -y

# gitlab runner installation, we use Ubuntu 22.04 
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
sudo apt install -y gitlab-runner
sudo gitlab-runner register -n \
  --url https://gitlab.com/ \
  --registration-token ${registration_token} \
  --executor shell \
  --description "Shell Runner Ubuntu" \
  --tag-list "shell" \
  --run-untagged \
  --locked false
sudo gitlab-runner restart

# terraform installation
sudo apt update && sudo apt install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update -y
sudo apt install terraform=1.4.2

# infracost installation
# Downloads the CLI based on your OS/arch and puts it in /usr/local/bin
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh

# TFlint installation
sudo apt install unzip
curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash

# TFsec installation
curl -s https://raw.githubusercontent.com/aquasecurity/tfsec/master/scripts/install_linux.sh | bash

Gitlab runner infrastructure graph:

staging-infrastructure-configuration is for deploying infrastructure on AWS with Gitlab CI/CD. This infrastructure will have: 1 VPC, 1 application load balancer with 2 target groups, 3 network security groups, 1 EFS volume, and 3 EC2 instances.

Staging infrastructure graph:

The main project structure:

├── gitlab-runner-configuration
│   ├── backend.tf
│   ├── config_runner.tpl
│   └── main.tf
├── images
├── README.md
└── staging-infrastructure-configuration
    ├── backend.tf
    ├── data.tf
    ├── main.tf
    ├── modules
    │   └── vpc
    │       └── main.tf
    ├── outputs.tf
    ├── scripts
    │   ├── dev.tpl
    │   └── prod.tpl
    └── variables.tf

Prerequisites:

To follow along with this tutorial, you will need:
A GitLab account with an active project.
An AWS account with permissions to create resources and an S3 bucket for storing the Terraform backend.
The Terraform CLI is installed on your local machine.

Deployment:

Step 1: Create an SSH Key Pair

The EC2 instances require an SSH key pair for login.
You can create a key pair using the ssh-keygen command.
ssh-keygen -t rsa -b 4096 -f gitlab_runner

Step 2. Clone the repository and define the S3 bucket.

Clone the repository https://gitlab.com/Andr1500/aws-infrastructure-by-terraform.git, go to the folder gitlab-runner-configuration
and define the S3 bucket in backend.tf.

Step 3. Apply Gitlab runner configuration

A “registration_token” is required for a successful deployment. You should have enabled the project runners for this project under Settings -> CI/CD -> Runners, and here you can find the token.
Make init, plan and apply:
terraform apply -var registration_token=your_registration_token

Step 4. Push the repository to your GitLab account

Create a new project in GitLab. In your local Git repository, add the GitLab repository as a remote. Push your local repository to GitLab.

Step 5. Get an API key for the Infracost tool

Get the API key for the Infracost tool following this info: https://www.infracost.io/docs/ and add the key as a variable
into GitLab Settings -> CI/CD -> Variables. Make sure to set the variable as a mask variable.

Step 6. Run the CI/CD pipeline

Make sure you have an active Gitlab runner for the project (should be named “Shell Runner Ubuntu”). Go to CI/CD -> Pipelines -> Run pipeline
and run the pipeline.

CI/CD pipeline:

Stages Infracost calculation, TFlint check, TFSec check use special tools for checking out terraform code for cost, linter issues, and security issues. Stages Terraform validate, Terraform plan, Terraform apply, Terraform destroy are used for deploying the necessary infrastructure to AWS. Stages Terraform apply and Terraform destroy are “manual” stages, and it is necessary manually run the stages.

gitlab-ci.yml configuration file:

variables:
  GIT_STRATEGY: none # no delete files after previous steps 
  HOME_DIR: ""
  TERRAFORM_DIR: "aws-infrastructure-by-terraform/staging-infrastructure-configuration"

stages:
  - Clone repo
  - Infracost calculation
  - TFlint check
  - TFsec check
  - Terraform validate
  - Terraform plan
  - Terraform apply
  - Terraform destroy
  - Delete repo

Clone repo:
  stage: Clone repo
  script:
    - pwd
    - if [ ! -d "aws-infrastructure-by-terraform" ]; then git clone https://gitlab.com/Andr1500/aws-infrastructure-by-terraform.git; fi
    - ls $TERRAFORM_DIR

Infracost calculation:
  stage: Infracost calculation
  script:
    - export INFRACOST_API_KEY=$INFRACOST_API_KEY
    - infracost breakdown --path $TERRAFORM_DIR

TFlint check:
  stage: TFlint check
  script:
    - tflint $TERRAFORM_DIR
  allow_failure: true

TFsec check:
  stage: TFsec check
  script:
    - tfsec $TERRAFORM_DIR
  allow_failure: true

Terraform validate:
  stage: Terraform validate
  script:
    - cd $TERRAFORM_DIR
    - terraform init
    - terraform validate

Terraform plan:
  stage: Terraform plan
  script:
    - cd $TERRAFORM_DIR
    - terraform init
    - touch tfplan
    - terraform plan  -out=tfplan
  artifacts:
    paths:
      - tfplan

Terraform apply:
  stage: Terraform apply
  when: manual
  script:
    - cd $TERRAFORM_DIR
    - terraform apply  -input=false tfplan

Terraform destroy:
  stage: Terraform destroy
  when: manual
  script:
    - cd $TERRAFORM_DIR
    - terraform destroy  -auto-approve

Delete repo:
  stage: Delete repo
  script:
    - if [ -d "aws-infrastructure-by-terraform" ]; then rm -rf aws-infrastructure-by-terraform; fi
  needs:
    - Terraform destroy

Conclusion:

Automating your AWS infrastructure deployment with Terraform can save you time and effort. The project provides a streamlined approach to deploying your infrastructure, making it easy to get started and customize it to your specific needs. n this post, we explored two different ways of deploying infrastructure with Terraform: creating a GitLab Runner with the necessary configuration and deploying infrastructure for staging on demand using the GitLab Runner.

If you found this post helpful and interesting, please click the reaction button below to show your support for the author. Feel free to use and share this post!
You can also support me with a virtual coffee https://www.buymeacoffee.com/andrworld1500 .