Terraform Cloud Project Bootcamp with Andrew Brown - Configuring Terraform & AWS CLI on Gitpod

Hi guys , in this piece we will be figuring out how to configure our Gitpod workspace for a project i.e installing Terraform in this case. The content will be broad enough to cove any other projects or languages that you might need.

This article is part of my Terraform journey with Terraform Bootcamp by Andrew Brown and Andrew Bayko, together with Chris Williams(I am also using his resources that he published here and the beloved Shala Warner. I am also using some other resources from Aaron Brooks so that I can better explain new terms. And a special shout out to Gwen Leigh for such a helpful outline that I used as a guide to create this series so that the documentation is easy to read!

As I learn more about Terraform, feel free to follow Andrew Brown on Youtube and see their free (and paid) content . Now let's jump in.

Table of Contents

• Why create a gitpod.yml file?
• Install the Terraform CLI
  • Considerations with the Terraform CLI changes
  • Refactoring into Bash scripts
  • Shebang Considerations
  • Execution considerations
  • Considerations for different Linux distributions.
  • Linux Permissions Considerations
• Creating a bash script to install Terraform
• Creating a bash script to install the AWS CLI
  • Add AWS credentials to our Gitpod environment
  • Create the script that automatically installs AWS
• The structure of the .gitpod.yml
  • Github Lifecycle (Init, Before, Command)
• Resources

Why create a gitpod.yml file?

A gitpod.yml file is a configuration file that is written in YAML that installs dependencies needed for your environment.
This makes it easier for continuing projects as you only need to configure the file once for a particular project and it automatically reloads as it is also stored in your preferred Version Control.
There is a base .gitpod.yml file that is automatically created whenever we launch a new workspace for anew project. Or you can run in the terminal the command below to generate the YAML file if it has not been automatically created.
gp init

Install the Terraform CLI

As we already learnt in the Git Basics, we will:

• Create a new issue and add a label and comment. Since we had already copied the template from Andrew Brown's Github, we already have the .gitpod.yml created however, we will go through the steps when creating a new project so as to properly configure your workspace.

Considerations with the Terraform CLI changes

Using the Terraform CLI installation instructions to install a local version of Terraform. We need to refer to the latest install CLI¹ instructions via Terraform documentation and change the scripting for install.

Refactoring into Bash scripts

• While fixing the Terraform CLI gpg deprecation issues, we noticed that the Bash script steps had a considerable amount more of code and we decided to create a bash script to install the Terraform CLI.
• This will keep the Gitpod Task file .gitpod.yml tidy.
• This allows us an easier to debug and executes manually Terraform CLI install .
• This will allow for better portability for other projects that need to install the Terraform CLI.

Shebang Considerations

A shebang (#!) tells the bash script what program will interpret the script e.g #!/bin/bash

ChatGPT recommend this format for bash: #!/usr/bin/env bash

• for portability for different OS distributions
• will allow us to search the user's PATH for the bash executable

Execution considerations

When executing the bash scripts we can use the ./ shorthand notation to execute the bash script as opposed to the '$ source gdjshh.sh' notation.

If we are using a script in Gitpod YAML, we need to point the script to a program to interpret it e.g $ source gdjshh.sh notation.

Considerations for different Linux distributions.

This project is built on Ubuntu.
Please check your distribution to check then change accordingly to distribution needs.

$cat /etc/os-release

>output
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Linux Permissions Considerations

Linux permissions works as follows:

1. In order to make a bash script executable, we need to change the Linux permissions for the file to be executable

chmod u+x ./bin/install_terraform_cli

alternatively:

chmod 744 ./bin/install_terraform_cli

Creating a bash script to install Terraform

Creating a bash script to automate the installation of Terraform in our Gitpod Linux environment.

But before we create the script we need to set the PROJECT_ROOT working directory. When we start our Gitpod workspace an example can be /workspace/terraform-beginner-bootcamp=2023
which we can now set in the terminal as an environment variable making it easy for us to access in the bash scripts.
We can set it using

echo $THEIA_WORKSPACE_ROOT
export PROJECT_ROOT='/workspace/terraform-beginner-bootcamp=2023'
echo $PROJECT_ROOT


#!/usr/bin/env bash

cd /workspace

sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl 

wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | 
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg

gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list

sudo apt update

sudo apt-get install terraform
cd $PROJECT_ROOT

• This will avoid us downloading junk files into our workspace
  
cd /workspace

• This line of code will update the system, ensure that gnupg, curl has been installed;


sudo apt-get update && sudo apt-get install -y gnupg software-properties-common curl

• We now install the Hashicorp GPG key
  
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg


• Verify the Hashicorp security key
  
gpg --no-default-keyring \
--keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg \
--fingerprint


• Add the official HashiCorp repository to your system. The lsb_release -cs command finds the distribution release codename for your current system, such as buster, groovy, or sid.


echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] \
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | \
sudo tee /etc/apt/sources.list.d/hashicorp.list


• Download the package information from HashiCorp and Install Terraform from the new repository.
  
sudo apt update
sudo apt-get install terraform -y


• We will now make the script executable by running the instructions above and then run the file using:
  
source ./bin/install-terraform_cli
or
./bin/install-terraform_cli


• To go back to the working directory
  
cd $PROJECT_ROOT


We will reference this file in .gitpod.yml to install the dependencies and store away the binary files whenever we (re)start a new workspace.

Creating a bash script to install the AWS CLI

1. Add AWS credentials to our Gitpod environment

• Add our AWS credentials that we want to set for the workspace
  
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-west-2


• Confirm that they are set
  env |grep AWS

• Set them to Gitpod by adding
  
gp env AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
gp env AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
gp env AWS_DEFAULT_REGION=us-west-2


We avoid using the aws configure to set the credentials for our workspace, as we do not want to use/create a credentials files that might be easily leaked when we forget to add it to .gitignore

2. Create the script that automatically installs AWS

#!/usr/bin/env bash

cd /workspace

rm -f '/workspace/awscliv2.zip'
rm -rf '/workspace/aws'

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

aws sts get-caller-identity

cd $PROJECT_ROOT

• This will avoid us downloading junk files into our workspace
  cd /workspace

• We remove previous aws cli installations in the workspace
  

rm -f '/workspace/awscliv2.zip'
rm -rf '/workspace/aws'

• Installs the goes to the https address and downloads the file, unzips it then installs it.
  
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install


• Confirms that we are using the 'correct' AWS account/credentials in json
aws sts get-caller-identity

• Switch back to the working directory
  cd $PROJECT_ROOT

We will reference this file in .gitpod.yml to install the dependencies and store away the binary files whenever we (re)start a new workspace.

The structure of the .gitpod.yml

Gitpod has tasks that allow it to launch terminals
Take a look at the sample gitpod.yml file below,

tasks:
  - name: terraform
    before: |
      source ./bin/install_terraform_cli

  - name: aws-cli
    env:
      AWS_CLI_AUTO_PROMPT: on-partial
    before: |
      source ./bin/install_aws_cli

vscode:
  extensions:
    - amazonwebservices.aws-toolkit-vscode
    - hashicorp.terraform

For new workspaces, we need to install the dependencies therefore we need the before first.

1. We will install the terraform CLI from using the bash script that we created above. (we use source to run the Terraform CLI script as it does not care if the file is to be made executable)

tasks:
  - name: terraform
    before: |
      source ./bin/install_terraform-cli

Github Lifecycle (Init, Before, Command)

We need to be mindful when using init as it will not run if we restart an existing workspace.

Resources

1. How to check your OS version in the Linux terminal
2. How to configure a Gitpod workspace
3. Install Terraform CLI¹
4. Linux Permissions
5. Installing the AWS CLI
6. Setting AWS Environment Variables
7. Gitpod Documentation