DEV Community

Cover image for Terraform Enhanced Backend on Terraform Cloud with AWS access keys
AF uarabei
AF uarabei

Posted on

Terraform Enhanced Backend on Terraform Cloud with AWS access keys

#aws #devops #cloud #terraform

Enhanced backend in Terraform means a backend that can store state AND perform operations for configuration management.

Terraform has 2 enhanced backends that can store state and execute operations:

  • local backend (default, stored locally)
  • remote backend (used in Terraform Cloud)

We worked with local backend quite extensively throughout these series. This article will show how to use Terraform Cloud and remote backend that will allow us to store backend remotely and execute commands like plan and apply with the output showing in the local terminal.

In this article we will setup our Terraform configuration to work with Terraform Cloud and a remote backend.

 

Here is the plan for this tutorial:

  1. Create a workspace
  2. Setup remote backend in our configuration
  3. Initialize our Terraform configuration
  4. Add AWS access tokens to the Terraform Cloud to allow remote execution of commands

 
The code for this tutorial can be found HERE

 

In a previous article I covered how to create a Terraform Cloud account and an organization.

In order to proceed you will need to login into your Terraform Cloud account and get the name of your organization:
Getting organization name from Terraform Cloud

 

We will also need a workspace for our. Click on "New workspace". Make sure that you have selected the correct organization.
Creating Terraform workspace pt1

 

Select the CLI-driven workflow:
Creating Terraform workspace pt2

 

Enter the name (optionally add description) and press Create workspace:
Creating Terraform workspace pt3

 

And our organization has a new workspace. Good job!
Creating Terraform workspace pt4

 

Now we will see how to use it in our configuration. First, let's create a remote backend and point Terraform to it. Make sure that you put the correct organization and workspace names:



# terraform.tf

terraform {
  backend "remote" {
    hostname     = "app.terraform.io"
    organization = "terraform-blog"

    workspaces {
      name = "demo-app-on-aws"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

 

It is assumed that you done terraform login and added Terraform API token.

Since we changed the settings we need to reinitialize Terraform:



$ terraform init
# Output
Initializing modules...

Initializing the backend...

Successfully configured the backend "remote"! Terraform will automatically
use this backend unless the backend configuration changes.
Enter fullscreen mode Exit fullscreen mode

If you get this error:



│ The "remote" backend encountered an unexpected error while reading the
│ organization settings: unauthorized
Enter fullscreen mode Exit fullscreen mode

It means that your Terraform Cloud API token is not valid anymore. This problem can be fixed by running terraform login and going the steps. You can also find a step-by-step guide in this article.

 

Let's see what happens when we try to execute a dry-run:



$ terraform plan
# Output
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/runs/run-fACfrx3XtjhNh1zW

Waiting for the plan to start...

Terraform v1.2.4
on linux_amd64
Initializing plugins and modules...
╷
│ Error: error configuring Terraform AWS Provider: no valid credential sources for Terraform AWS Provider found.
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, request send failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: i/o timeout
│ 
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on main.tf line 3, in provider "aws":
│    3: provider "aws" {
│ 
╵
Operation failed: failed running terraform plan (exit 1)
Enter fullscreen mode Exit fullscreen mode

 

We are interested in the first part of the output indicating that plan is running remotely:



Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/

Waiting for the plan to start...
Enter fullscreen mode Exit fullscreen mode

 

We also got an error stating that we did not provide valid credentials for the AWS provider.

Terraform Cloud allows us to add credentials as environment variables inside the workspace. Please note that you need your AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY to complete the next step.

Select your workspace. You will see a Variable tab at the top. Click that and you will arrive at the screen where you can add environment variables. That's where we will put our AWS credentials that will allow Terraform to deploy resources remotely.
Adding environment variables to Terraform Cloud enhanced remote backend

 

Now it's time to add our AWS access tokens to the environment. Select Environment variable radio button. AWS_SECRET_ACCESS_TOKEN will need to have the "sensitive" checkbox selected. This is important. Value field is where the actual key will go. See the picture for details:

Adding AWS access token to Terraform Cloud

 

This how it is supposed to look like after you are done. Pay attention how AWS_ACCESS_KEY is public, yet AWS_SECRET_ACCESS_KEY is private.
Adding AWS access token to Terraform Cloud pt2

Let's do another dry to verify that Terraform can use access to manage the infrastructure:



$ terraform plan
# output
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/terraform-blog/demo-app-on-aws/

Waiting for the plan to start...
# lots of output here with resources
Plan: 25 to add, 0 to change, 0 to destroy.
Enter fullscreen mode Exit fullscreen mode

 

And let's apply the changes:



$ terraform apply
# lots of output but you should see this at the end
Apply complete! Resources: 25 added, 0 changed, 0 destroyed.
Enter fullscreen mode Exit fullscreen mode

 

Verify that resources with the list command:



$ terraform state list
data.aws_ami.ubuntu
data.aws_availability_zones.available
data.aws_region.current
aws_eip.nat_gateway_eip
aws_internet_gateway.internet_gateway
aws_nat_gateway.nat_gateway
aws_route_table.private_route_table
aws_route_table.public_route_table
aws_route_table_association.private
aws_route_table_association.public
aws_security_group.private_sg
aws_security_group.public_sg
aws_security_group_rule.private_in
aws_security_group_rule.private_out
aws_security_group_rule.public_http_in
aws_security_group_rule.public_https_in
aws_security_group_rule.public_out
aws_security_group_rule.public_ssh_in
aws_subnet.private_subnet
aws_subnet.public_subnet
aws_vpc.vpc
module._server_from_local_module.data.aws_ami.ubuntu
module._server_from_local_module.aws_instance.web_server
module.another_server_from_a_module.data.aws_ami.ubuntu
module.another_server_from_a_module.aws_instance.web_server
module.autoscaling_from_github.data.aws_default_tags.current
module.autoscaling_from_github.data.aws_partition.current
module.autoscaling_from_github.aws_autoscaling_group.this[0]
module.autoscaling_from_github.aws_launch_template.this[0]
module.autoscaling_from_registry.data.aws_default_tags.current
module.autoscaling_from_registry.data.aws_partition.current
module.autoscaling_from_registry.aws_autoscaling_group.this[0]
module.autoscaling_from_registry.aws_launch_template.this[0]
module.my_server_module.data.aws_ami.ubuntu
module.my_server_module.aws_instance.web_server
Enter fullscreen mode Exit fullscreen mode

 

Awesome! We have connected Terraform Cloud, added AWS API tokens in a secure manner. Now we can create, modify, and destroy resources using our local terminal with the state store remotely.

 

Finally, let's destroy the resources:



$ terraform destroy
# output
Apply complete! Resources: 0 added, 0 changed, 25 destroyed.
Enter fullscreen mode Exit fullscreen mode

 

You can also view the history of commands executed by this backend by clicking on Runs tab within your organization:
History of commands on Terraform Cloud and enhanced backend

 

This wraps up this tutorial. Thank you for reading! See you in the next article.

Top comments (0)

Read next

omobolaji_25 profile image

The Need for Security in Firms:

Favour ogungbade -

karlrobinson profile image

AWS re:Invent 2024 Reflection

Karl Robinson -

ahmedsalem2020 profile image

Leverage On-Premises Infrastructure in Amazon EKS Clusters with Amazon EKS Hybrid Nodes

Ahmed Salem -

jatinmehrotra profile image

EKS Auto Mode Unlocked for Existing Clusters with Terraform

Jatin Mehrotra -