Basics of AWS Tags & Terraform with S3 - Part 1

Managing AWS resources can be an extremely arduous process. AWS doesn't have logical resource groups and other niceties that Azure and GCP have. This nonwithstanding, AWS is still far and away the most popular cloud provider in the world. Therefore, it's still very important to find ways to organize your resources effectively.

One of the most important ways to organize and filter your resources is by using AWS tags. While tagging can be a tedious process, Terraform can help ease the pain by providing several ways to tag your AWS resources. In this blog and accompanying video series, we're going to take a look at various methods and strategies to tag your resources and keep them organized efficiently.

These posts are written so that you can follow along. You will just need an environment that has access to the AWS API in your region. I typically use AWS Cloud9 for this purpose, but any environment with access will do.

Github repo: https://github.com/CloudForecast/aws-tagging-with-terraform

Tag Blocks

The first method we can use to tag resources is by using a basic tag block. Let's create a main.tf file and configure an S3 bucket to take a look at this.

Configure Terraform to use the AWS provider

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.0"
    }
  }
}

Configure the AWS Provider

provider "aws" {
  region = "us-west-2"
}

Create a random ID to prevent bucket name clashes

resource "random_id" "s3_id" {
    byte_length = 2
}

We utilize the random_id function:
https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id
to create the entropy needed in our bucket names to ensure we do not overlap with the name of another S3 bucket.

Create an S3 Bucket w/ Terraform and Tag It

resource "aws_s3_bucket" "devops_bucket" {
  bucket = "devops-bucket-${random_id.s3_id.dec}"

  tags = {
      Env = "dev"
      Service = "s3"
      Team = "devops"
  }
}

Now, let's run terraform apply -auto-approve.

Once the apply is finished, let's run terraform console and then run aws_s3_bucket.devops_bucket.tags to verify the tags:

> aws_s3_bucket.devops_bucket.tags
tomap({
  "Env" = "dev"
  "Service" = "s3"
  "Team" = "devops"
})

To exit the console, run exit or ctrl+c. You can also just run terraform state show aws_s3_bucket.devops_bucket.tags, terraform show, or just scroll up through the output to see the tags.

As you can see, AWS tags can be specified on AWS resources by utilizing a tags block within a resource. This is a simple way to ensure each s3 bucket has tags, but it is in no way efficient. Tagging every resource in AWS like this is not only tedious and the complete opposite of the DRY (Don't Repeat Yourself) principle, but it's also avoidable to an extent!

Default AWS Tags & Terraform

In order to specify deployment-wide tags, you can specify a default_tags block within the provider block. This will allow you to specify fallback tags for any resource that has no tags defined. If, however, you do specify tags on a specific resource, those tags will take precedence. Let's take a look:

Using Terraform to Create a Second S3 bucket

resource "aws_s3_bucket" "finance_bucket" {
  bucket = "cloudforecast-finance-${random_id.s3_id.dec)"

  tags = {
    Env = "dev"
    Service = "s3"
    Team = "finance"
  }
}

Once you have added the second bucket definition and saved the file, go ahead and apply the configuration with terraform apply -auto-approve.
Once you have applied, you can run terraform console and access both buckets by their resource name:

> aws_s3_bucket.devops_bucket.tags
tomap({
  "Env" = "dev"
  "Service" = "s3"
  "Team" = "devops"
})
> aws_s3_bucket.finance_bucket.tags
tomap({
  "Env" = "dev"
  "Service" = "s3"
  "Team" = "finance"
})

If we were to deploy 10s, 100s, or even 1000s of resources, this would not be very efficient. Let's add default tags to make this more efficient:

Add Default AWS Tags w/ Terraform

Within the provider block of our configuration, add the default tag in order to assign both resources the Env tag:

provider "aws" {
  region = "us-west-2"
    default_tags {
      tags = {
          Env = "dev"
    }
  }
}

Remove Env tags w/ Terraform

Now that we've added the default tags, let's remove the Env tag from the AWS S3 buckets:

resource "aws_s3_bucket" "devops_bucket" {
    bucket = "devops-bucket-${random_id.s3_id.dec}"

    tags = {
        Service = "s3"
        Team = "devops"
    }
}

resource "aws_s3_bucket" "finance_bucket" {
    bucket = "finance-bucket-${random_id.s3_id.dec}"

    tags = {
        Service = "s3"
        Team = "finance"
    }
}

Run terraform apply -auto-approve again and, once it's finished deploying,
run terraform console. Within the console, type the resource address of each S3 bucket and view the output:

> aws_s3_bucket.devops_bucket.tags
tomap({
  "Service" = "s3"
  "Team" = "devops"
})
> aws_s3_bucket.finance_bucket.tags
tomap({
  "Service" = "s3"
  "Team" = "finance"
})

Do you notice something missing? Default tags are not displayed within the tags attribute. Default tags are found within the tags_all attribute, so re-run the previous commands with tags_all replacing tags:

> aws_s3_bucket.devops_bucket.tags_all
tomap({
  "Env" = "dev"
  "Service" = "s3"
  "Team" = "devops"
})
> aws_s3_bucket.finance_bucket.tags_all
tomap({
  "Env" = "dev"
  "Service" = "s3"
  "Team" = "finance"
})

There they are! Keep this in mind. If you are querying the state to perform actions based on tags, you will want to use the tags_all attribute instead of just tags by themselves.

Tag Precedence

Now, for one last quick test to see the tag precedence in action, let's add the Env tag back to our finance bucket, but define it as prod instead of dev:

resource "aws_s3_bucket" "finance_bucket" {
  bucket = "finance-bucket-${random_id.s3_id.dec}"

  tags = {
    Env = "prod"
    Service = "s3"
    Team    = "finance"
  }
}

Run terraform apply -auto-approve again:

  # aws_s3_bucket.finance_bucket will be updated in-place
  ~ resource "aws_s3_bucket" "finance_bucket" {
        id                                   = "finance-bucket-52680"
      ~ tags                                 = {
          + "Env"     = "prod"
            # (2 unchanged elements hidden)
        }
      ~ tags_all                             = {
          ~ "Env"     = "dev" -> "prod"
            # (2 unchanged elements hidden)
        }
        # (17 unchanged attributes hidden)
    }

Notice the changes made, then run terraform console:

> aws_s3_bucket.finance_bucket.tags_all
tomap({
  "Env" = "prod"
  "Service" = "s3"
  "Team" = "finance"
})

Notice the Env tag has now been changed to prod, our updated value, overriding the default tags.

Destroy Resources

Now, if you're ready, go ahead and destroy your resources!

terraform destroy -auto-approve

Conclusion

Alright, so now that we have an idea of how to assign custom tags and default tags, join me on the next part in this series where we dive deeper!

Original Post