Tagging with Terraform

Sourced from my blog.

Using cloud resources can be accelerating to the business, liberating to the engineering teams, and expensive to bank account. If you have every left a large compute or database instance running over a weekend (or accidentally committed API keys to GitHub) it is easy to experience a large increase in operating costs for the month. Using the resource metadata tagging functionality responsible parties(1) can audit, track, and manage resources. It is even possible to enact automatic based on tag values (or lack thereof).

Basic Usage

The basic tagging structure in Terraform is straightforward enough. If you are familiar with the JSON data interchange format you should recognize this immediately. It is a Java inspired object declaration with quoted key values pairs using a colon as separator and a comma as a delimiter. Yeay, another DSL to learn. #welcometowebdev

...
"aws_resource_type" "aws_resource" {
    ...
    tags {
        "key": "value",
        "Name": "Value",
        "department": "engineering",
        "team": "core_api",
        "app": "name",
        "env": "dev"
        ...
    }
    ...
}

Nothing to difficult about that. Most (not all) resources in AWS support tagging in this manner. Straight forward JSON styled key/value pairs. Terraform even allows us to use variables in place of values, but not keys(2).

...
"aws_resource_type" "aws_resource" {
    ...
    tags {
        "key": "value",
        "Name": "Value",
        "department": "${var.dept_name",
        "team": "${var.team_name",
        "app": "${var.app_name",
        "env": "${var.app_env}"
        ...
    }
    ...
}

"But David" you will say "application infrastructure can be very complicated. Will I have to copy / paste the 'tag' attribute all over the place?" Short answer; No. Long Answer: ...

Advanced Usage

Using some Terraform-fu we can assign the default sets of key/value pairs to a map type variable (local or imported) and that variable as the default tags data set. The format changes slightly between a JSON object to a Terraform (HCL) map data type; but not by much.

variable "default_tags" { 
    type = "map" 
    default = { 
        key: "value",
        Name: "Value",
        department: "${var.dept_name",
        team: "${var.team_name",
        app: "${var.app_name",
        env: "${var.app_env}"
  } 
}
...

With the tags abstracted as a map the tag attribute for the resource is linimized to a one liner.

...
"aws_resource_type" "aws_resource" {
    ...
    tags = var.default_tags
    ...
}
...

The power level up is to merging the default tags map variable with custom inline tags. We for this we get fancy and start using the TF merge(). Providing the default tag map variable as one function argument and the custom tags as a second map type function argument we get to use both the default provided tags AND custom inline tags! Boom, Magic!

...
"aws_resource_type" "aws_resource" {
    ...
    tags = "${merge(map( 
            "Special_Key", "some special value", 
            "Special_Key_2", "some other special value",
            ...
        ), var.default_tags)}"
    ...
}
...

Wrap Up

Hashicorp continues to improve Terraform with each release. Say what you will about HCL being nearly JSON but with additional functionality; it is a powerful and featurful tool set to manage a projects infrastructure. With an appropriate tagging strategy it also becomes a powerful way to track that infrastructure as well.

Notes

• I did not say "managers" on purpose. In a true DevOps environment every developer, operator, SysAdmin, on every team within every technology value stream is should be aware of how and responsible for the system as a whole. Share the burden. No silos.
• Using map() it is possible to use a variable as a key. This is because map() evaluated the variable "key" before returning.
• This concept weirded me out at out first. But after thinking about the mechanics it makes sense. Why would the last operation in a function be on a variable NOT being returned?

Additional Reading

• https://weidongzhou.wordpress.com/2018/12/13/how-to-do-tagging-efficiently-in-terraform/
• https://groups.google.com/forum/#!topic/terraform-tool/1yjotodsBog
• https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html
• https://blog.scottlowe.org/2018/06/11/using-variables-in-aws-tags-with-terraform/
• https://blog.scottlowe.org/2018/06/11/using-variables-in-aws-tags-with-terraform/
• https://www.terraform.io/docs/configuration/locals.html (0.12+)