Write Terraform Files in Go with hclwrite

I am one of the lead maintainers of the PagerDuty Terraform Provider which means I find myself writing a lot of Terraform definitions in HashiCorp Configuration Language (HCL). The provider itself contains a healthy collection of acceptance tests, but I often still write some of my own HCL to verify use cases and make sure bug fixes address specific issues raised by users.

To create these Terraform definitions I commonly write the HCL by hand. However, this can get tedious if I need to create, say, 150 of the same resource to test pagination coming from the PagerDuty API. Historically, I used a Python script that verbosely wrote out the HCL syntax like so

def createService(label, id, output):
    tf_code = "resource \"pagerduty_service\" \""+id+"\" {\n\
    name = \""+label+"\"\n\
    escalation_policy = pagerduty_escalation_policy."+str(id)+".id \n\
    alert_creation = \"create_alerts_and_incidents\" \n\
}\n\n"

    output.write(tf_code)

Functions like this createService above would be written for each resource and put inside of a loop to generate the HCL resource blocks that were needed for the definition. Honestly, this method worked fine for me. At least it did until I wondered if Go provided a better way. That's when I discovered the hclwrite package from HashiCorp.

The project documentation describes hclwrite as a package that, "deals with the problem of generating HCL configuration and of making specific surgical changes to existing HCL configurations." This turned out to be exactly the package I needed. Rather than writing out HCL syntax by hand the hclwrite package would do that for me while I just called functions to create the objects.

This article will help you get started with the hclwrite package by walking you through how I used it to generate a Terraform configuration for creating 150 Business Services in PagerDuty. I'll step through some of the concepts I struggled with and wished there were more examples of.

The hclwrite package is imported from github.com/hashicorp/hcl/hclwrite. This tripped me up a bit, partly because I'm still relatively new to Go and partly because the path to the package listed in the documentation is slightly different. For the rest of you also new to Go, your imports should look something like this:

import (
    "fmt"
    "os"

    "github.com/hashicorp/hcl/v2/hclwrite"
    "github.com/zclconf/go-cty/cty"
)

You'll notice another import for the go-cty package. This package (pronounced see-tie) provides some infrastructure for a type system that might be useful for applications that need to represent configuration values. You'll see it used throughout the examples when setting types to attribute values.

To get things started you'll need to create two different types of file objects. One for hclwrite and another for the filesystem. This can be done with the following code.

// create new empty hcl file object
hclFile := hclwrite.NewEmptyFile()

// create new file on system
tfFile, err := os.Create("bservelist.tf")
if err != nil {
    fmt.Println(err)
    return
}
// initialize the body of the new file object
rootBody := hclFile.Body()

The content of every object in hclwrite is stored in a body object. To add or append anything to an object you'll need to reference its body. In the code above you see that we named the body of the HCL document object rootBody. The first thing we need to do with rootBody is set up the provider block for the PagerDuty provider. The HCL for this block looks like this

provider "pagerduty" {
  token = "yeahRightN0tgo1ng2t3llyOuTh@t"
}

Constructing this block using the hclwrite package requires the AppendNewBlock function which is expecting two arguments. First is a string which will set the type of block and the second argument is an array of strings that act as labels for the block. In the case of this provider block we want to set the label simply as pagerduty. That would look something like this:

provider := rootBody.AppendNewBlock("provider", 
    []string{"pagerduty"})

Inside this block we need to set a token attribute to the value of a PagerDuty API Key. Remember, the attribute needs to be added to the body of the block. So, we'll first set that body value to the providerBody variable and then call SetAttributeValue, passing the label and value of the attribute as arguments.

providerBody := provider.Body()
providerBody.SetAttributeValue("token", 
    cty.StringVal(os.Getenv("PAGERDUTY_TOKEN")))

The next block we are going to create is the terraform block where we define which providers we're going to use and the versions of those providers. What's interesting about these definitions is they require two nested blocks that don't have any labels. The AppendNewBlock function handles his by accepting a nil argument for the labels. Creating the two blocks looks like this:

tfBlock := rootBody.AppendNewBlock("terraform", 
    nil)
tfBlockBody := tfBlock.Body()
reqProvsBlock := tfBlockBody.AppendNewBlock("required_providers", 
    nil)
reqProvsBlockBody := reqProvsBlock.Body()

Inside the required_providers block we need to define an attribute called pagerduty that contains a value of an object with two key-value pairs as fields. This is done by setting the value to a cty.ObjectVal as a map.

reqProvsBlockBody.SetAttributeValue("pagerduty", 
    cty.ObjectVal(map[string]cty.Value{
        "source":  cty.StringVal("PagerDuty/pagerduty"),
        "version": cty.StringVal("1.10.1"),
}))

The generated HCL for this code will look like this:

terraform {
  required_providers {
    pagerduty = {
      source  = "PagerDuty/pagerduty"
      version = "2.3.0"
    }
  }
}

Now it's time to start creating actual resources. Remember, our task was to create 150 Business Services. In HCL, a resource block is just like the other block types we've created. The main difference is resource blocks contain multiple labels. In this case, each resource block contains the pagerduty_business_service label for the resource type along with the identifier label for the resource. Because we only care about creating a whole lot of Business Services it doesn't matter what they're named. So, we're just going to use the index i from our loop to put a number variable into each Business Service name. For example, the name of the first Business Service will be "Business Service 1". The code for creating those resource blocks looks like this:

bs := rootBody.AppendNewBlock("resource", 
    []string{"pagerduty_business_service", 
        fmt.Sprintf("bs%v", i)})
bsBody := bs.Body()
bsBody.SetAttributeValue("name", 
    cty.StringVal(fmt.Sprintf("Business Service %v", i)))
rootBody.AppendNewline()

When the looping is done, and the HCL for all 150 Business Service definitions has been generated, the last thing to do is to write all the definitions out to a .tf file. There are a few ways to do this. I went for the way I was most familiar with, where I wrote the bytes from the hclwrite file object to the tfFile, like this:

tfFile.Write(f.Bytes())

And that's it. You should be able to run the configurations in the .tf file you created and populated. To see all the code used in these examples checkout the Create Terraform Files in Go project over on GitHub. This was also only an introduction to the hclwrite package. Go see the hclwrite documentation to see all the available functions.