Terramate this SAP BTP!

Introduction

Let us assume that we have a company that wants to standardize the setup of SAP BTP subaccounts.

The company decides to define a standard setup of SAP BTP subaccounts with the following constraints:

• A subaccount should be created under a directory
• Every subaccount should get assigned some basic entitlements for app development
• A Cloud Foundry environment should be created
• Emergency subaccount admins should be added
• The resources should be labelled according to the company’s standards

Of course, the company wants to leverage Terraform i.e. the Terraform provider for SAP BTP for the setup. It uses a three-stage approach namely a subaccount for development, test, and production.

To get some inspiration it checks the available samples for SAP BTP available on GitHub. It finds a sample that seems to fit to the requirements. Maybe some adoptions are needed, but it is a good starting point.

The sample uses a Terraform module to setup a subaccount. It creates the three stages by iterating over a list containing the stages and executes the module which results in the expected setup.

While the example is a good starting point to get ideas on how to do such a setup and what options you have with Terraform, it also shows some drawbacks that we should address:

• Although it makes sense to keep the setup logically together, the current configuration is tightly coupled in one configuration. The blast radius is high as all three stages might be affected by a change.
• It is difficult to distinguish environments when it comes to e.g., introducing variants depending on the stage.
• Let us say we want to test a new version of the module or a new version of the Terraform provider on the development environment, this is difficult to achieve.
• One huge state gets created containing all three environments and this is definitly a too tight coupling. This makes follow-up activities like drift detections hard to do.

Let us be fair, the code on GitHub is just a sample, but it shows that you need to design the setup properly to avoid these drawbacks.

The "usual" solution for this is to split the configuration into three, one for each stage. This makes things more self contained, but now we have another huge drawback: how do we keep the configurations in sync? Do we copy&paste the configurations? This is not a good idea as it is error-prone and not maintainable on the long run.

Terraform per se does not provide a perfect solution for this, but fortunately there is a huge ecosystem around Terraform that we can leverage. One tool that is worth to take a closer look at is Terramate. This tool seems to exactly address the issues we are facing. So let us try it out.

What is Terramate?

Terramate according to its documentation is a productivity tool for Terraform. Its value proposition is to fill the gaps mentioned before (and even more) that come into play when dealing especially with large/complex Terraform configurations by helping you around automation, orchestration, and observation.

Now you might say: "oh no, not another tool" or at least "hopefully I do not need to learn/use another language in addition to the configuration language that comes with Terraform".

I think this is where the Terramate team made some very smart decisions:

• Terramate is acting on top of Terraform, so no restrictions concerning standard Terraform functionality. It enhances the features and functionalities of Terraform by covering the gaps. It is "just" another CLI you must install, but it doesn’t try to be a substitute for the Terraform CLI.
• It uses the Terraform configuration language to configure the additional features and functions that come with Terramate. No need to leave the realms of the language you are anyway using when working with Terraform.

Before diving into the application of Terramate we must do some homework around the concepts of Terramate.

Note - There is a lot of features and functionality that Terramate provides. We will not cover all of them in this blog post. We will focus on the features that help us to address the issues we are facing with the current setup. If you want to get an overview or want to dig deeper in some topics, you should check the Terramate documentation.

The main concept that will help us is Terramate's concept of stacks. Stacks put an additional "layer" on top of Terraform by making the configuration and state a well-defined unit that can be managed via Terramate based on the Terramate configuration. Using stacks we can define stack-specific or stack agnostic Terramate functionality e.g. code-generation that probably helps us getting rid of copy&pasting activities.

Now you might say "great another layer equals more complexity ... as if the scenario wasn't complex enough". We will see soon that this is not the case. Of course, you must understand the additional concepts, but they fall in place quite well and help you to manage the complexity in a convenient way. Having said that, complex things remain complex (no matter what management tries to tell you), but Terramate helps you to manage the complexity in a more structured way.

Before repeating what is anyway available in the Terramate documentation, let us make things tangible and rebuild the setup for the dev-test-prod scenario leveraging Terramate.

Applying Terramate to the dev-test-prod scenario

The staged setup with dev, test and production is a perfect candidate for applying the stack concept. Let us start from scratch in an empty directory. First, we create the following directory structure to host the stacks on the file system:

| - stacks
|   | - dev
|   | - test
|   | - prod

Next, we create the stack configuration for the three stages via the Terramate CLI in each of the directories via the Terramare CLI. We key in the following commands:

terramate create --name "development" --description "Stack for BTP development setup" --tags "dev" stacks/dev
terramate create --name "testing" --description "Stack for BTP testing setup" --tags "tst" stacks/test
terramate create --name "production" --description "Stack for BTP production setup" --tags "prd" stacks/prod

As you can see, we gave each stack a name, a description, and a tag. I love the concept of tags in general, so great to have this feature in Terramate to flexibly categorize the stacks.

As a result, we find a stack.tm.hcl file in every directory. This file contains the stack-specific metadata. Here an example for the development stack:

stack {
  name        = "development"
  description = "Stack for BTP development setup"
  tags        = ["dev"]
  id          = "29300b60-f0bb-4dda-bb4a-f0320148b0ed"
}

As you can see the content reflects the parameters we defined. In addition, it contains a unique identifier for the stack.

After we have initialized the stacks, we want to put in the necessary Terraform configuration. We do not want to copy&paste the configuration back and forth, but we want to define things once and use the configuration in the different stacks considering the specific requirements for the stages.

To solve that Terramate brings another functionality to the table namely code genration. We will use this feature to generate the basic setup for the stacks. I am a big pro-ponent of code generation instead of generic magic, so this feature is very appealing to me.

Generating the Terraform configuration

Let us now create the stage/stack-specific Terraform configurations. We start with the provider configuration. The layout of the provider should be the same for all three stages. However, we want to allow some flexibility when it comes to the development stage to test new versions of the Terraform CLI as well as new versions of the Terraform provider.

To achieve that we define global variables that can be accessed in the code generation process. We put these variables in the configs.tm.hcl file in the root directory:

// Configure default Terraform version and default providers
globals "terraform" {
  version     = ">= 1.6.0"
  version_dev = ">= 1.8.0"
}

globals "terraform" "providers" "btp" {
  version     = "~> 1.4.0"
  version_dev = "~> 1.4.0"
}

globals "terraform" "modules" "btp_subaccount_module" {
  source = "github.com/btp-automation-scenarios/btp-subaccount-module?ref=67cb61948e19497377fb4e23f01dd301319c6907"
}

Here we specify several version constraints as well as the source of the module on GitHub.
Next, we create a directory called templates where we will put in the code templates for the code generation. We add a file called generate_provider.tm.hcl which serves as template for the code generation resulting in a provider configuration. It contains the following content:

generate_hcl "_terramate_generated_provider.tf" {
  content {
    terraform {
      required_version = tm_ternary(tm_contains(terramate.stack.tags, "dev"), global.terraform.version_dev, global.terraform.version)

      required_providers {
        btp = {
          source  = "SAP/btp"
          version = tm_ternary(tm_contains(terramate.stack.tags, "dev"), global.terraform.providers.btp.version_dev, global.terraform.providers.btp.version)
        }
      }
    }
    provider "btp" {
      globalaccount = var.globalaccount
    }
  }
}

The syntax is good to understand:

• The generate_hcl block defines the contents of the configuration to be generated.
• The block label "_terramate_generated_provider.tf" defines the file name of the generated file.
• The content block contains the actual content of the file. Here we define the provider configuration. As you can see, we make use of some Terramate specific functions to assign the version of the Terraform CLI and the Terraform provider depending on the stage with the help of the global variables and the stack specific information. We check if the stack is tagged as “dev” using the tm_contains function together with the information available via the terramate.stack object. Depending on the availability we decide which version to use leveraging the tm_ternary function.

One last piece is missing that we need for the generation: Teramate needs to know which templates to use for the generation. We define this in the imports.tm.hcl file on root level. The file has the following content:

# Import helper files
import {
  source = "./templates/generate_provider.tm.hcl"
}

Okay, ready to go? Let us kick off our first code generation via:

terramate generate

We see the following output:

And indeed, every stack now contains a generated provider.tf file according to our naming:

Next stop: the variables. We want to make the call of the setup as easy as possible, so we will default the values where possible i.e., we will only leave the globalaccount variable open for the user to provide. We will also make use of the stack specific information to:

• Assign the architect role to the emergency admins in the development stage, otherwise the organizationally assigned admins
• Assign the entitlement free for HANA Cloud in the development stage, otherwise the entitlement hana.

To achieve this, we create a new template file for the variables in the templates directory called generate_variables.tm.hcl. We put the following code into the file:

generate_hcl "_terramate_generated_variables.tf" {
  content {
    variable "globalaccount" {
      type        = string
      description = "The globalaccount subdomain where the sub account shall be created."
    }

    variable "region" {
      type        = string
      description = "The region where the account shall be created in."
      default     = "us10"
    }

    variable "unit" {
      type        = string
      description = "Defines to which organisation the sub account shall belong to."
      default     = "Sales"
    }

    variable "unit_shortname" {
      type        = string
      description = "Short name for the organisation the sub account shall belong to."
      default     = "sls"
    }

    variable "architect" {
      type        = string
      description = "Defines the email address of the architect for the subaccount"
      default     = "genius.architect@test.com"
    }

    variable "costcenter" {
      type        = string
      description = "Defines the costcenter for the subaccount"
      default     = "1234509874"
    }

    variable "owner" {
      type        = string
      description = "Defines the owner of the subaccount"
      default     = "someowner@test.com"
    }

    variable "team" {
      type        = string
      description = "Defines the team of the sub account"
      default     = "awesome_dev_team@test.com"
    }

    variable "emergency_admins" {
      type        = list(string)
      description = "Defines the colleagues who are added to each subaccount as emergency administrators."
      default     = tm_ternary(tm_contains(terramate.stack.tags, "dev"), ["somearchitect@test.com"], ["jane.doe@test.com", "john.doe@test.com"])
    }

    variable "entitlements" {
      description = "List of entitlements for a BTP subaccount"
      type = list(object({
        group  = string
        type   = string
        name   = string
        plan   = string
        amount = number
      }))

      default = [
        {
          group  = "Audit + Application Log"
          type   = "service"
          name   = "auditlog-viewer"
          plan   = "free"
          amount = null
        },
        {
          group  = "Alert"
          type   = "service"
          name   = "alert-notification"
          plan   = "standard"
          amount = null
        },
        {
          group  = "SAP HANA Cloud"
          type   = "service"
          name   = "hana-cloud"
          plan   = tm_ternary(tm_contains(terramate.stack.tags, "dev"), "free", "hana")
          amount = null
        },
        {
          group  = "SAP HANA Cloud"
          type   = "service"
          name   = "hana"
          plan   = "hdi-shared"
          amount = null
        }
      ]
    }
  }
}

We add the template to the imports.tm.hcl file that now contains two blocks:

# Import helper files
import {
  source = "./templates/generate_provider.tm.hcl"
}

import {
  source = "./templates/generate_variables.tm.hcl"
}

Good to go for a second round of code generation:

terramate generate

The output shows that the variables are generated for every stack:

And we also find the generated files in the stacks:

But hold on a second: the output also shows that only the delta is generated. Files that do not need to be touched are not changed even if we re-run the terramate generate command. I like that!

Last but not least we must generate the main configuration file. The only requirement we have is that the usage attribute of the subaccount is set to USED_FOR_PRODUCTION in the testing and production stage, while the development environment should be set to NOT_USED_FOR_PRODUCTION. In addition, we want to use the label of the stack to feed into the stage parameter of the module that creates the subaccount.

We know the drill by now and create a new template called generate_main.tf.hcl in the templates directory:

generate_hcl "_terramate_generated_main.tf" {

  lets {
    stage = tm_upper(tm_element(terramate.stack.tags, 0))
  }

  content {
    # ------------------------------------------------------------------------------------------------------
    # Creation of directory
    # ------------------------------------------------------------------------------------------------------
    resource "btp_directory" "parent" {
      name        = "${var.unit}-${terramate.stack.name}"
      description = "This is the parent directory for ${var.unit} - ${terramate.stack.name}."
      labels      = { "architect" : ["${var.architect}"], "costcenter" : ["${var.costcenter}"], "owner" : ["${var.owner}"], "team" : ["${var.team}"] }
    }

    # ------------------------------------------------------------------------------------------------------
    # Call module for creating subaccoun
    # ------------------------------------------------------------------------------------------------------
    module "project_setup" {

      source = "${global.terraform.modules.btp_subaccount_module.source}"

      stage  = "${let.stage}"
      region = var.region

      unit                = var.unit
      unit_shortname      = var.unit_shortname
      architect           = var.architect
      costcenter          = var.costcenter
      owner               = var.owner
      team                = var.team
      emergency_admins    = var.emergency_admins
      parent_directory_id = btp_directory.parent.id
      usage               = tm_ternary(tm_contains(terramate.stack.tags, "dev"), "NOT_USED_FOR_PRODUCTION", "USED_FOR_PRODUCTION")
    }
  }
}

The source of the module is defined via the global variable we defined before, and the usage is set depending on the stack label using the known Terramate functions. For the sake of demoing the capabilities we introduced local variables via a lets block to define the stage variable. We make use of two further Terramate functions to get the first element of the stack tags (tm_element) and to convert it to upper case (tm_upper).

We add the import to the imports.tm.hcl file which now finally looks like this:

# Import helper files
import {
  source = "./templates/generate_provider.tm.hcl"
}

import {
  source = "./templates/generate_variables.tm.hcl"
}

import {
  source = "./templates/generate_main.tm.hcl"
}

After the execution of the generation command, we see the main.tf files with the expected content in the stacks:

Shortly summarizing what we did until now:

• We created Terramate stacks with some metadata representing the stages of the setup
• We used the Terramate code generation feature to generate the provider configuration, the variables, and the main configuration for the Terraform setup including stack specific adjustments. As a consequence, we did not need to repeat ourselves or do some copy&paste exercises. In addition, we can now easily adapt the configuration in the future and regenerate the files.

Now time to get some infrastructure set up!

Using Terramate to run Terraform commands

As we have a complete Terraform configuration in the single directories, we could hop into each of those and execute the Terraform commands there. But can't we do better with Terramate? Yes, we can.

Terramate has the terramate run command that allows you to execute any command in the stacks. Let us try out some things.

First, we want to initialize all stacks. we do so by executing the following command:

terramate run terraform init

In the console we se that the Terraform commands gets executed per stack:

We recognize that the usual suspects appear in the file system:

Initialization successful. Next, we might want to plan the setup. We do so by executing the following command:

terramate run terraform plan -var globalaccount=<YOU GLOBAL ACCOUNT>

In the output we will see that the plan is executed for all three stacks in sequence. The output is a bit lengthy, so I will not provide a screenshot here.

The output reflects that the plan is executed for all three stacks. We will also see a different number of resources that are to be created as we have only one emergency administrator for the development environment, while we defined two for the test and production environment. Things work as expected.

We can also check the execution sequence via:

terramate list --run-order

The output looks like this and as we have no dependencies all stacks are on the same level ordered by the sequence in the file system:

The execution of the commands can also be done in parallel:

Terramate will do the parallelization if the stacks are independent from each other. If there are dependencies between the stacks, Terramate will execute the stacks in the correct order:

terramate run --parallel=3 terraform plan -var globalaccount=<YOU GLOBAL ACCOUNT>

This is then reflected in the output of the command:

We can also restrict the execution to a specific stack e.g., via the path:

terramate run --chdir=stacks/dev terraform plan -var globalaccount=<YOU GLOBAL ACCOUNT>

or even more convenient using tags:

terramate run --tags dev terraform plan -var globalaccount=<YOU GLOBAL ACCOUNT>

The output reflects the filtering:

You can of course also execute all the other commands and apply your setup. But I think the value of executing the commands via Terramate is clear now.

Note - Maybe you stumble across Terramate's built-in safeguarding to prevent executions on uncommitted changes when doing things on your machine. You can switch off all safeguards by adding the -X flag to the terramate run commands. It certainly makes sense though to check the documentation what are the effects before doing so.

Where to find the code

You find the code including a copy of the original monolithic setup on GitHub: https://github.com/btp-automation-scenarios/btp-terramate.

Conclusion and Outlook

When dealing with more complex Terraform setups that usually arise quickly, Terramate is a very valuable tool that you should take a look at. It helps you to manage the complexity in a structured way and comes with useful features to make your life in the Terraform world easier.

The rewriting of the dev-test-prod setup showed where Terramate could shine and helps us tackling the complexity that comes with these setups. Two main ingredients for achieving this is the concept of stacks and the code generation feature.

However, we left out one important aspect of remote backends. Here again the stack-based approach comes in handy as you can see in this sample provided by the Terramate team that showcases how to generate stack specific backend configurations: https://github.com/terramate-io/terramate-examples/blob/main/01-keep-terraform-dry/imports/generate_backend.tm.hcl.

There is so much more to explore in Terramate and in which scenarios it can support your setup. Looking at the documentation I just scratched the surface and need to spend a bit more time to understand all the details. But I am convinced it is worth it.

With that happy Terraforming with Terramate!

Note - I focused on Terraform in this blog post, but you can of course also use OpenTofu. There is no barrier built into Terramate that would prevent you from doing so.