Infrastructure as Code using Terraform

Introduction to Infrastructure-as-Code

Infrastructure-as-code (laC) is the practice of creating and managing infrastructure through machine-readable code. The code is normally a high-level construct that is easy for humans to write and understand, but can also be parsed and executed to actually create or manage the infrastructure directly, usually by means of an underlying API provided by the infrastructure platform.

Definition of Infrastructure

When we refer to infrastructure in this context we generally mean virtual infrastructure, such as virtual machines, load balancers, storage services, etc. For the purposes of this document, however, we extend this definition to include any cloud service that can be managed through an API, such as databases, analytics services, and Al cognitive services.

Why Infrastructure-as-Code

lac brings many benefits including:

• Automated deployment of infrastructure/services without requiring tickets to be manually processed by infrastructure teams.
• Version-controlled infrastructure in the same way application code is version-controlled, bringing all the associated benefits including audit ability and ability to roll back changes.
• Automated testing of new infrastructure configuration.
• Ability to quickly spin up ephemeral environments for testing, experiments, or to reproduce an issue.
• Prevents configuration drift caused by manual modifications to existing environments over time.
• Helps enforce parity between development and production environments as they are built from the same code.

What is Terraform

Terraform is a cross-platform infrastructure-as-code tool from HashiCorp. The core product is an open-source command-line interface (CL) tool, however, the company also provides enterprise and cloud-based offerings that add additional functionality.

Terraform consists of the following three key facets:

• Language: A declarative syntax for describing resources (infrastructure) called HCL (Hashicorp Configuration Language).

• Providers: These are Terraform plugins specific to each target platform. For example, the Azure provider tells Terraform what resources may exist on Azure, what configuration they may have, and how to make the API calls against Azure to actually instantiate and manage those resources

• Executable: The Terraform CLI which is used to execute Terraform

How does it work

1. Write Terraform code - use the AzureRM provider documentation or internal examples.
2. Download the Terraform CLI and Azure provider.
3. Execute the Terraform CL which does the following.
4. Parses the HCL files to determine the desired state.
5. Look at the current state of what's deployed(Terraform keeps track of resources under its control in a State file, and queries the platform API to check on the actual state of the resources).
6. Builds a dependency graph between resources to determine what needs to be created/updated/destroyed in what order.
7. Executes the necessary API calls against the platform to affect the desired state.

This workflow can be built into a controlled continuous integration and deployment (CVCD) pipeline.

Why Terraform

Terraform is not the only option for realizing the benefits of infrastructure-as-code Some other solutions are possible:

Custom scripts

One could write custom scripts that use Azure's CLI tool or APIs to create and manage infrastructure While this could achieve some of the benefits of lac, scripts containing a set of imperative commands would be very difficult to manage incremental changes to an environment over time mean it would be difficult to quickly spin up a new identical environment and configuration drift across environments is likely to occur. It would be difficult to define dependencies between resources, and scripts would be difficult to test.

ARM Templates

Azure's management and deployment layer the Azure Resource Manager (ARM) has its own native la format called ARM templates. There are various advantages and disadvantages to using Terraform or ARM templates, some of which are subjective.

Here are some of the reasons why Terraform may be preferred:

• Syntax: Terraforms HCL syntax is a lot more readable and concise than ARMS JSON format and includes convenient variable interpolation features.

• Modules and composability: Terraforms approach for modularizing infrastructure code is easy to use and more natural than the ARM approach of using nested or linked templates.

• Dependency resolution: Terraform automatically infers dependencies between resources so that changes are applied in the correct order in ARM templates, dependencies must be explicitly stated. (Terraform also allows dependencies to be explicitly defined where necessary).

• Workflow using "plan": Terraforms standard workflow relies on a planning phase which outputs a human and machine-readable delta showing a preview of exactly what infrastructure changes will be made. The plan can be reviewed before the changes are applied ARUM has recently introduced a similar operation called "what'' however this is not an inherent part of the ARM template deployment workflow, and the state could have changed between the execution of the "what-if" and the real execution. Conversely, Terraform refreshes the state before applying the previously generated plan, making unexpected, potentially destructive, results less likely.

• Cross-platform: Terraform has providers for all public cloud platforms, as well as providers for subsidiary platforms deployed on clouds such as Kubernetes and Azure Databricks There are also providers for many other virtualized platforms (for example, FS BigIP, VMware vSphere, etc.) so could be used for hybrid deployments of infrastructure/services that span multiple platforms, including on-premises services Developers can also build their own providers.

There are a few potential drawbacks of using Terraform over ARM templates

• Feature support lag: Although the Terraform Azure provider is well maintained and supported by Microsoft, providing support for new Azure services or features can lag behind the release of these features in Azure. In those cases where a newly released feature is required and not yet supported by the provider, an approach to work around this is to use the Terraform provider's ARM template resource, effectively allowing Terraform to wrap an ARM template.

• Azure Portal support: For Azure beginners, the ability to create a resource manually in the Portal and then download and examine the ARM template equivalent is a useful feature. It is not possible to generate an equivalent Terraform configuration from the Portal However, Terraform code is arguably much easier to write.

• Terraform State: Terraform needs to store the known state of the infrastructure it is managing independently of the deployed resources, whereas ARM templates could be considered stateless The Terraform state must be protected appropriately and stored in a highly available location Managed properly, however, using a recommended backend such as Azure Storage, Terraforms approach to state management is part of what enables Terraforms advantageous workflow and cross-platform capabilities.

Open source vs Terraform Enterprise

Terraform Enterprise (TFE) and the equivalent SaaS-based offering Terraform Cloud offers some enhancements to the core open-source product (usually just called Terraform CLI) which can be read about here It is important to point out that the must-have features needed for working with Terraform in an enterprise environment such as ours do not require TEE. Specifically, with reference to those features mentioned in the link above:

• Remote Terraform Execution: - This is achieved with Terraform CU by running it within a build and deployment pipeline(using ADO, GitLab CI, etc.), just as we would do for any code deployment.

• Version Control Integration: Similarly this is already achieved in our Cl environment where code check-ins and merge requests automatically trigger the pipeline which can include the execution of Terraform CLI.

• Remote State Management: The basic requirement for remote state storage is to have a secure, highly-available centralized state store that supports locking Terraform CLI supports Azure Blob storage which is the natural choice in our environment.

• Team-Based Permissions System: This is only necessary by virtue of the fact that TFF provides its own interface and execution platform to run Terraform. Terraform CLI alone doesn't require separate RBAC controls as that is managed at other layers of the stack, namely the permission model already in place that governs access to our Azure subscriptions, and the permissions model that governs access to our existing deployment tools from which Terraform CLI is run.

Resources

• Official Terraform documentation
• Terraform Provider documentation
• Terraform Modules