Automating Infrastructure For Geo-Distributed Applications With Terraform

An increasing number of products are being developed with performance and scalability in mind. Modern developers need to rely on improved tooling to efficiently and reliably build, test and deploy their applications.

To continue my development of The Largest River (my first foray into global application development), I’ve chosen to use Google Compute Engine.

Major cloud providers like Google provide both web and command-line interfaces to create and manage virtual machines, as well as offering countless other products. However, when deploying apps at scale, this can become quite an arduous task! What if the deployment environment needs to be replicated, say, to create a test or staging environment? Surely, we’d want this environment to directly mirror that of production. Any configuration step which isn’t automated leaves us open to unexpected bugs.




Infrastructure as Code

Rather than relying on time-consuming manual tasks or a series of shell scripts to create and modify infrastructure, I’ve decided to use Terraform to do this from a single configuration file.

Although I’ve only scratched the surface with its functionality, the benefits are immediately apparent. Here are some of the ways I’ve begun using Terraform with The Largest River.

Using the Google Cloud Platform Provider, I’m easily able to spin up and tear down my project infrastructure on GCP.

# main.tf

terraform {
 required_providers {
   google = {
     source  = "hashicorp/google"
     version = "4.24.0"
   }
  }
}

provider "google" {
 credentials = file([path_to_json_keyfile_downloaded_from_gcp])
 project = [gcp_project_name]
 region  = "us-central1"
 zone    = "us-central1-c"
}

After downloading my JSON keyfile from GCP, I’ve set up the Google provider, which can be used to securely connect to my project in the cloud. Although my application resources will be spread across multiple regions and zones, I’ve chosen the us-central1 region and us-central1-c zone as my project defaults.

Keeping Things Private

With the Google provider configured, I’m able to set up network, firewall, and instance resources.

In the case of The Largest River, setting up a VPC network is important for multiple reasons:

• I’ll be able to communicate between servers in a multi-region deployment, without having to traverse the public internet. This comes with latency and security benefits.
• As I’ll use this VPC for my multi-region YugabyteDB Managed deployments, my application servers and database nodes will live within the same global network.

Here’s how the network is initialized using the google_compute_network resource.

resource "google_compute_network" "vpc_network" {
 name = "tlr-network"
}

To make use of this network, I’ve added some firewall rules to enable SSH and HTTP traffic to my instances, using the google_compute_firewall resource.

resource "google_compute_firewall" "ssh-rule" {
 name    = "ssh-rule"
 network = google_compute_network.vpc_network.name
 allow {
   protocol = "tcp"
   ports    = ["22"]
 }
 target_tags   = ["allow-ssh"]
 source_ranges = ["0.0.0.0/0"]
}

resource "google_compute_firewall" "http-rule" {
 name    = "http-rule"
 network = google_compute_network.vpc_network.name
 allow {
   protocol = "tcp"
   ports    = ["80", "8080"]
 }
 target_tags   = ["allow-http"]
 source_ranges = ["0.0.0.0/0"]
}

These rules include some new parameters, namely, target_tags and source_ranges:

• Target tags apply specific firewall rules to an instance in the network. We’ll see how this works soon, as I begin to configure some virtual machines.
• Source ranges determine the IP addresses this rule applies to.

In this example, we’ve opened up connections to the whole internet, which, you know, isn’t great! In reality, we’d set our source ranges to a sensible range within our subnet for instances that don’t need to be exposed to the public internet.

Choosing the Right Compute Instances

What good is a network without any instances? We currently have roads leading to nowhere. Let’s change that, using the google_compute_instance resource.

variable "instances" {
 type = map(object({
   name = string
   zone = string
 }))
 default = {
   "usa" = {
     name = "instance-usa"
     zone = "us-central1-c"
   },
   "europe" = {
     name = "instance-europe"
     zone = "europe-west3-b"
   },
   "asia" = {
     name = "instance-asia"
     zone = "asia-east1-a"
   }
 }
}

resource "google_compute_instance" "vm_instance" {
 for_each = var.instances

 name                      = each.value.name
 machine_type              = "f1-micro"
 zone                      = each.value.zone
 allow_stopping_for_update = true

 tags = ["allow-ssh", "allow-http"]

 boot_disk {
   initialize_params {
     image = "cos-cloud/cos-stable"
   }
 }

 metadata = {
   gce-container-declaration = "spec:\n  containers:\n    - name: tlr-api\n      image: [container_image_url_in_gcr]\n      stdin: false\n      tty: false\n  restartPolicy: Always\n"
 }

 service_account {
   email = [email_for_project_service_account]
   scopes = [
     "https://www.googleapis.com/auth/cloud-platform"
   ]
 }

 network_interface {
   network = google_compute_network.vpc_network.name
   access_config {
   }
 }
}

Now that we have some containerized houses at the end of the road, we’re starting to get somewhere...

In all seriousness, GCP offers a wide range of instance types. For The Largest River, I’ve chosen Google Container Registry, and thus, a Container-Optimized OS.

To deploy to multiple zones without code duplication, I’ve set the instances variable, which is looped to create instances in the USA, Europe and Asia. I’ve added two tags to these instances, allow-ssh and allow-http. These tags match the target tags specified in our firewall rule blocks, which means these rules will be applied to the deployed instances.

Wrapping Up

With the core elements defined in our configuration, we can make use of the Terraform CLI to provision the infrastructure. You don’t even need to click in the GCP console as Terraform elegantly tracks changes to this configuration, making planning and updating a breeze.

Much like core app development, the infrastructure as code community has fully adopted code reuse and expressive language support. The Terraform Language includes many such features and I look forward to diving deeper, as I continue to build this geo-distributed application. You can revisit the start of my journey here, and stay tuned for more updates!