Deploying an HTTP app using Docker + GKE + Cloudflare

Table of contents

• Recap
• Living document intention
• Serving an HTTP app using Docker, GKE and Cloudflare
  • 1. Write an HTTP app
  • 2. Build the app into an image
  • 3. Register and onboard with Google Cloud
  • 4. Setup a Google Artifacts Registry project
  • 5. Tag the Docker image based on Google's Artifact Registry
  • 6. Install the gcloud command on your machine
  • 7. Configure and authenticate gcloud
  • 8. Push the Docker image to Artifact Registry
  • 9. Set up a Shared VPC project
  • 10. Set up a shared GKE cluster project
  • 11. Set up Cloud SQL instances
  • 12. Create a Kubernetes cluster
  • 13. Prepare secrets for staging and production
  • 14. Create your first GKE workload
  • 15. Apply secrets to GKE workloads
  • 16. Prepare Cloudflare for Total SSL
  • 17. Expose your application using Cloudflare Tunnel
  • 18. See your application live 🎉
• Where does our trip take us next? 🚀
• Changelog
• Stay connected 💬

Recap

Software Engineering Entrepreneurship » Issue 5 » Up and running with Google Cloud

Morgan Wowk ・ Feb 18 '23

#googlecloud #architecture #startup #career

In issue 4-5 of my series Software Engineering Entrepreneurship we covered the technology stack serving the foundation of our future apps as a Software Engineer-by-day turned Entrepreneur.

Living document intention

At the start of this series I disclosed that I would not give a step-by-step tutorial at any point. This document, however, will serve as the most important log and perhaps the one exception for myself and others to avoid hurdles building this stack in the future.

Find below an ordered, roughly detailed guide to setting up this technology stack. I will update this document as new discoveries are made or as new steps are introduced. See the bottom of this document for a changelog documenting such edits. Use this document only as an assistant to setting up infrastructure based on your distinct use case and requirements.

Serving an HTTP app using Docker, GKE and Cloudflare

Subject to change (see changelog)

1. Write an HTTP app

In your language of choice write an HTTP app that when ran, serves on a specified port.

In an ideal world, use Docker for both your development and your production build. You may have a different Dockerfile for development and production. Your production Dockerfile should run an executable (using the CMD keyword) that serves a long running process on a port you specify. For example, in Go you might have a Dockerfile.dev that runs an air command and a separate production Dockerfile that runs a go build and then CMD ["yourapp"].

2. Build the app into an image

docker build -t example-api-build-<build-version> -f ./deployment/docker/api/Dockerfile .

Replacing the name appropriately and <build-version> with a versioning mechanism you have decided on (or test if you wish to defer that task for later). See Container Image Versioning by Rahul Sharma.

3. Register and onboard with Google Cloud

This step is likely the most time consuming.

Budget decision

First, make sure you're in a position with enough available budget to commit to using Google Cloud. Deploying Google Cloud VMs and services could easily exceed $100 / month and this may come as a surprise to some. You should be careful with the resources you create as you can unintentionally rack up costs. Google will strive for minimum costs in many cases and provide estimates as much as possible. I would still recommend getting into the habit of looking up "[Insert GCP Product] Pricing" and at least doing a rough estimate of your own.

Check out Google's free tier as well.

Registering with Google Cloud

If you're comfortable moving forward you can roughly follow the steps below:

1. If you haven't already, register a domain and setup an email with the domain you're going to use to administrate Google Cloud.
2. Register with Google Cloud.
3. Warning: Be cautious following Google's on-screen onboarding flow. Its purpose is to scaffold Google products for you based on your needs and answers to prompts. While I went through it and learned from it I ended up re-doing all the work it did on my own and actually removed most of the scaffolding it had done on my behalf. The reason was that over time I learned more about what "projects" are and how I wanted resources to be organized within GCP. I also discovered that some of the resources Google scaffolded for me actually immediately started incurring costs which consumed all of my initial GCP credits. After speaking with Google's support I was able to confirm there is no cost or credit forgiveness policy at Google either.
4. As part of onboarding, set up a billing account you will attach to all the projects you later create.

Example project structure

For your inspiration, below is the project structure I have implemented:

Some notes on this structure:

4. Setup a Google Artifacts Registry project

1. Create a project apps-artifacts or apps-docker-artifacts.
2. Open the project.
3. In the left navigation select Artifact Registry.
4. Enable the Artifact Registry API.
5. Create a registry to hold Docker images following the prompts in the UI to your liking.

5. Tag the Docker image based on Google's Artifact Registry

The format to use for tagging Artifact Registry images is LOCATION-docker.pkg.dev/PROJECT-ID/REPOSITORY/IMAGE.

Command synopsis:

docker tag <image-to-tag-from-earlier-step> <tag>

Example command:

docker tag example-api-build-test northamerica-northeast2-docker.pkg.dev/apps-artifacts/docker-images/example-api-build-test:latest

6. Install the gcloud command on your machine

On your local machine you're going to be using the gcloud CLI every day soon enough. Now's a good time to install it. Follow Google's instructions to install gcloud CLI.

7. Configure and authenticate gcloud

The first exciting thing to do with your new gcloud command is to setup your configuration.

See your existing configurations:

gcloud config configurations list

Create a new configuration:

gcloud config configurations create config-example-company

Set the google account on your currently active configuration:

gcloud config set account yourname@exampleapps.net

Authenticate gcloud with your Google account:

gcloud auth login

Other useful commands:

• gcloud projects list
• gcloud config set project <project-name>

8. Push the Docker image to Artifact Registry

Configure Docker for pushing to Artifact Registry

gcloud auth configure-docker LOCATION-docker.pkg.dev

# Example: gcloud auth configure-docker northamerica-northeast2-docker.pkg.dev

Push the Docker image to Artifact Registry

docker push <tag-from-earlier-step>

# Example: docker push northamerica-northeast2-docker.pkg.dev/apps-artifacts/docker-images/example-api-build-test:latest

9. Set up a Shared VPC project

If you're following the same structure of projects as I have then you're going to want to setup a Shared VPC project; A project that will serve as your networking "host" for resources spread across multiple projects. This will allow your resources across projects to communicate over private IP, reducing latency and improving security compared to public IP without the Shared VPC.

1. Create a project apps-shared-vpc in the directory Apps/Common.
2. Within the project navigate to VPC Network -> Shared VPC.
3. Create a Shared VPC. Reference the section Create a network and two subnets from Google's Setting up clusters with Shared VPC article.
4. Navigate to Kubernetes Engine -> Clusters.
5. Enable the Kubernetes Engine API. It will remain unused but is required for the Shared VPC to later work with GKE.

10. Set up a shared GKE cluster project

From issue 5 of my series Software Engineering Entrepreneurship you would have heard my recommendation to use a shared GKE cluster while you are in a pre-seed / development phase of building your own apps. You can repeat the steps below for one or many GKE projects you create depending on your use case.

Create the project

Create a project apps-shared-cluster in the directory Apps/Common.

Grant permissions to Artifact Registry

Because the Artifact Registry is in a separate project you need to explicitly allow the apps-shared-cluster project to read images from the apps-artifacts project. Without doing so, your Kubernetes deployments will fail with an image pull error.

1. Within the apps-shared-cluster project navigate to IAM & Admin.
2. Copy the Principal email for the name Google APIs Service Agent, Compute Engine default service account - as well as the email that resembles <id>@cloudbuild.gserviceaccount.com.
3. For each Principal email, navigate to the apps-artifacts project.
4. Navigate to IAM & Admin for the artifacts project.
5. Select "Grant access" and assign the role Artifact Registry Reader and Compute Image User to each Principal copied from the previous step.

Grant permissions to networking within the Shared VPC

In later steps you will deploy a Kubernetes cluster and relevant workloads. As part of this process GKE will attempt to automatically create firewall rules for you based on what services are available to the outside world. In order for GKE to do this it needs access in the apps-shared-cluster project to control networking in the apps-shared-vpc project that is the networking host. Follow the steps below to configure necessary permissions:

1. Open the project apps-shared-cluster.
2. Navigate to IAM & Admin.
3. For both the Principals named Google APIs Service Agent and Compute Engine default service account select "Grant access" and assign the principals (by email) the following roles:
   • Compute Network Admin
   • Service Networking Service Agent

11. Set up Cloud SQL instances

If your app is using Cloud SQL similar to my use case then you may follow the steps below to get setup.

1. Create a project apps-shared-sql if you plan on using a single shared VM for both your staging and production databases during initial development. Otherwise, create or select the project of your choosing to host your Cloud SQL instance (e.g. staging-resources).
2. Within the chosen project navigate to SQL.
3. Continue to create a Cloud SQL instance.
   1. Ensure the instance is created on the Shared VPC you put together earlier.
   2. If prompted and you do are not familiar enough with writing your own IP ranges, choose to let Google automatically allocate an IP range (still within the Shared VPC network).
4. With the instance created you can use Google's UI to create databases if you like. I recommend creating databases such as example_app_staging, example_app_production. Including the environment in the database name is helpful when you are using a shared instance or migrate to a shared instance in the future - you will avoid conflicts with database names.
5. You can also use Google's UI to create users. I recommend creating users such as <lastname><firstinitial>_RW, code_example_app_staging and code_example_app_production. Following best practices of ensuring app environment have separate users from those that your team members connect to on their own clients.
6. Securely store the username, passwords and connection details for each user created.

12. Create a Kubernetes cluster

A kubernetes cluster represents an instruction to Google to reserve a node pool (physical VMs) that will later host GKE workflows. When creating GKE clusters you will have the option to self manage nodes or use Google's recommended GKE Autopilot cluster - Autopilot will automatically allocate the minimum resources (CPU, RAM and storage) to meet the needs of your workloads.

1. Open your previously created GKE project (e.g. apps-shared-cluster).
2. Navigate to Kubernetes Engine -> Clusters.
3. Continue to create an Autopilot cluster (recommended) or manually manage a cluster. See GKE pricing.
   • If you are going to be using a shared cluster for staging and production to save costs then use a name such as apps-shared.
   • Ensure the GKE cluster is created within the Shared VPC network created earlier.
   • Here is an example configuration for my own cluster based on the Shared VPC we created earlier.
   • If you're unsure, choose to make the cluster public instead of private for now.

13. Prepare secrets for staging and production

One task you will need to endeavour is having a secrets strategy for your local, staging and production environments. In this section of the document I will share with you the secrets strategy I am personally using. This is before setting up any sort of continuous delivery approach.

1. Setup a Gitlab/Github organization to version control applications and secrets.
2. Setup an isolated group and projects to store secrets for applications. For example:
3. Have a folder for each environment within your secrets project. E.g staging and production.

4. Store files within those folders that will be pushed to Kubernetes as secrets later on. Here is an example secrets yml file:
   

   DB:
     Connections:
       Primary:
         Host: 11.22.70.4
         Port: 3306
         Username: code_example_app_production
         Password: A5odaPLjBo[o#f}A$TBQ
         Database: example_app_production
   

5. Commit these files to your repository. We will come back to this later on when we are ready to push secrets to your Kubernetes namespaces.

14. Create your first GKE workload

You're now getting really close to having a running, accessible application. Now's the time to create your GKE workload. Here you will deploy your previously created artifact (HTTP application). Disclaimer though, if your application relies on secrets to establish a database connection then your workload will fail to spin up pods until those secrets are applied. Regardless though, Google's UI will only let you deploy the workload first and then apply secrets after. Follow the steps below:

Create your workload

1. Open your GKE project in Google's console.
2. Navigate to Kubernetes Engine -> Workloads.
3. Select "Deploy".
   1. Select "Existing container image".
   2. Change the artifact registry source project to the project you created earlier to host your Artifact Registry (e.g. apps-artifacts).
   3. Select the image tag/sha you wish you deploy (e.g. example-api-build-test:latest.
4. Continue.
5. Configure your workload
   1. For the deployment name, think about what you are serving. For example, if it's an HTTP API use api or backend. Note: I recommend having separate namespaces for each environment. E.g. The namespace example-app-staging. If you do this, you don't need to include the environment staging or production in your deployment name. You can simply use api instead of api-staging. There are other scoping benefits later on such as the name of Kubernetes secrets.
   2. For the labels I suggest the format app: example-app-staging to control workload scheduling based on app and environment.
   3. For the GKE cluster select the one you created in the previous steps.
6. Select "Continue".

7. Select "Expose deployment as a new service" to allow Google to expose a public endpoint (IP) for your created service / HTTP app.

   • "Port 1" should be the port that will be publicly accessible.
   • "Target port" should be the port your HTTP app is told to run on within the codebase and/or Dockerfile.
   • "Load balancer" is the appropriate service type if you wish to expose the application to the internet directly from the Kubernetes workload.

   Important: Exposing your services at this level circumvents future layers of your infrastructure such as rate limiting, caching and API gateways. For the remainder of this article we will choose "Cluster IP" instead and expose the service through private IP and Cloudflare instead.

8. Select "Deploy"

Heads up

Your Workload will attempt to start running based on the Docker image pushed to Artifact Registry. However, if that application depends on a database connection or other secret information then it is likely going to fail to start and that is completely normal for now.

15. Apply secrets to GKE workloads

This step covers how you can apply secrets to your GKE namespaces in order for your dependent workloads to successfully run.

Connect to your cluster

1. Navigate to Kubernetes Engine -> Clusters.
2. Next to the relevant cluster select "Connect".
3. Choose to connect through your own command line. Tip: I recommend having the kubectx command on your machine to always have visibility on which K8 context you are in.
4. Once you are connected to the correct context you can now run through the following steps to apply secrets.

Navigate to your secret directory

Based on the environment you are applying secrets, navigate in terminal to the directory of your CI resources / secrets repository holding the relevant file(s).

Check your namespaces

kubectl get namespace
# Work within the relevant namespace for all the following commands using the '-n <namespace>' flag

Create the secret

Here is an example of applying a yml file as a secret to a staging application.

kubectl -n <namespace> create secret generic <secret-name>-<secret-environment> --from-file=./<secret-file>

# Example: kubectl -n example-app-staging create secret generic app-config-example-app-staging --from-file=./config.yml

Take notice that even though it's applied within the specific staging namespace we're still including the environment staging in the name of the secret. This is a future-proof incase our team decided to move multiple environments into a single namespace at any point (not recommended.

Update your deployment to mount the secret

Even though the secret is created, it's only holding a place in Kubernetes data store without actually being used in any particular workload / deployment. We now need to edit your failing workload to (1) register a volume derived from a secret and then (2) mount a volume within the created pods.

1. Open your GKE workload in Google Cloud.
2. Select "Edit".

3. Make the following additions to the deployment yml. Use this as a structural and naming reference.
   

   spec:
     template:
       spec:
         containers:
         - image: <do-not-edit>
           volumeMounts:
             - mountPath: "/secrets"
               name: app-config-example-app-staging
               readOnly: true
         volumes:
           - name: app-config-example-app-staging
             secret:
               secretName: app-config-example-app-staging
   

   Note that correct indentation / placement is important here.

4. Select "Save". Heads up: Because Kubernetes and your Autopilot cluster is always managing your deployment and making changes, you may be asked to "Forcefully apply" the new deployment. This is okay to select.

With the secret now mounted into the workload at the location /secrets/config.yml the workload will now refresh, generate new pods (allow some time) and should succeed. If you continue to encounter errors you can always look at the "Logs" section of your Google Cloud workload or the "Logs Explorer" product itself in your navigation.

16. Prepare Cloudflare for Total SSL

As part of our technology stack we are going to use Cloudflare as an edge DNS, SSL, Cacheing and Tunnel provider. Below are my recommendations for setting up Cloudflare:

1. Add your domain to Cloudflare.
2. Open your Cloudflare Dashboard.
3. Navigate to SSL/TLS -> Edge Certificates.
4. Enable "Total TLS" and opt-in to the paid add-on "Advanced Certificate Manager" ($10 / month at time of writing).

By enabling Total TLS, Cloudflare will automatically issue SSL certificates for new DNS records whether they're created manually or through a Tunnel. This serves as a huge convenience for live applications and local development.

17. Expose your application using Cloudflare Tunnel

Earlier we made the decision to expose our GKE Workload using "Cluster IP". This means the application itself is only served internally so we can add additional layers before it is exposed to the internet. In this case we are interested in placing Cloudflare between our application and the internet.

We will deploy a sibling GKE Workload that leverages Cloudflare Tunnel to expose our Cluster IP to the internet via custom domain.

1. Install the cloudflared CLI on your local machine.
2. Run cloudflared tunnel login to authenticate your CLI with Cloudflare. Select the appropriate domain you will be creating tunnels for.
3. Run cloudflared tunnel create example-app-staging replacing example-app-staging with your desired tunnel name.
4. Run cloudflared tunnel route dns example-app-staging tunnel.example.com again replacing the tunnel name appropriately.
5. Copy the created credential json file into your secrets storage (i.e. Gitlab repository) for tracking. Use a name such as cloudflare-tunnel-credential-example-app-staging.json.
6. Run kubectl -n example-app-staging create secret generic cloudflare-tunnel-credential-example-app-staging --from-file=credentials.json=/<path-from-earlier-output>/<tunnel-id>.json replacing the namespace, secret name and tunnel credential path appropriately.

7. Now create your Cloudflare Tunnel config using a name such as cloudflare-tunnel-config-example-app-staging.yml. Reference the following template or see the bottom of Cloudflare's K8 example yaml:
   

   tunnel: example-app-staging
   credentials-file: /etc/cloudflared/creds/credentials.json
   # Serves the metrics server under /metrics and the readiness server under /ready
   metrics: 0.0.0.0:2000
   # Autoupdates applied in a k8s pod will be lost when the pod is removed or restarted, so
   # autoupdate doesn't make sense in Kubernetes. However, outside of Kubernetes, we strongly
   # recommend using autoupdate.
   no-autoupdate: true
   # The `ingress` block tells cloudflared which local service to route incoming
   # requests to. For more about ingress rules, see
   # https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/configuration/ingress
   #
   # Remember, these rules route traffic from cloudflared to a local service. To route traffic
   # from the internet to cloudflared, run `cloudflared tunnel route dns <tunnel> <hostname>`.
   # E.g. `cloudflared tunnel route dns example-tunnel tunnel.example.com`.
   ingress:
     # The first rule proxies traffic to the httpbin sample Service defined in app.yaml
     - hostname: example-app.staging.exampleapps.net
       service: http://api-service:80
     # This rule matches any traffic which didn't match a previous rule, and responds with HTTP 404.
     - service: http_status:404
   

8. Create a secret for the config within the same namespace as the service you're exposing.
   

   kubectl -n example-app-staging create secret generic cloudflare-tunnel-config-example-app-staging --from-file=config.yaml=./cloudflare-tunnel-config-example-app-staging.yml
   

9. Construct a Kubernetes deployment (i.e. api-cloudflare-tunnel.yml) based on Cloudflare's example yml here.

   • Use a deployment name of your choosing. One suggestion is cloudflare-tunnel-<deployment-to-expose> (e.g. cloudflare-tunnel-api).

10. Make the following edits to the deployment yml. Use this as a structural and naming reference.
    

    spec:
      template:
        spec:
          containers:
          - image: <do-not-edit>
            volumeMounts:
              - mountPath: "/etc/cloudflared/creds"
                name: cloudflare-tunnel-credential-example-app-staging
                readOnly: true
              - mountPath: "/etc/cloudflared/config"
                name: cloudflare-tunnel-config-example-app-staging
                readOnly: true
          volumes:
            - name: cloudflare-tunnel-credential-example-app-staging
              secret:
                secretName: cloudflare-tunnel-credential-example-app-staging
            - name: cloudflare-tunnel-config-example-app-staging
              secret:
                secretName: cloudflare-tunnel-config-example-app-staging
    

    Note that correct indentation / placement is important here.

11. Apply the new deployment under the same Kubernetes context and namespace as the service you're trying to expose. With our staging example app as an example, that would be:
    

    kubectl -n example-app-staging apply -f api-cloudflare-tunnel.yml
    

Your GKE Workload pods will recognize the new changes and should now succeed in starting the tunnel.

Important: Be sure to set the maximum number of replicas for your Cloudflare GKE Workload to 1. This is to support only having 1 tunnel running as duplicates will fail.

18. See your application live 🎉

This is where all of your hard work pays off. While there are still many improvements to make to the current infrastructure you should see that your application is now available through your custom domain using private IP and a secure Tunnel to Cloudflare!

You now have an application available to the internet with a database and auto-scaling capabilities.

Where does our trip take us next? 🚀

The fun doesn't stop here. Continue visiting this page and following the Software Engineering Entrepreneurship series. I will take you along the journey as we work on improved security measures, building out CI/CD and more.

Changelog

February 24, 2023

• Removed the previous use of exposing our GKE Workload using a public IP load balancer. This document now covers securing your application behind a private IP and establishing a Tunnel directly to Cloudflare and your custom domain.

Stay connected 💬

I would love to connect with you and follow journeys of your own in life. Connect with me on LinkedIn, Twitter or our home at DEV.