I hadn't had a chance to try Kubernetes for a long time, but finally a few weeks ago I got one: I've been working on a simple Ruby application (Slack bot), and decided to deploy it to Google Cloud Kubernetes Engine.
There was an additional requirement to set up continuous deployment process. For that, I chose CircleCI service.
NOTE: This post is just a playbook containing all the actions I made to get the application up and running.
Software versions:
- Kubernetes 1.9.4-gke.1
- Google Cloud SDK 195.0.0
Preparing GCloud
Let's assume that you already have a Google Cloud account.
Go to http://console.cloud.google.com/ and create a new project (my-project
) with Kubernetes API enabled (more thorough description of this step could be found here).
Next step–install gcloud
CLI:
# for MacOS/Homebrew users it's pretty simple
brew install caskroom/cask/google-cloud-sdk
Then you must authenticate yourself:
gcloud auth login
(Optionally) Configure the default project (to avoid specifying it with every command):
gcloud config set project my-project
Create K8S cluster:
# "g1-small" is a minimal instance type (as of the time of writing) that allows having only one node
gcloud container clusters create my-cluster --machine-type g1-small --num-nodes 1
Obtain the cluster's credentials:
gcloud container clusters get-credentials my-cluster
Get the list of clusters to verify that everything is okey:
gcloud container clusters list
The output looks like this:
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
my-app-web us-east1-b 1.9.4-gke.1 35.227.92.102 g1-small 1.9.4-gke.1 1 RUNNING
Preparing application
As I've already told, the application I want to deploy is a simple Ruby-only app (i.e., no databases, caches, persistent stores). See the links at the end of the post on how to deploy Rails applications.
We need to build a Docker image and push to Google Container Registry.
Here is my Dockerfile
:
FROM ruby:2.5.0
RUN apt-get update && apt-get install -y build-essential git
RUN mkdir -p /app
WORKDIR /app
ENV LANG C.UTF-8
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5
COPY . .
# We want to include the current git revision information
# to a build to track releases
ARG BUILD_SHA=unknown
ENV BUILD_SHA ${BUILD_SHA}
EXPOSE 3000
ENTRYPOINT ["bundle", "exec"]
CMD ["puma", "-p", "3000"]
Let's build it:
docker build -t my-app -f Dockerfile.prod .
NOTE: I use Dockerfile.prod
file for production and Dockerfile
for development.
Then tag it using a specific GCloud format (containing region and project name):
docker tag my-app us.gcr.io/my-project/my-app:v1
And push:
gcloud docker -- push us.gcr.io/my-project/my-app:v1
Ok. We're almost there. Now we have to tell somehow to our K8S cluster to get this image and run.
For that we're going to use K8S deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
labels:
name: web
spec:
replicas: 2
selector:
matchLabels:
name: web
template:
metadata:
labels:
name: web
spec:
containers:
- name: web
image: gcr.io/my-project/my-app:latest
ports:
- containerPort: 3000
livenessProbe:
httpGet:
path: /_health
port: 3000
initialDelaySeconds: 10
timeoutSeconds: 1
readinessProbe:
httpGet:
path: /_health
port: 3000
initialDelaySeconds: 10
timeoutSeconds: 1
K8S documentation is a good place to read about deployments; no need to recall it here.
I'd only like to pay attention to the livenessProbe
and readinessProbe
sections: they tell K8S how to monitor the application state to provide rolling update functionality and make sure that the desired number of replicas are up and running. Note that your application should provide endpoints for these checks (/_health
in my case).
Now we need to install one more tool–kubectl
–to operate our cluster:
gcloud components install kubectl
It's time to deploy finally deploy our application! Let's create our deployment:
kubectl create -f kube/web-deployment.yml
To get the information about our deployment you can see the list of pods:
kubectl get pods
More information about a pod:
kubectl describe pod web-<pod-id>
Updating application manually
Your application is up and running. How to push a new release?
If you want to update your deployment configuration then just run the command below:
kubectl apply -f kube/web-deployment.yml
For updating a codebase, you should tell your cluster to use a new image:
docker tag my-app us.gcr.io/my-project/my-app:v1.1
gcloud docker -- push us.gcr.io/my-project/my-app:v1.1
kubectl set image deployment web web=gcr.io/my-project/my-app:v1.1
K8S will take care of creating new pods and replacing the old ones.
Running three commands and manually providing new versions is not the most convenient way to deploy, isn't it?
Let's make someone else do all the dirty work.
CircleCI integration
CircleCI 2.0 is a very flexible automation tool.
We want to make our deployment as easy as just making git push
(or merging PR into master
branch).
Below you can find an example .circle/config.yml
with the comments explaining every step:
version: 2
workflows:
version: 2
build_and_deploy:
jobs:
- build
- deploy:
# Run the "deploy" job only if the "build" job was successful
requires:
- build
# Run deploy job only on master branch
filters:
branches:
only:
- master
jobs:
# This job is responsible for running tests and linters
build:
docker:
- image: circleci/ruby:2.5.0
environment:
RACK_ENV: test
steps:
- checkout
- run:
name: Install gems
command: bundle install
- run:
name: Run Rubocop and RSpec
# Our default rake task contains :spec and :rubocop tasks
command: bundle exec rake
# Cache all the project files to re-use in the deploy job
# (instead of pulling the repo again)
- persist_to_workspace:
root: .
paths: .
deploy:
docker:
# official image which includes `gcloud` and `kubectl` tools
- image: google/cloud-sdk
# project information
environment:
GOOGLE_PROJECT_ID: my-project
GOOGLE_COMPUTE_ZONE: us-east1-b
GOOGLE_CLUSTER_NAME: my-app
steps:
# Attach previously cached workspace
- attach_workspace:
at: .
# SERVICE_KEY provides access to your GCloud project.
# Read more here https://circleci.com/docs/2.0/google-auth/
- run: echo $GCLOUD_SERVICE_KEY > ${HOME}/gcloud-service-key.json
# Authenticate gcloud
- run: gcloud auth activate-service-account --key-file=${HOME}/gcloud-service-key.json
# Configure gcloud (the same steps as you do on your local machine)
- run: gcloud --quiet config set project ${GOOGLE_PROJECT_ID}
- run: gcloud --quiet config set compute/zone ${GOOGLE_COMPUTE_ZONE}
- run: gcloud --quiet container clusters get-credentials ${GOOGLE_CLUSTER_NAME}
# Enable remote Docker (https://circleci.com/docs/2.0/building-docker-images/)
- setup_remote_docker
# Build Docker image with the current git revision SHA
- run: docker build -t brooder -f Dockerfile.prod --build-arg BUILD_SHA=${CIRCLE_SHA1} .
# Use the same SHA as our image version
- run: docker tag brooder gcr.io/my-project/my-app:${CIRCLE_SHA1}
# Using remote Docker is a little bit tricky but this "spell" works
- run: gcloud docker --docker-host=$DOCKER_HOST -- --tlsverify --tlscacert $DOCKER_CERT_PATH/ca.pem --tlscert $DOCKER_CERT_PATH/cert.pem --tlskey $DOCKER_CERT_PATH/key.pem push gcr.io/my-project/my-app:${CIRCLE_SHA1}
# and finally, "deploy" the new image
- run: kubectl set image deployment web web=gcr.io/my-project/my-app:${CIRCLE_SHA1} --record
Keeping secrets
Let's cover one more topic here–managing app secrets.
We use Kubernetes Secrets to store sensitive information (such as third-party services API tokens).
First, you have to create secrets definition file:
# kube/app-secrets.yml
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
# The value should be BASE64 encoded
github_token: b2t0b2NhdA==
slack_token: Y3V0bWVzb21lc2xhY2s=
Don't forget to encode the value into base64. For example:
echo -n "myvalue" | base64
Push secrets to the cluster:
kubectl create -f kube/app-secrets.yml
#=> secret "mysecret" created
NOTE: Remove app-secrets.yml
right after pushing to K8S (or, at least, add to .gitignore
).
Now you can pass the secrets to your app through env variables:
# web-deployment.yml
apiVersion: apps/v1
kind: Deployment
# ...
spec:
# ...
templaee:
# ...
spec:
containers:
- name: web
image: gcr.io/my-project/my-app:latest
# ....
env:
- name: MY_GITHUB_TOKEN
valueFrom:
secretKeyRef:
name: mysecret
key: github_token
- name: MY_SLACK_TOKEN
valueFrom:
secretKeyRef:
name: mysecret
key: slack_token
Links
- Kubernetes Concepts
- Kubernetes and Deploying to Google Container Engine
- Deploying Rails on Kubernetes
- Rails 5.2 Credentials and Kubernetes
Read more dev articles on https://evilmartians.com/chronicles!
Top comments (1)
Thank you for sharing, really great article !