Kubernetes tip: automatically update your app configuration using this technique

The "Hands-on guide: Configure your Kubernetes apps using the ConfigMap object" blog post covered how to use the ConfigMap object in Kubernetes to separate configuration from code.

Using environment variables in your application (Pod or Deployment) via ConfigMap poses a challenge — how will your app uptake the new values in case the ConfigMap gets updated? You can obviously delete and recreate the Deployment, but this is undesirable in most cases.

A possible approach is to load ConfigMap contents as a Volume. In this case, Kubernetes ensures that the Volume contents (files containing the config value) are refreshed if the ConfigMap gets updated.

I would love to have your feedback and suggestions! Don't be shy, just tweet or drop a comment.

Let's learn this with a practical example!

The code is available on GitHub

Pre-requisites

You will need a Kubernetes cluster to begin with. This could be a simple, single-node local cluster using minikube, Docker for Mac etc. or a managed Kubernetes service from Azure (AKS), Google, AWS etc.

To access your Kubernetes cluster, you will need kubectl, which is pretty easy to install.

this example uses minikube

Deploy the app

Create the ConfigMap first.

kubectl apply -f https://raw.githubusercontent.com/abhirockzz/kubernetes-configmap-auto-update/master/config.yaml

To keep things simple, the YAML file is being referenced directly from the GitHub repo, but you can also download the file to your local machine and use it in the same way.

Here are the contents of the ConfigMap - it contains three key-value pairs as part of the data section

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default
data:
  foo: bar
  hello: world
  john: doe

Create the application (as a Kubernetes Deployment) which uses the ConfigMap

kubectl apply -f https://raw.githubusercontent.com/abhirockzz/kubernetes-configmap-auto-update/master/app.yaml

Let's look at the Pod spec section:

spec:
  containers:
  - name: configmaptestapp
    image: abhirockzz/configmaptestapp
    volumeMounts:
    - mountPath: /config
      name: appconfig-data-volume
    ports:
    - containerPort: 8080
  volumes:
    - name: appconfig-data-volume
      configMap:
        name: app-config

As explained in one of the previous articles, each key in the ConfigMap is added as a file to the directory specified in the spec i.e. spec.containers.volumeMount.mountPath and the value is nothing but the contents of the file.

By using this approach you can benefit from the fact that volumes are automatically updated if the ConfigMap changes.

Quick note on what the app does

It's a simple Go app which:

• loads the configuration data within an in-memory map
• and allows you to get the value of a specific config key using a REST endpoint

Two REST endpoints:

• /readconfig: to get value for a config key
• /reload: to force the app to read (refresh) config data from disk into memory

To access the REST endpoints, let's expose the app using a NodePort Service and fetch the random port value

kubectl expose deployment configmaptestapp --type=NodePort
PORT=$(kubectl get service configmaptestapp -o=jsonpath='{.spec.ports[0].nodePort}')

Don't worry if you don't know what NodePort service is. For the time being, jsut understsand that it is a way to provide access to your apps in Kubernetes

Test it out...

Get the IP of your minikube host

MINIKUBE_IP=$(minikube ip)

Introspect the REST endpoint of the app. In this case, foo and john correspond to the configuration keys — these are nothing but files in the directory /config

curl http://$MINIKUBE_IP:$PORT/readconfig/foo
//output - bar 

curl http://$MINIKUBE_IP:$PORT/readconfig/john
//output - doe

curl http://$MINIKUBE_IP:$PORT/readconfig/junk
// output - Configuration 'junk' does not exist

You can directly introspect the Pod to (double) check. Start by getting the Pod name

POD_NAME=$(kubectl get pods -l=app=configmaptestapp -o=jsonpath='{.items[0].metadata.name}')

Peek into the /config directory — lists all the files (configuration keys)

kubectl exec $POD_NAME -- ls /config/

//output
foo
hello
john

Look at specific keys

kubectl exec $POD_NAME -- cat /config/foo
//output - bar

kubectl exec $POD_NAME -- cat /config/john
//output - bar

kubectl exec $POD_NAME -- cat /config/junk
//output - cat: can't open '/config/junk': No such file or directory

Ok, this was just a sanity test ...

... what about automatic updates?

Get the config.yaml (ConfigMap) file

curl https://raw.githubusercontent.com/abhirockzz/kubernetes-configmap-auto-update/master/config.yaml -o config.yaml

Make changes to the values (e.g. change values of foo to baz and hello to universe) and update the ConfigMap

kubectl apply -f config.yaml

If you check the Pod file system right away — you will not see the update.

kubectl exec $POD_NAME -- cat /config/foo

What's going on??

This is an eventually consistent mechanism which involves a delay - the reason is that this is handled by the frequency of the kubelet sync process and the TTL (time-to-live) of kubelet ConfigMap cache. You should see the updated value after a few seconds.

Let’s confirm that the REST API also behaves the same way

curl http://$MINIKUBE_IP:$PORT/readconfig/foo

Why do we see the old value? This is because in this specific case, our app reads the data from file system (during start up phase) and saves it in a map. It also provides a reload facility via a REST endpoint — invoke it

curl http://$MINIKUBE_IP:$PORT/reload/

You should now see the updated values

curl http://$MINIKUBE_IP:$PORT/readconfig/hello
//output - universe

curl http://$MINIKUBE_IP:$PORT/readconfig/foo
//output - baz

That's it. You saw how our application was able to make use of the updated ConfigMap without being restarted.

Closing thoughts….

This approach has its pros, cons (and caveats)

Pros

The big plus is that you do not need to restart your app (Deployment) for them to start using the updated data in the ConfigMap.

Cons

• Code changes: if you’re using environment variables (as most apps do), you will need to update you code to read from file system and you'll have to provide a re-load capability as well
• Eventually consistent: You will have to factor in the time delay between the actual ConfigMap update and the volume data sync - whether your app is sensitive to it is something you will need to consider.

Caveat - This feature will not work in case you are loading ConfigMap as a subPath volume

That's all for this blog. If you found this article useful, please like and follow 😃😃

If you are interested in learning Kubernetes and Containers using Azure, simply create a free account and get going! A good starting point is to use the quickstarts, tutorials and code samples in the documentation to familiarize yourself with the service. I also highly recommend checking out the 50 days Kubernetes Learning Path. Advanced users might want to refer to Kubernetes best practices or watch some of the videos for demos, top features and technical sessions.