Deploying an AWS EKS with StatefulSet and EBS Volume using Terraform. Kubernetes Series - Episode 2

Cloud people!

In the last Episode covered steps to set up a VPC, subnets, route tables, NAT gateways, and deploying an EKS cluster with managed node groups.

Deploying an AWS EKS using Terraform. Kubernetes Series - Episode 1 (Deprecated)

Javier Sepúlveda ・ Nov 15 '23

#kubernetes #aws #terraform #tutorial

In this episode, the focus shifts to deploying the first application within Kubernetes while ensuring persistent storage for the application's data even when pods are ephemeral.

Requirements

• AWS CLI
• Kubectl
• Terraform
• k9s

Let's see how we can do this using terraform and raw manifest.

Reference Architecture

The reference architecture operates within a node.

Currently, the focus is on adding add-ons to the EKS cluster, considering that the EKS cluster was deployed in the previous episode.

Step 1.

Adding the aws-ebs-csi-driver allows the utilization of the EBS driver for the application.

This is the link for all code terraform.

  cluster_addons = {
    coredns = {
      most_recent = true
    }
    kube-proxy = {
      most_recent = true
    }
    vpc-cni = {
      most_recent = true
    }
    aws-ebs-csi-driver = {
      most_recent = true
    }
  }

Step 2.

It is very important that the next step is to add the permissions policy in the node configuration so that the driver can create the EBS Volume, for this it is necessary to add the following code that allows to use a managed policy for the ebs csi driver.

  iam_role_additional_policies = {
      AmazonEBSCSIDriverPolicy  = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
  }

When the eks driver addon is installed, it is possible to list the ebs pods that are running in the kube-system namespace.

kubectl get pods -n kube-system

kubectl get storageclass -n kube-system

Now, that the EBS driver is installed. It is possible to build an application using the raw manifests.

Step 3.

First apply the service. Remember, you must create the namespace manually.

kubectl create ns episode2

kubectl apply -f manifests/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql-svc
  namespace: episode2
  labels:
    app: mysql-db
spec:
  selector:
    app: mysql-db
  ports:
    - name: mysql-db
      port: 3306
      targetPort: 3306
  clusterIP: None

Output:

service/mysql-svc created

Step 4.

Now, it is necessary to use a stateful set, there are some reasons to use a stateful set and not a deployment and the first reason is that the app must have persistent storage.

Deployments are excellent for apps stateless, and this is not the case.

kubectl apply -f manifests/statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql-db
  namespace: episode2
spec:
  serviceName: mysql-svc
  replicas: 2
  selector:
    matchLabels:
      app: mysql-db
  template:
    metadata:
      labels:
        app: mysql-db
    spec:
      containers:
      - name: mysql-db
        image: mysql:latest
        ports:
          - containerPort: 3306
        volumeMounts:
          - name: statefulset-storage
            mountPath: /var/lib/mysql
        env:
          - name: MYSQL_ROOT_PASSWORD
            value: "segoja7secure!" #This is not recommend, use secrets!
          - name: MYSQL_USER
            value: "user" #This is not recommend, use secrets!
          - name: MYSQL_PASSWORD
            value: "episode2" #This is not recommend, use secrets!
  volumeClaimTemplates:
  - metadata:
      name: statefulset-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "gp2"
      resources:
        requests:
          storage: 2Gi

Output:

statefulset.apps/mysql-db created

Note: Now, the storage class was automatically configured when the cluster was created.

In the aws console, these are EBS volumes that were created.

Step 5.

StorageClass:

pvc:

Step 6.

Now you can perform a storage validation, create a database and put some data there.

First it is necessary to perform a port forwarding to connect to the database from a client, using k9s is easy.

Now, you can access from the client, in this case the client is heidi, you can use any client.

From customer, a database named episode2 has been created, additional a table named mytable with two rows FirstName and LastName.

Adding some data.

Step 7.

Recreating pods of statefulset.

In this step, the statefulset pods have been removed so that they are recreated, the age is only seconds.

By connecting back on a Heidi client to the mysql pod, you can see that the data is persistent and the database, table and data are stored in the ebs volume, which shows that the data is being persisted beyond the lifecycle of the pod.

In this phase, a statefulset database pod was created using the ebs volume.

The following addons have been installed:
• ebs-csi

Successful!!