DEV Community

Cover image for 🚀 Unleashing the Power of Cloud Magic: Transforming a Lone AWS EC2 Instance into a K8s Powerhouse! 🌐🔥
Ukeme David Eseme
Ukeme David Eseme

Posted on

🚀 Unleashing the Power of Cloud Magic: Transforming a Lone AWS EC2 Instance into a K8s Powerhouse! 🌐🔥

#cloudnative #aws #devops #kubernetes

Table of Content:

  1. Prequisite
  2. Introduction
  3. SSH into EC2
  4. Install Docker
  5. Install Kubectl
  6. Install KIND
  7. Setup Kubernetes Cluster
  8. Setup Visualizer (KubeOps View)

Perquisite

  1. EC2 instance running Amazon Linux 2023 AMI - How to Video, Doc
  2. Available private key pair for the instance

Introduction

Welcome to the realm of cloud enchantment! In this captivating journey, we will delve into the art of transforming a solitary AWS EC2 instance into a formidable Kubernetes (K8s) powerhouse. Brace yourself as we unravel the secrets of cloud magic, unlocking the potential of your EC2 instance to orchestrate a dynamic Kubernetes cluster. With a touch of innovation and a dash of determination, you'll soon wield the power of the cloud like never before.

SSH into EC2

SSH stands for "Secure Shell." It is a cryptographic network protocol used for securely connecting to a remote server or device over an unsecured network.

To connect via SSH to the virtual Machine (EC2), you would need a secure shell client like Putty or MobaXterm or just your plain terminal.

I would be using Tabby Terminal.

  • Open the terminal and navigate to the directory were you downloaded the *EC2 Key Pair *.

In my case its

cd ~/Downloads/
Enter fullscreen mode Exit fullscreen mode
  • Run this command, if necessary, to ensure your key is not publicly viewable. 
chmod 400 "your-Key-pair-file.pem"
Enter fullscreen mode Exit fullscreen mode
  • Connect to your instance using its Public DNS For example 
ssh -i "your-Key-pair-file.pem" ec2-user@ec2-your-ip.compute-1.amazonaws.com
Enter fullscreen mode Exit fullscreen mode

Once you are connected to the instance, you should see a welcome screen like the below.

Amazon Linux Welcome Screen

Install Docker

docker logo

Amazon Linux 2023 uses dnf as its package manager.

1. Update AL2023 Packages

Since its a new linux VM, run the below command to perform an update.

sudo dnf update
Enter fullscreen mode Exit fullscreen mode

This command is used to update the installed packages and package cache on a Fedora system.

2. Installing Docker on Amazon Linux 2023 

sudo dnf install docker
Enter fullscreen mode Exit fullscreen mode

The above installs the Docker Engine, the Docker command-line interface, and the containerd runtime.

Install docker

3. Start and Enable Docker Service

After installation docker services, don't start up by default, we have to manually start the process.

sudo systemctl start docker
Enter fullscreen mode Exit fullscreen mode

Also, we want to set docker to automatically start with system boot

sudo systemctl enable docker
Enter fullscreen mode Exit fullscreen mode

To be sure docker is currently running as expected, we need to check its status.

sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

You should have a similar result, like the image below.

Docker status

4. Enable Docker to run without requiring sudo

Once the installation is finished, it's cumbersome to use sudo every time you want to execute Docker commands. To alleviate this inconvenience, we need to include our current user in the Docker group. Utilize the provided command to accomplish this."

sudo usermod -aG docker $USER
Enter fullscreen mode Exit fullscreen mode

apply the changes to the docker group

newgrp docker
Enter fullscreen mode Exit fullscreen mode

To verify and check docker version

docker version
Enter fullscreen mode Exit fullscreen mode

You should have data similar to the below image

Docker version

Install Kubectl

kubectl
kubectl is a command-line interface (CLI) tool used to interact with Kubernetes clusters. It allows users to perform various operations on Kubernetes resources, such as deploying applications, managing pods, services, and deployments, inspecting cluster resources, and debugging cluster issues.

Note: You should ensure that the version of kubectl you use is within one minor version of your Kubernetes cluster. For instance, a client with version v1.29 can communicate effectively with control planes of versions v1.28, v1.29, and v1.30. Utilizing the most recent compatible kubectl version is essential to prevent unexpected complications.

1. Install the kubectl binary on Linux using curl:

Download the latest release with the command:

     curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
Enter fullscreen mode Exit fullscreen mode

2. Validate the binary (optional)

Download the kubectl checksum file:

   curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
Enter fullscreen mode Exit fullscreen mode

Validate the kubectl binary against the checksum file:

echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
Enter fullscreen mode Exit fullscreen mode

If valid, the output is:
kubectl: OK

3. Install kubectl 

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
Enter fullscreen mode Exit fullscreen mode

Test to ensure the version you installed is up-to-date:

kubectl version --client
Enter fullscreen mode Exit fullscreen mode

Or use this for detailed view of version:

kubectl version --client --output=yaml
Enter fullscreen mode Exit fullscreen mode

Install KIND

Kind Logo
We would be using KIND to create our kubernetes cluster.
What is KIND ?

In Kubernetes, "Kind" refers to Kubernetes in Docker. It is a tool for running local Kubernetes clusters using Docker container "nodes".
Its is a lightweight and easy-to-use Kubernetes environment for testing and development purposes.

For AMD64 / x86_64 

[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.21.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
Enter fullscreen mode Exit fullscreen mode

Confirm KIND is installed. 

 kind --version
Enter fullscreen mode Exit fullscreen mode

You should see the current version of KIND installed.

Setup Kubernetes Cluster

in your terminal create a new file with the below command:

nano three-node-cluster.yml
Enter fullscreen mode Exit fullscreen mode

Paste this code in the editor

# three node (two workers) cluster config
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 32000
    hostPort: 32000
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 32100
    hostPort: 32100
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 30000
    hostPort: 30000
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 30100
    hostPort: 30100
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 30200
    hostPort: 30200
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 30300
    hostPort: 30300
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 30400
    hostPort: 30400
    listenAddress: "0.0.0.0"
    protocol: tcp
- role: worker
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 8000
    hostPort: 8000
    listenAddress: "0.0.0.0"
    protocol: tcp
  - containerPort: 8080
    hostPort: 8001
    listenAddress: "0.0.0.0"
    protocol: tcp

- role: worker
Enter fullscreen mode Exit fullscreen mode

Use Ctrl+x To save the changes and exit editing.

Now let's break down the configuration:

kind: Specifies the kind of resource being defined, which is a Cluster in this case.
apiVersion: Specifies the version of the Kubernetes API being used.
nodes: Specifies the configuration for the nodes in the cluster.

The first node:

Defined is a control-plane node (role: control-plane). This node has extraPortMappings configured, which maps container ports to host ports. This is useful for accessing services running inside Kubernetes from outside the cluster. The listed container ports are mapped to the same host ports (32000, 32100, 30000, 30100, 30200, 30300, 30400) and listen on all available network interfaces (0.0.0.0) using the TCP protocol.

The second node (role: worker)

Also has extraPortMappings configured. It maps container ports 80, 8000, and 8080 to host ports 80, 8000, and 8001 respectively.

The last node

is simply specified with role: worker, but it doesn't have any extraPortMappings configured.

Create the KIND Cluster 

kind create cluster --config three-node-cluster.yml
Enter fullscreen mode Exit fullscreen mode

Once its done, to get the cluster info

kubectl cluster-info --context kind-kind
Enter fullscreen mode Exit fullscreen mode

create KIND cluster

Get a list of the running nodes

 kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

kubectl get nodes

View all running pods across all namespaces

 kubectl get pods -A
Enter fullscreen mode Exit fullscreen mode

All pods running across all namespaces

Setup Visualizer

kubeops view logo
KubeOps View is a read-only system dashboard for multiple Kubernetes clusters, providing a common operational picture for understanding cluster setups in a visual way. It allows users to render nodes, indicate their overall status, show node capacity, and more.

1. Install Git 

sudo dnf install git
Enter fullscreen mode Exit fullscreen mode

2. Clone Git Repo 

git clone https://github.com/UkemeSkywalker/kube-ops-view
Enter fullscreen mode Exit fullscreen mode

3. Apply kubeOps deployment

Navigate to the clone repository

cd kube-ops-view/
Enter fullscreen mode Exit fullscreen mode

Apply deployment

kubectl apply -f deploy/
Enter fullscreen mode Exit fullscreen mode

4. Check Deployment 

kubectl get pods
Enter fullscreen mode Exit fullscreen mode

kubectl get pods

5. Update EC2 security group inbound rules

In your Ec2 instance details page, scroll down and navigate to the security section.
Click on the default security group. It should take you to the dashboard.

Ec2 security section

Click on edit inbound rules, and add a new rule

Type: Custom TCP
Port: 32000
Source: select My IP

Click on save rule.

Inbound Rule

6. Finally, Access the visualizer on your browser 

http://your-ec2-pubic-ip:32000/#scale=2.0
Enter fullscreen mode Exit fullscreen mode

kube-ops view on Ec2

Conclusion

As our adventure draws to a close, you now possess the knowledge and prowess to harness the full potential of your AWS EC2 instance. From the exhilarating setup of Docker and KIND to the orchestration of your very own Kubernetes cluster, you've embarked on a journey filled with discovery and empowerment.

With KubeOps View offering a visual glimpse into your cloud domain, the possibilities are endless. Embrace the magic of the cloud, and may your Kubernetes endeavors continue to flourish in the ever-expanding landscape of technology.

Until next time, may your clouds be clear and your clusters be mighty! ✨🌐🚀

Top comments (0)

Read next

luqmanbello profile image

Building Production-Grade ECS Anywhere Infrastructure with Custom Capacity Providers

Luqman Bello -

evgenyigumnov profile image

Need Advice on Improving My Codebase: Increased Readability and Error Management

Evgeny Igumnov -

faboulaye profile image

Highly scalable image storage solution with AWS Serverless at ip.labs - Part 1

Firdaws Aboulaye -

she0407 profile image

AWS CODE PIPELINE

Shivani Patel -