Raji Risikat Yewande for AWS Community Builders

Posted on Apr 23, 2023

Costly EKS Cluster Set Up? AWS to the rescue!

#beginners #aws #kubernetes #eks

Storyline

The goal was to do an automated Terraform Kubernetes Deployment. I eventually provisioned a production grade EKS Cluster using Terraform-EKS-modules and it created other AWS resources to be intergrated along with it. As the beginner that I was, I left the cluster running for days hoping to sort out deploying different types of microservice applications to the same cluster then finally clean up. The total bill accrued after a while was jaw-dropping and problematic but AWS came to my rescue and sorted it all out.

Aim

This walks you through the process of creating a standard EKS-Cluster using Terraform modules in AWS EKS Cluster Creation and steps to request a total cost/bills waiver from AWS in AWS Bills Waiver. The waiver process is useful for other workloads and not just this cluster set up.

AWS EKS Cluster Creation

Essentials:

An AWS Account, create a free tier here.
Be logged in an IAM user with Admin priviledges, secret and access keys.
Ubuntu Host OS (Local PC or Virtual Machine).
VS code or any IDE of your choice.
An S3 bucket to store the terraform state files remotely
Basic Linux Terminal use

Installations:

Install latest Terraform on the command line:

sudo apt update && apt upgrade -y
sudo apt install -y gnupg software-properties-common curl
curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt update
sudo apt install terraform
terraform -v

Install the Terraform extension by Anton Kulikov on VS code for syntax highlights and efficiency.
Install AWS CLI :

sudo apt-get update
sudo apt-get install -y less curl unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -q awscliv2.zip
sudo ./aws/install
aws --version

Configure AWS CLI with aws configure command:
- fill up outputs with secret and access keys previously created from IAM user
- add us-east-1 as region
- Test configuration details with aws configure list
- No need to create environment variables as runnng env | grep AWS will not recognize the enviroment variables set in another terminal tab hence aws configure is appropriate for global setup.
```
export AWS_ACCESS_KEY_ID="YOUR_AWS_ACCESS_KEY_ID"
export AWS_SECRET_ACCESS_KEY="YOUR_AWS_SECRET_ACCESS_KEY"
export AWS_DEFAULT_REGION="us-east-1"
```

Install Kubectl as the command line tool that interacts with the Kubernetes Cluster:

sudo apt-get update -y
sudo apt-get install -y apt-transport-https ca-certificates curl
curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update -y
sudo apt-get install -y kubectl
kubectl version --client

(Optiona) Install Graphviz to visualize the terraform plan process:

sudo apt install graphviz

Architecture

This shows the summarized version of how the cluster would be formed via the terraform module(more like a group of interwoven resources used together). Other resources such as VPC, Security groups, NAT and Internet gateways for routing traffic from inside the private subnets and the internet respectively..etc. would be created alongside it.
This set up has the VPC created with three Public and three Private Subnets which tallies with the three availabilty zones present in us-east-1 region.

The values in the Terraform Configuration aren't fixed and can be changed according to individual's preference but I'll suggest that beginners follow through these exact values in order to eradicate bugs and avoidable errors.

Procedures:

Navigate to preferred working directory by git cloning my repository git clone https://github.com/wandexdev/k8s-microservices-deployment-via-CICD-terraform.git, then cd terraform/eks_creation-terraform/ to switch directory.

Note

Ensure your present working directory is ~/k8s-microservices-deployment-via-CICD-terraform/terraform/eks_creation-terraform

Confirm with the command pwd.

All required Terraform files(.tf) are present in the folder on my repo, you can create a folder for yours to manually write the .tf files if you did not clone the repository.

All required .tf files
The majority configuration of the module used was gotten from terraform-aws-modules/eks in Terraform's Registry. You can add more/switch the non required inputs or decide which preferred resources or versions you want outside the ones I already chose in my repository.
Create a providers.tf file to recognize the specific terraform-provider plugins that directly interact with AWS service.


        provider "aws" {
            region  = var.region
        }

Create a s3_backend.tf file to store remote state files in an external S3 bucket as this ensures multiple users or CI server gets the latest state data before running terraform. Its also more secure. > PS > > Please ensure the S3 bucket is created prior to terraform being initialized


        terraform {
        required_version = ">= 0.12"
        backend "s3" {
            bucket         = var.S3_bucket_name
            key            = var.bucket_key
            region         = var.region
        }
     }

Create a variables.tf file to declare all variables to be used in this project.


        #----------------------------BACKEND-------------------------
        variable "S3_bucket_name" {
            description = "name of S3 bucket for remote backend storage"
        }

        variable "bucket_key" {
            description = "It's the file path inside the bucket"
        }

        variable "region" {
            description = "The region of cluster deployment"
        }

        #--------------------------------VPC-------------------------------#

        variable "vpc_cidr_block" {
            description = "ip range for vpc"
        }

        variable "private_subnet_cidr_blocks" {
            description = "ip range for private subnet"
        }

        variable "public_subnet_cidr_blocks" {
            description = "ip range for public subnets"
        }

Create a vpc.tf file for the configurations of the Virtual Private Environment the EKS-cluster would be created in.


        #-----------DATA SOURCE-------------------#
        # By state
        data "aws_availability_zones" "available" {
        }

        module "wandek8s-vpc" {
        source                                    = "terraform-aws-modules/vpc/aws"
        version                                   = "2.64.0"

        name                                      = "wandek8s-vpc"
        cidr                                      = var.vpc_cidr_block
        private_subnets                           = var.private_subnet_cidr_blocks
        public_subnets                            = var.public_subnet_cidr_blocks
        azs                                       = data.aws_availability_zones.available.names 

        enable_nat_gateway                        = true
        single_nat_gateway                        = true
        enable_dns_hostnames                      = true

        tags                                      = {
                "kubernetes.io/cluster/wandek8s-eks-cluster" = "shared"
        }

        public_subnet_tags                        = {
                "kubernetes.io/cluster/wandek8s-eks-cluster" = "shared"
                "kubernetes.io/role/elb"            = 1
        }

        private_subnet_tags                       = {
                "kubernetes.io/cluster/wandek8s-eks-cluster" = "shared"
                "kubernetes.io/role/internal-elb"   = 1
        }
        }

Create an outputs.tf file to share data between Terraform configurations and external tools. Outputs are also the only way to share data between child module and root module of the configuration.


    output "cluster_id" {
    description       = "EKS cluster ID."
    value             = module.eks.cluster_id
    }

    output "cluster_name" {
    description       = "Amazon Web Service EKS Cluster Name"
    value             = module.eks.cluster_name
    }

    output "cluster_endpoint" {
    description       = "Endpoint for Amazon Web Service EKS "
    value             = module.eks.cluster_endpoint
    }

    output "region" {
    description       = "Amazon Web Service EKS Cluster region"
    value             = var.region
    }

    output "cluster_security_group_id" {
    description       = "Security group ID for the Amazon Web Service EKS Cluster "
    value             = module.eks.cluster_security_group_id
    }

    output "cluster_ca_certificate" {
    value             = module.eks.cluster_certificate_authority_data
    }

Create an eks-cluster.tf file which specifies the Terraform Kubernetes Provider in order to create Kubernetes objects, the module configurations, the partly managed nodes groups + node instances to be created in the cluster etc.


        #-----------provider to communicate with Kubernetes’ resources----------#

        provider "kubernetes" {
            host                        = module.eks.cluster_endpoint # end point of k8s (API server)
            token                       = data.aws_eks_cluster_auth.wandek8s-eks-cluster.token
            cluster_ca_certificate      = base64decode(module.eks.cluster_certificate_authority_data)
        }

        #--------Query data ------------#
        #data "aws_eks_cluster" "wandek8s-eks-cluster" {
        #    name                        = module.eks.cluster_id
        #}

        data "aws_eks_cluster_auth" "wandek8s-eks-cluster" {
            name                        = module.eks.cluster_name
        }

        module "eks" {
        source                        = "terraform-aws-modules/eks/aws"
        version                       = "19.10.0"

        cluster_name                  = "wandek8s-eks-cluster"
        cluster_version               = "1.24"

        subnet_ids                    = module.wandek8s-vpc.private_subnets
        vpc_id                        = module.wandek8s-vpc.vpc_id
        cluster_endpoint_public_access = true

        tags = {
            environment = "development"
            application = "wandek8s"
        }

        eks_managed_node_groups = {
            one = {
            min_size     = 1
            max_size     = 4
            desired_size = 3

            instance_types = var.instance_type
            }
        }
        }

Finally Create a terraform.tfvars file to input the variable values defined in variables.tf file. Herein lies the ranges for VPC CIDR block, specific resources names etc. Please ensure to input the absolute path for private and public key located in the HOST OS as mine is removed for security reasons.

This is a very sensitive file that shouldn't be pushed to version control.


        S3_bucket_name               = "wandek8s-bucket"

        bucket_key                   = "cicd-server/state.tfstate"

        region                       = "us-east-1"

        key_pair                     = "cicd-server"

        local_public_key_location    = ""

        local_private_key_location   = ""

        instance_type                = ["t3.medium"]

        vpc_cidr_block               = "10.0.0.0/16"

        public_subnet_cidr_blocks    = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]

        private_subnet_cidr_blocks   = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]

Initialize the current working directory with terraform init
Terraform init Output
Next terraform plan to preveiw the list of the desired state to be executed.
Terraform plan part Output
Optionally, we can visualize the terraform plan process before finally creating the resources in a graph via terraform graph -type plan | dot -Tsvg > graph.svg. A graph.svg file would be created that can abe viewed on your PC or via any svg viewing tool.
Next is terraform apply and patience because this process would take a little while.
Terraform apply Output
Console Verification: After a successful apply, we can confirm the resources created on the AWS console
Validate the cluster with kubectl via aws eks update-kubeconfig --name wandek8s-eks-cluster and now the cluster is fully ready for interaction and deployments.
Please do remember to Clean up the infrastructure as the spend for this set up is costly. Terraform makes this super easy by typing terraform delete. If not proceed to the next section where I explain how I got my bills waived as a 1 time occurence.

AWS Bills Waiver.

Steps:

Navigate to the AWS console and select support centre from the drop down list

Figure 1: Support Centre on web console
Ensure Account & Billing is ticked then proceed to Create Case

Figure 2: Create Case in Account & Billing
Follow the option specifications in Figure 3 below.

Figure 3: Selecting Service and Category options
Fill the Subject Summary. I recommend an honest breakdown of the events that lead to the cost build, documents can be attached also. Here is my description version here, you can use it as a template.

Hello AWS Support team,

I am requesting your assistance with my recent AWS charges. As a beginner to AWS processes, I have incurred very high unexpected charges that I am not able to afford at this moment while setting up an EKS Cluster with modules and I am hoping to find a solution that will allow me to continue learning and using AWS services without experiencing any financial burden.


I understand that some of the charges may have been due to my lack of knowledge about AWS pricing and usage. I have since taken steps to educate myself and ensure that I am aware of the free tier limits and best practices for minimizing costs and curbing hidden costs.


I would be very grateful if my account be waived off all the pending AWS charges on my account as a one-time courtesy due to my status as a new user. This would allow me to continue my learning journey with AWS without any worry about unexpected charges.


Thank you for your time and attention to this matter. I appreciate your assistance in resolving this issue.

Select Web and you'll be contacted via chat with a wave off alongside few aws credits to clear future debts. Depending on your location, you can also select Phone, provide your phone number and you'll be contacted by the team.

Thank you for coming on this journey with me and I hope I have been able to guide you in 2 different ways with my experiences with AWS awesomeness.

Feel free to reach out in the comment section with doubts or questions as well as directly on twitter. Cheers to a fruitful learning journey with AWS.