DEV Community

Frederic
Frederic

Posted on

Setup KOPS on AWS with GruCloud

#aws #kops #iac

The aim of this tutorial is to automatically create and destroy the AWS resources required by kops, a tool to create a Kubernetes cluster.

The section 'setup your environment' from the official kops documentation will be automated with GruCloud

Below is the diagram generated from the target code, it illustrates the resources and their association with each other:

kops-graph

Regarding this DNS scenario, the case of a subdomain where a top-level hosted zone already exists is implemented.

TD;DR

  1. Get this example code and install the dependencies.
  2. Edit the configuration file and set the domain name, the subdomain name, the region, and the zone.
  3. gc apply

All the AWS resources required by kops should have been created. The environment file kops.env containing the necessary information should have been generated too.

You are now ready to create a cluster with kops,

Here are a few npm scripts wrapper: npm run kops:create, npm run kops:update and npm run kops:validate.

Steps

Here is a description of the steps that are automated:

IAM

  • create a kops group, attach 5 IAM policies.
  • create a kops user, attach the user to the kops group.
  • create access and secret key for the kops user.

Route53

  • create a hosted zone for a subdomain.
  • create a DNS record of type NS in the top-level hosted zone with the DNS servers as values from the subdomain hosted zone.

S3

  • create an S3 bucket with encryption and versioning.

kops.env file

  • create a file containing the environment variable for kops

You will be free from performing all these commands manually. The same applies to the destruction of all these resources.

Requirements

Install

Clone this code, change to the kops folder, install the npm dependencies:

git clone https://github.com/grucloud/grucloud
cd grucloud/examples/aws/kops
npm install
Enter fullscreen mode Exit fullscreen mode

Configuration

Edit config.js and set the domainName, the subDomainName, the zone and the region:

Double check your configuration with gc info:

  - provider:
      name: aws
      type: aws
    stage: dev
    config:
      projectName: @grucloud/create-aws-kops
      kops:
        domainName: grucloud.org
        subDomainName: kops.example.grucloud.org
        groupName: kops
        userName: kops
      stage: dev
      zone: us-east-1a
      accountId: 4444454555555
      region: us-east-1
Enter fullscreen mode Exit fullscreen mode

The domain name must be registered with Route53 for the current AWS user.
Let's also verify that a top level hosted zone already exists.
You could use the gc list command with the Route53Domain and HostedZone type filter:

gc list  -t Route53Domain -t HostedZone
Enter fullscreen mode Exit fullscreen mode 
[...Truncated]
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ aws                                                                                         │
├────────────────────┬────────────────────────────────────────────────────────────────────────┤
│ Route53Domain      │ grucloud.org                                                           │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ HostedZone         │ grucloud.org.                                                          │
└────────────────────┴────────────────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

iac.js

For your information, the architecture is described in iac.js.
In this use, the cloud provider is AWS, so we'll use the GruCloud AWS Provider to create the resources.

Target Graph

Another way to explore the iac.js is to generate a diagram of the target resources:

gc graph
Enter fullscreen mode Exit fullscreen mode

Deploying

Ready to deploy the user, group, s3 bucket, route53 hosted zone and record ?

gc apply
Enter fullscreen mode Exit fullscreen mode

The AWS resources should have been deployed.
Let's find out our live resources as well as a diagram showing the dependencies between these resources:

gc list --graph --our
Enter fullscreen mode Exit fullscreen mode 
┌─────────────────────────────────────────────────────────────────────────────────────────────┐
│ aws                                                                                         │
├────────────────────┬────────────────────────────────────────────────────────────────────────┤
│ IamPolicy          │ AmazonEC2FullAccess                                                    │
│                    │ AmazonRoute53FullAccess                                                │
│                    │ AmazonS3FullAccess                                                     │
│                    │ IAMFullAccess                                                          │
│                    │ AmazonVPCFullAccess                                                    │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ IamGroup           │ kops                                                                   │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ S3Bucket           │ kops.example.grucloud.org                                              │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ IamUser            │ kops                                                                   │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ HostedZone         │ kops.example.grucloud.org.                                             │
│                    │ grucloud.org.                                                          │
├────────────────────┼────────────────────────────────────────────────────────────────────────┤
│ Route53Record      │ kops.example.grucloud.org-ns                                           │
└────────────────────┴────────────────────────────────────────────────────────────────────────┘
11 resources, 15 types, 1 provider
Enter fullscreen mode Exit fullscreen mode

diagram-live.partial.svg

Envirornment variables

At the end of the deployment, the environment file kops.env is generated with the variables required by kops:

# kops.env
export AWS_ACCESS_KEY_ID=XXXXXXNBM2ZQEPXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXiXmSB3aZTK/AxOOvSPcGby3XXXXXX
export NAME=kops.example.grucloud.org
export KOPS_STATE_STORE=s3://kops.example.grucloud.org
export REGION=eu-west-2
export ZONE=eu-west-2a
Enter fullscreen mode Exit fullscreen mode

Source with variables with:

source kops.env
Enter fullscreen mode Exit fullscreen mode

When the deploment is destroyed with gc destroy, kops.env is removed.

The file hook.js is the place where this logic is implemented.

NPM kops scripts

The following npm scripts manage the kops commands, the environment variables are sourced from kops.env.

npm run kops:create
npm run kops:update
npm run kops:validate
Enter fullscreen mode Exit fullscreen mode

List Resources

Let's fetch all the live resources, we'll see that kops creates many resources such as autoscaling groups, ec2 instances, subnets, vpc, internet gateway, volumes, key pair and so on:

gc list --graph --all --default-exclude --types-exclude Certificate --types-exclude Route53Domain --types-exclude NetworkInterface
Enter fullscreen mode Exit fullscreen mode

kops-diagram-live-all

You could inspect and generate a diagram of any existing AWS infrastruture for the most used resources

Destroy

To destroy the resources created by GruCloud, use the destroy command.

Ensure the cluster is destroyed before.

npm run kops:destroy
gc destroy
Enter fullscreen mode Exit fullscreen mode

Alternatively, gc could also destroy all the resources created by kops, use the all flag:

gc destroy --all
Enter fullscreen mode Exit fullscreen mode

Further Step

Congratulations, you know how to create and destroy a Kubernetes cluster with kops.
What about a load balancer, DNS records, SSL certificates ? Grucloud provides some ready made modules distributed with npm, the node package manager.

Have a look at:

On the Kubernetes side, be aware of the GruCloud Kubernetes Provider. In a nutshell, instead of writing YAML manifest, Javascript is used instead to define the manifests, no more templating engine, enjoy a real programming language instead.

Would you like to deploy a full stack application on EKS ? Choose the flavour depending on who is reponsible to create the load balancer, target groups, listener and rules:

  • Load balancer resources created inside the cluster with the AWS Load Balancer Controller: eks-lbc.

  • A leaner solution where the load balancer resources are created by GruCLoud outside the cluster: eks-lean.

Links

Top comments (0)

Read next

seifrajhi profile image

Simplify AWS resource access with EKS Pod Identity Agent and Python SDK

saifeddine Rajhi -

mkdotam profile image

EC2 instance deployment unification across AWS Organizations

mikayel ghazaryan -

bharath_madhu profile image

Sending Emails with AWS SES: A Comprehensive Guide

BHARATH M -

szymonszym profile image

AWS Lambda with Go, initial boilerplate

szymon-szym -