Learn How to Setup a CI/CD Pipeline from Scratch

In this tutorial, we will take an example of a Go application and setup a CI/CD pipeline.

Go is becoming increasingly popular amongst developers for its ability to simplify and secure the building of modern applications. The language was created by Google and has gained traction due to its open-source nature and ability to write programs in the Go language. Go also provides users with the freedom to build their own front-end websites and applications, as well as making it easy to develop, maintain and use. Enterprises can rely on Go to help build and scale cloud computing systems while enjoying its powerful concurrency features. Furthermore, Go offers high performance without utilizing too many resources.

Today, we will create a simple Go application and set up a CI/CD pipeline for the same. Let's Go!

Prerequisites

• Create a free SingleStore account. For this tutorial, we'll be using SingleStore as our database solution. SingleStore is a high-performance, in-memory database that supports both SQL and NoSQL data models.

• Create a free Harness cloud account to setup CI/CD

• Download & install Go quickly from here

• Kubernetes cluster access from any cloud provider to deploy our application (you can also use Minikube or Kind to create a single node cluster).

• Docker, preferably Docker Desktop

Tutorial

The example repository is accessible here; feel free to fork it or just follow along. I will not go into many details about the application code itself. It is a sample “Hello World” app that prints the text “Hello World” on the local host 8080.

Here is the code for the main.go file:

package main

import (
    "fmt"
    "log"
    "net/http"
)

func homePage(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Home Page")
}

func wsEndpoint(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello World")
}

func setupRoutes() {
    http.HandleFunc("/", homePage)
    http.HandleFunc("/ws", wsEndpoint)
}

func main() {
    fmt.Println("Hello World")
    setupRoutes()
    log.Fatal(http.ListenAndServe(":8080", nil))
}

The application also has a test file main_test.go with simple test case.

package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestHomePage(t *testing.T) {
    req, err := http.NewRequest("GET", "/", nil)
    if err != nil {
        t.Fatal(err)
    }
    rr := httptest.NewRecorder()
    handler := http.HandlerFunc(homePage)
    handler.ServeHTTP(rr, req)
    if status := rr.Code; status != http.StatusOK {
        t.Errorf("handler returned wrong status code: got %v want %v",
            status, http.StatusOK)
    }
    if rr.Body.String() != "Home Page" {
        t.Errorf("handler returned unexpected body: got %v want %v",
            rr.Body.String(), "Home Page")
    }
}

func TestWsEndpoint(t *testing.T) {
    req, err := http.NewRequest("GET", "/ws", nil)
    if err != nil {
        t.Fatal(err)
    }
    rr := httptest.NewRecorder()
    handler := http.HandlerFunc(wsEndpoint)
    handler.ServeHTTP(rr, req)
    if status := rr.Code; status != http.StatusOK {
        t.Errorf("handler returned wrong status code: got %v want %v",
            status, http.StatusOK)
    }
    if rr.Body.String() != "Hello World" {
        t.Errorf("handler returned unexpected body: got %v want %v",
            rr.Body.String(), "Hello World")
    }
}

The Dockerfile you see in the repo will be used to build and push our application as an image to the Docker Hub.

FROM golang:1.16.4-buster AS builder

ARG VERSION=dev

WORKDIR /go/src/app
COPY main.go .
RUN go build -o main -ldflags=-X=main.version=${VERSION} main.go 

FROM debian:buster-slim
COPY --from=builder /go/src/app/main /go/bin/main
ENV PATH="/go/bin:${PATH}"
CMD ["main"]

The next thing is we will build the image and push it to the Docker Hub using the command,

docker buildx build --platform=linux/arm64 --platform=linux/amd64 -t docker.io/<docker hub username>/<image name>:<tag> --push -f ./Dockerfile .

Once the build and push are successful, you can confirm it by going to your Docker Hub account.

We will be deploying our application on a Kubernetes cluster.
You can see the deployment.yaml and service.yaml files in the forked repo, which define the deployment and service to help us deploy and expose our application. At this point, make sure your Kubernetes cluster is up and running.

Our deployment.yaml file is shown below

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-app-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-app
  template:
    metadata:
      labels:
        app: go-app
    spec:
      containers:
      - name: go-app
        image: pavansa/golang-hello-world:latest
        ports:
        - containerPort: 8080
        env:
        - name: PORT
          value: "8080"

The service.yaml file is shown below,

apiVersion: v1
kind: Service
metadata:
  name: go-app-service
spec:
  selector:
    app: go-app
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080
  type: LoadBalancer

It is time to set up a Harness account to do CI/CD. Harness is a modern CI/CD platform with AI/ML capabilities.

Create a free Harness account and your first project. Once you sign-up at Harness, you will be presented with the new CI/CD experience and capabilities.

Add the required connectors, such as Harness Delegate, your GitHub repo, Docker Hub account and secrets. Delegate in Harness is a service/software you need to install/run on the target cluster [Kubernetes cluster in our case] to connect your artifacts, infrastructure, collaboration, verification and other providers with the Harness Manager. When you set up Harness for the first time, you install a Harness Delegate.

Continuous Integration

After signing in at Harness, it will first ask you to create a project.

Invite collaborators to the project if you want.

Select the Continuous Integration module.

Create your first pipeline.

Connect with your source control management like GitHub, where the application code is present.

Next, configure your pipeline with the proper language of the project.

When you save and continue, you see the below default setup in the pipeline studio.

When you click on the 'Go Build App' under execution, you will see the below setup details.

Let's modify the previous/above step, name it as 'Test Go App' add the following code in the command tab, and save and run the pipeline.

You should see a successful output:)

We successfully created a CI pipeline for our application where the build and testing of the code happens. Let's extend the idea of deploying this error-free application code our target environment, i.e Kubernetes.

Continuous Delivery and Deployment

It is time to deploy our Go application, create the deployment stage.

Add a name to your stage, select the deployment type as 'Kubernetes', and click 'Set up Stage'.

This is what you should be seeing after creating the deploy stage.

We need to create a service. Hence, click on 'add service'. In the next step, we need to add a name to our service and manifest details.

Click on 'Add manifest' to add the details.

Then select K8S manifest and continue.

Specify the K8S manifest store. We know that our manifest files are present in GitHub. Hence select GitHub.

Add a new GitHub connector to connect your manifest files.

Specify all the details step by step.

Add credentials through inbuilt secrets.

Connect with your delegate.

Make sure the connection of your manifest is successful with your delegate.

Now, add the manifest details from your GitHub repo.

Save everything and continue.
You should see the service with manifest details.

Your pipeline studio will look like this, with your added service.

Add a new environment, save and continue.

Similarly, add new infrastructure. Select infrastructure type as 'Kubernetes' and add the cluster details.

Save and continue.

In the next step, you need to select the deployment strategy type. We are selecting 'Rolling' as our deployment strategy.

We are all done and this is how our CD pipeline looks like.

Now, save everything and run the pipeline.

You should see a successful pipeline execution starting from CI and then CD step by step.

Automate CI/CD

The last step is to automate our CI/CD pipeline by creating Triggers. Let's do it.

In the pipeline studio, you can see the 'Triggers' tab.

Click the Triggers tab and create a new trigger.

Add the GitHub trigger, whenever someone pushes a new code to the main branch, the pipeline should trigger automatically.

Save everything and create the trigger.

You should see the created trigger under the Trigger tab.

Now, let's confirm if our CI/CD is automated and working properly. Add a readme file to our GitHub repo and see if it triggers our pipeline.

You see, the pipeline triggered when a new code commit happened.

We have successfully automated the CI/CD process for our Go application using Harness.

Also, checkout my other articles on continuous integration and deployment.

A Step-by-Step Guide to Continuous Integration for Your Node.js Application

Pavan Belagatti ・ Feb 2 '23

#javascript #devops #node #tutorial

Deploying an Application on Kubernetes: A Complete Guide!

Pavan Belagatti ・ Feb 13 '23

#kubernetes #tutorial #devops #cicd