DEV Community

Tommy Heng-Chuan Lin
Tommy Heng-Chuan Lin

Posted on • Updated on

Pipeline example using Terraform, Jenkins, Ansible, and Docker on Google Cloud-Part 1

#devops #beginners #googlecloud #tutorial

Project brief intro:

This project I made is for the demonstration of pipelines. I utilised Terraform Cloud provisioning infras on GCP, using Ansible to configure remote VM instances for initiation and deployment, Docker image as artifact for deployment and finally, a Jenkins server integrating webhook from GitHub and further CICD process.

This project was split into two articles for the sack of large context, I tried to template the source code so that it should be easy to follow stepwise. The first part would be the prerequisite walkthrough, provisioning infra and initialising Jenkins on CICD server.

Workflow

Image description

Prerequisite

  • OS: Linux/MacOS
  • Local installation of Terraform and Ansible
  • Google Cloud Platform Account
  • Terraform Cloud Account
  • GCP CLI

Enable GCP APIs

  • Compute Engine API
  • Google Container Registry API
  • Google Cloud Storage API

Create Organization and Workspace on Terraform Cloud

Terraform | HashiCorp Cloud Platform

Create a new GCP project

go to GCP console then create a new project and have your gcloud cli configured to that new project

Google Console Page

Clone Pipeline Project repo

Pipeline Project Links



git clone https://github.com/aks60808/Devops-Aigames
Enter fullscreen mode Exit fullscreen mode

Configure GCP

To make terraform cloud provisioning infra from GCP, the following are required to provide:

  • Google Service Account
  • Grant permission for the role
  • Create a Service Account Key for further authentication

Login to GCP first 



gcloud auth login
Enter fullscreen mode Exit fullscreen mode

Create Service Account

replacing SA_NAME with your preferred Service Account name.



gcloud iam service-accounts create SA_NAME \
    --description="DESCRIPTION" \
    --display-name="DISPLAY_NAME"
Enter fullscreen mode Exit fullscreen mode

Grant IAM as an editor role

replacing PROJECT_ID with your PROJECT_ID on gcp console



gcloud projects add-iam-policy-binding PROJECT_ID \
    --member="serviceAccount:SA_NAME@PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/editor"
Enter fullscreen mode Exit fullscreen mode

Create a service account key

It’s up to you to determine the location of the service account key, which typically would be stored in ~/.ssh/ directory.



gcloud iam service-accounts keys create YOUR_SA_KEY_PATH \
    --iam-account=SA_NAME@PROJECT_ID.iam.gserviceaccount.com
Enter fullscreen mode Exit fullscreen mode

Configure Terraform to Terraform Cloud

in these .tf files, they will provision several resources including

  • Google Compute Instance *2 (e2.medium) for service hosting
  • Google VPC and corresponding firewall rules allowing http and ssh
  • Google Container registry for docker image push in Part 2.
  • Google Cloud Storage for backing up Jenkins in Part 2.

Frist, Let's get into the project directory:



cd /Devops-Aigames
Enter fullscreen mode Exit fullscreen mode

Modify the following main.tf



# store terraform states on terraform cloud
terraform {
  cloud {
    # replace with your organization
    organization = "REPLACE WITH YOUR ORGANIZATION NAME"

    workspaces {
    # replace with your workspace name
      name = "REPLACE WITH YOUR WORKSPACE NAME"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

create a file named terraform.tfvars in project directory



touch terraform.tfvars
Enter fullscreen mode Exit fullscreen mode

add the following content to terraform.tfvars
Regions & their corresponding zones can be referred to Google compute Regions-Zones List



project_id = "REPLACE WITH　YOUR　GCP　PROJECT　ID"
region = "REPLACE WITH　YOUR　PREFERRED　REGION"
zone = "REPLACE WITH　YOUR　PREFERRED　ZONE"

# these are variables that will be used in the terraform configuration
Enter fullscreen mode Exit fullscreen mode

Add your GCP service account key to Terraform Cloud

Go to the Terraform Cloud service, select your right organization/workpace, click Variables on the left Menu Bar

Menu Bar on terraform cloud

  1. Click Add variable button
  2. Select Terraform variable
  3. In KEY section, you MUST type in GOOGLE_CREDENTIALS
  4. COPY and PASTE all content of the Service Account key(JSON file) into the Value section.
  5. Don’t forget to tick Sensitive and click Save

The place you should put your SA key

For the reasoning of the above configuration, please refer to Terraform documentation

GCP credential configuration

Terraform Registry

Login to Terraform Cloud

run terraform login and have the token value pasted into terminal



terraform login
Enter fullscreen mode Exit fullscreen mode

Initial your local terraform to terraform cloud

run the following command to get terraform connected to terraform cloud 



terraform init
Enter fullscreen mode Exit fullscreen mode

Start Provisioning your infra on GCP using terraform 



terraform plan
terraform apply
terraform output # to see your VM ip
Enter fullscreen mode Exit fullscreen mode

Ansible Configure management

Now we will utilise ansible to configure the remote CICD server

Generate ssh key if needed 



# USERNAME should match your current username
ssh-keygen -t rsa -f ~/.ssh/KEY_FILENAME -C USERNAME -b 2048
Enter fullscreen mode Exit fullscreen mode

Visit GCE instance page click metadata>> ssh keys>>add ssh key

GCE metadata page

copy your generated .pub key then saved.

Create an inventory file in project directory 



touch inventory
Enter fullscreen mode Exit fullscreen mode

put the following content into inventory file



[cicd]
x.x.x.x 
# replace above with CICD server IP address
[cicd:vars]
ansible_ssh_private_key_file= /path/to/your/private/ssh/key
[depolyment]
x.x.x.x 
# replace above with depolyment server IP address
[depolyment:vars]
ansible_ssh_private_key_file= /path/to/your/private/ssh/key
Enter fullscreen mode Exit fullscreen mode

Configure ansible.cfg file

check the location of ansible.cfg by running the following:



ansible-config --version
Enter fullscreen mode Exit fullscreen mode

if you saw the config file = None, then in current directory, run



ansible-config init --disabled > ansible.cfg
Enter fullscreen mode Exit fullscreen mode

modify the ansible.cfg by setting:



host_key_checking=False
Enter fullscreen mode Exit fullscreen mode

then saved.

Try reaching out CICD and Deployment server via ansible

run the following command to test the ssh connectivity of two hosts.



# -i flag specify your inventory path
ansible all -m ping -i ./inventory
Enter fullscreen mode Exit fullscreen mode

you should see the following output:

Ping success

Configure CICD server using Ansible

before start configuring, go to ansible_playbook/ then open

ansible_deployment.yml to modify the project_id, replace it with yours.



- name: deploy app to server
  hosts: all
  become: yes
  vars:
    project_id: YOUR_GCP_PROJECT_ID
Enter fullscreen mode Exit fullscreen mode

cicd_server_init_config.yml to modify the service account key and ansible_deployment.yml playbook location(ansible_deployment.yml is in project dir/ansible_playbooks)

the Path should be in absolute path format, key_dest and playbook_dest are set to jenkins home directory, so you don't need to modify them.



- name: Activate Service account
  hosts: cicd
  become: yes
  vars:
    key_src: YOUR/LOCAL/ABSOLUTE/PATH/TO/sa_key.json 
    key_dest: /var/lib/jenkins/key.json

......

- name: Upload playbook to CICD server
  hosts: cicd
  become: yes
  vars:
    playbook_src: YOUR/LOCAL/ABSOLUTE/PATH/TO/ansible_deployment.yml
    playbook_dest: /var/lib/jenkins/ansible_deployment.yml
Enter fullscreen mode Exit fullscreen mode

the ansible playbook will

  • pass your service account key to CICD server
  • install Docker, Jenkins, and Ansible
  • Configure what’s necessary for running the operations including adding jenkins to docker group and cgroup mounting
  • pass deployment playbook to CICD server

in the project directory, 



ansible-playbook -i inventory \
                    ansible_playbook/cicd_server_init_config.yml
Enter fullscreen mode Exit fullscreen mode

After completion, you should open your browser via
Jenkins initialisation
PS. Initial ADMIN PWD will can be found either in the file location or the stdout on your terminal

Image description

Cleanup

DO NOT CLEAN UP UNLESS YOU WANT TO SKIP PART II!

If you wish to terminate the whole setup, in the project directory run the following command:



terraform destroy
Enter fullscreen mode Exit fullscreen mode

DON'T FORGET TO ENTER YES TO PROCEED!

To be continued

In the next article, I will continue building Jenkins CICD pipeline to build, test, package and finally deploy for hosting a website offering algorithm-based fun games written by my good friend, Noah(big shoutout to you:)).

Part 2 article link

Top comments (0)

Read next

monarene profile image

Designing a fault-tolerant etcd cluster

Michael Mekuleyi -

mohsenkamrani profile image

Master Cron Expressions In 2 minutes

Sam -

mrcaption49 profile image

Terraform - Infrastructure as a Code

Pranav Bakare -

petrussola profile image

Install Simple Analytics in an Astro js project

Pere Sola -