TECH STACK: AWS | CodeCommit | CodeArtifact | CodeBuild | CodeDeploy | CodePipeline | GitHub| EC2 | S3 | IAM.
Setting up CI/CD can be a tedious task and takes a lot of time to create a pipeline that runs perfectly, handle every edge case, and compiling it all together. We set up our codebase on platforms like GitHub, GitLab, JIRA, etc., and platforms like Jenkins to set up CI/CD. Breaking up the whole pipeline into multiple platforms is not an easy task to manage.
AWS provides the complete solution for the creation and managing of the complete CICD pipeline in AWS itself. AWS provides CodePipe to create pipelines, CodeComit to manage code base, CodeBuild for building the code, and CodeDeploy for deploying the code into the various environments. Even it supports Jenkins if you want to use that. It supported server or serverless type of deployment. A lot of possibilities and a lot of learning.
APPROACH
Create a codebase in CodeComit and create a complete CICD pipeline that will read the source code with each push activity, build the application and finally deploy the application. The entire process should be automatic.
Prerequisite:
Have an AWS account with proper access.
And, most important this blog needs your time. Before implementing it, I request you to read it completely.
PS:
For the experiment, I am using a convention of using the name “BLOG-CICD”.
Don’t forget to delete all the resources once you are done with the experiment otherwise, it will incur charges.
Steps required for creating the pipeline
Step 1: Add credentials.
a: Generate credentials.
b: Add permission.
Step 2: Create an EC2 machine for the deployment.
Step 3: Create IAM Role for CodeDeploy.
Step 4: Create Repository.
Step 5: Add application file to Repository.
Step 6: Configure CodeBuild.
Step 7: Configure CodeDeploy.
Step 8: Create CodePipeline.
Step 9: Troubleshooting
Step 10: Open port for the application.
Now, it's time to start hands-on. Happy Clouding!!!
Step 1
First, we need to create CodeCommit credentials so that users can access the repository and also, need to permit the same.
Generate code commit credentials for the IAM user:
Go to the IAM console.
Click on the user name to whom you want to give permissions.
Click on
Security Credentials
Scroll down and go to HTTPS
Git credentials for AWS CodeCommit
section.Click on
Generate Credentials
and it will create code commit credentials for that IAM user.
So, for now, we have created CodeCommit credentials for the IAM user. Next, we need to attach Code commit policy to that user.
Go to the IAM console.
Click on the user for whom you created code commit credentials earlier.
Under the Permissions tab, go to add policy.
Search for
AWSCodeCommitPowerUser
policy and add it.
Step 2
As shown in the architecture diagram, we are deploying the application on the EC2 server. So, we need to create an EC2 server for the same.
Go to the EC2 console.
Click on Lunch instance.
Search for Linux as shown in the diagram and click on select.
Select instance type here.
Remember: Big instances incur more cost. For this experiment t2.micro is more than sufficient which comes under free tier service.
In the next step, we can configure our instance. No need to go into more details here. Keep everything default and click on the
next
Add Tag, this is a very important step. With the help of this tag, CodeBuild will identify the deployment instance. We will use it later. I have used name tag only to keep it simple as shown in the diagram below and click
next
.
For now, keep it as it is for making it simple and click on
next
.
Next, create or use the existing key pair for your instance and
launch instance
.
Server creation is done. happy 😄!
Step 3: Create an IAM Role for AWS CodeDeploy allowing CodeDeploy to call other AWS services
Go to the IAM console.
Click on Role left side.
Click on Create Role.
Select CodeDeploy as AWS service.
Scroll down and select CodeDeploy as your use case.
Click on
Next: Tags
Click on
Next: Review
Give
Role name
Click on
Create role
Step 4: Create a Repository in AWS for storing applications(or, you can create a repository on GitHub also)
From the AWS console go to the
CodeCommit
service.Give Repository name.
Optional: enable Amazon CodeGuru to validate the quality of code.
Copy the repository URL and clone the repository to your local system. Use the credential generated above for cloning.
Step 5: Add application source file to this repository
Add application source code to this repo and push it. you may refer to the application here:
Source code contains two important files
buildspec.yaml
and appspec.yaml
. buildspec.yaml
file contains the steps for creating the production build which will be stored in Code Artifact. appspec.yaml
has the steps to set up the environment and start the production server on the EC2 instance. Confuse😇: Don’t worry, we will discuss YAML files in the later section.
Step 6: Configure CodeBuild
Click on
Getting started
under Code build.Click on create a project.
Give a project name.
Select
Source provider
asAWS CodeCommit
under the source section.Select your repository.
Select
Reference type
as a branch.-
Select Environment image under Environment section
- We can use AWS-managed OS images or custom Docker images.
- For this pipeline, I am using AWS Managed image.
-
Select your Operating system.
- NOTE: Make sure it is the same as mentioned in appspec.yaml file which we will use in the Code Deploy step.
- Here, I am using
Amazon Linux 2
.
Select Runtime as Standard.
Select Image. In my case, I am using
aws/codebuild/amazonlinux2-x86_64-standard:3.0
Click on the radio button of New Service Role, it will create a new role for you with the required permissions.
Now, come to the build spec section.
- Here, we have 2 options, either we can put buildspec.yaml file in your code source or else we can directly define the pipeline configuration in this console.
Note: Buildspec file is a collection of build commands and related settings, in YAML format.
Sample buildspec.yaml
file:
# Do not change version. This is the version of aws buildspec, not the version of your buildspec file.
version: 0.2
phases:
pre_build:
commands:
#installs dependencies
- "Commands for Installing Dependencies" ## E.g. npm install / pip install -r requirements.txt / etc.
build:
commands:
- echo Production Build started
- "Build Commands here" ## E.g. npm run build / python setup.py / etc.
post_build:
commands:
- echo Build completed
# Include build files required for your application to run.
artifacts:
files:
- "file names here" ## E.g. dist/*, appspec.yaml (make sure to include this one), build/*, etc.
Here’s a sample appspec.yaml file which will be used to set up the deployment environment and start the application :
# This is an appspec.yml template file for use with an EC2/On-Premises deployment in CodeDeploy.
version: 0.0
# Specify "os: linux" if this revision targets Amazon Linux
os: linux
# Place artifacts file inside EC2 instance
files:
- source: /
destination: /home/ec2-user/server
permissions:
- object: /
pattern: "**"
owner: ec2-user
group: ec2-user
hooks:
# During the BeforeInstall deployment lifecycle event, run the commands
# in the script specified in "location" which is under destination path.
BeforeInstall:
- location: relative_path_inside_your_project_folder/before_install.sh
timeout: 1000
runas: root
# During the AfterInstall deployment lifecycle event, run the commands
# in the script specified in "location".
AfterInstall:
- location: relative_path_inside_your_project_folder/after_install.sh
timeout: 1600
runas: root
# During the ApplicationStart deployment lifecycle event, run the commands
# in the script specified in "location".
ApplicationStart:
- location: relative_path_inside_your_project_folder/app_start.sh
timeout: 300
runas: root
In this deployment, I am going with the buildspec file. So, for this, I have already saved my buildspec.yaml file in my source code. If you are moving with the
insert build commands option
, just insert the necessary command here and click on Continue to pipeline.Leave the Batch configuration and Artifacts section as it is.
You may enable logs, it will be very helpful in resolving the bugs if something wrong happens. But this is optional.
Step 7: Configure CodeDeploy
Click on
Getting Started
of CodeDeploy.Click on
Create application
.Give application name.
Select Compute platform for your pipeline. I am deploying the application in EC2. So, I am going with
EC2/On-premises
option. If you want to deploy your application on another platform select accordingly.
Click on
Create application
.Now we need to create a deployment group.
For the deployment group, we will use the EC2 instance created in the earlier step.
Click on the Deployment group.
Give Deployment Name.
Select the service role you created earlier.
Select Deployment type as In-place.
In the environment configuration, select
Amazon EC2 instances
.Here, we need to search for the instance/instances based on the tag on which you want to deploy the application. In my case, I am using a name tag for searching. Once you give your tag you can see the number of matched instances.
Coming to section
Agent configuration with AWS Systems Manager
. We need a code deploy agent to be configured in your EC2 machine. Either we can configure it manually. But why should we do it manually? AWS is there to help us. So, under the sectionAgent configuration with AWS Systems Manage
, selectNow and schedule updates
.Select the period for the code deploy agent to update. So, after the given period CodeDeploy agent will be automatically updated.
Select
CodeDeployDefault.AllAtOnce
in the deployment setting.
Skip load balancer for this pipeline.
Finally, click on the
Create Deployment Group
Step 8: It’s time to create a pipeline
Click on
Getting Started
under CodePipeline.Click on
Create pipeline
.Give pipeline name.
It is better to create a new service role. Select a new service role. AWS will create a role with the required permissions.
In the next step add source provider as AWS CodeCommit.
Select your Repository name from the drop-down menu.
Select branch name as well from the drop-down menu.
Go with the recommended Amazon Cloudwatch Events as change detection options.
Select CodePipeline default as the output artifact format.
Choose AWS CodeBuild as a build provider.
Cross-check your region here.
Select your Project created earlier.
Select build type as Single build.
Now come to Add deploy stage.
Select Deploy provider as AWS CodeDeploy.
Verify your region as well.
Select your application from the drop-down menu.
Select the deployment group created earlier.
Finally, Review your Pipeline and click on
Create pipeline
.
Go to the pipeline, you can see your pipeline is running. Wait for the job to complete. It will take some time. If your job is completed successfully. Cheers 🎉. You are done.
Step 9: (Optional): Troubleshooting
If you have completed the above steps successfully you are a lucky guy. If your jobs failed. Don’t worry. Try troubleshooting with the following steps.
Troubleshooting 1:
Configure Code deploy agent manually to your EC2 instance. Check the steps below how to do?
-
Run the below command one by one.
REMEMBER: I am using a Linux machine. If you are using a different OS, please follow this:install code deploy agent- ssh into your server.
sudo yum update
sudo yum install ruby
sudo yum install wget
cd /home/ec2-user
-
Download code deploy agent with the below command.
wget https://<bucket-name>.s3.<region>-identifier.amazonaws.com/latest/install
- MAKE SURE to replace the bucket name and region name as mentioned in the below document as per your region.
- https://docs.aws.amazon.com/codedeploy/latest/userguide/resource-kit.html#resource-kit-bucket-names
-
Run the installation script by the below command
chmod +x ./install
-
Check status of Code deploy agent. You will get status as running.
sudo service codedeploy-agent status
-
If not running, make it running by the below command.
sudo service codedeploy-agent status
Trigger the pipeline again and wait for the job to be complete.
Troubleshooting 2:
-
ssh into the server and check the logs with the below command and take action properly.
tail -f /var/log/aws/codedeploy-agent/codedeploy-agent.log
Troubleshooting 3:
- Make sure you have added permission for CodeDeploy, EC2, and S3 to the IAM role used in deployment.
Once your pipeline is executed successfully, we will come to see the below screen in your pipeline.
Step 10
Open the port on which your application is running. For this application, I am using port 3000.
How to open: Check security steps used in EC2 creation in the above section.
Once the port is open, we can now access the application with server DNS. ENJOY 😄
You have successfully set up the complete CI-CD pipeline. Hope you enjoyed the whole journey of creating the pipeline 🎉 !!!!
Reference Title Image: https://exlskills.com/
Top comments (0)