In this post, I’ll walk you through setting up an advanced CI/CD pipeline on AWS using native services and show you how to create and configure components like CodeBuild, CodeDeploy, and CodePipeline using AWS CLI commands and configuration files. I will also include outputs for each command so you can see what to expect when you run these commands yourself.
Note:
- Flowchart was created using mermaid.live
- Since CodeCommit is no longer onboarding new customers, choosing GitHub as the source repository. If you already have a GitHub repository for your project, simply update the configuration files accordingly.
Prerequisites: Please make sure that the AWS CLI is installed and configured with the proper IAM permissions.
1. Create a Source Repository in GitHub
First, let’s create a new GitHub repository where our source code will be there.
git init
git remote add origin https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo.git
git add .
git commit -m "Initial commit"
git push -u origin main
2. Set Up a CodeBuild Project
Next let's create a CodeBuild project that builds our application. In this example, we’ll use a simple Node.js project. Save the following JSON configuration as codebuild-project.json:
{
"name": "advanced-ci-cd-build",
"description": "Build project for advanced CI/CD pipeline demo",
"source": {
"type": "GITHUB",
"location": "https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo"
},
"artifacts": {
"type": "NO_ARTIFACTS"
},
"environment": {
"type": "LINUX_CONTAINER",
"image": "aws/codebuild/standard:6.0",
"computeType": "BUILD_GENERAL1_MEDIUM",
"environmentVariables": [
{
"name": "NODE_ENV",
"value": "production"
}
]
},
"serviceRole": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeBuildServiceRole"
}
Create project with:
aws codebuild create-project --cli-input-json file://codebuild-project.json
Output:
{
"project": {
"name": "advanced-ci-cd-build",
"arn": "arn:aws:codebuild:us-east-1:123456789012:project/advanced-ci-cd-build",
"description": "Build project for advanced CI/CD pipeline demo",
"source": {
"type": "GITHUB",
"location": "https://github.com/<YOUR_GITHUB_USERNAME>/advanced-ci-cd-repo"
},
"artifacts": {
"type": "NO_ARTIFACTS"
},
"environment": {
"type": "LINUX_CONTAINER",
"image": "aws/codebuild/standard:6.0",
"computeType": "BUILD_GENERAL1_MEDIUM",
"environmentVariables": [
{
"name": "NODE_ENV",
"value": "production"
}
]
},
"serviceRole": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeBuildServiceRole",
"created": "2024-05-10T15:00:00Z"
}
}
Note: Make sure CodeBuild service role has permissions for GitHub, S3 (if you use artifacts), and CloudWatch logging.
3. Define a Build Specification (buildspec.yml)
Create a buildspec.yml file in the root of the repository. This file instructs CodeBuild on how to run our build.
version: 0.2
phases:
install:
runtime-versions:
nodejs: 16
commands:
- echo "Installing dependencies..."
- npm install
pre_build:
commands:
- echo "Running lint and tests..."
- npm run lint
- npm test
build:
commands:
- echo "Building the project..."
- npm run build
post_build:
commands:
- echo "Build completed on `date`"
artifacts:
files:
- '**/*'
discard-paths: yes
Push this file to the GitHub repository so CodeBuild uses it during the build process.
4. Set Up a CodeDeploy Application and Deployment Group
For blue/green deployments, we’ll use CodeDeploy.
Create a CodeDeploy Application
Command:
aws deploy create-application \
--application-name advanced-ci-cd-app \
--compute-platform Server
Output:
{
"applicationId": "abcdef12-3456-7890-abcd-ef1234567890",
"applicationName": "advanced-ci-cd-app",
"computePlatform": "Server"
}
Create a Deployment Group
Save the following as deployment-group.json (update below values as needed):
{
"applicationName": "advanced-ci-cd-app",
"deploymentGroupName": "advanced-ci-cd-deploy-group",
"deploymentConfigName": "CodeDeployDefault.OneAtATime",
"ec2TagFilters": [
{
"Key": "Name",
"Value": "MyAppServer",
"Type": "KEY_AND_VALUE"
}
],
"serviceRoleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodeDeployServiceRole",
"blueGreenDeploymentConfiguration": {
"terminateBlueInstancesOnDeploymentSuccess": {
"action": "TERMINATE",
"terminationWaitTimeInMinutes": 5
},
"deploymentReadyOption": {
"actionOnTimeout": "CONTINUE_DEPLOYMENT"
}
},
"autoRollbackConfiguration": {
"enabled": true,
"events": ["DEPLOYMENT_FAILURE"]
}
}
Create Deployment group with:
aws deploy create-deployment-group --cli-input-json file://deployment-group.json
Output:
{
"deploymentGroupId": "abcdef12-3456-7890-abcd-ef1234567890",
"deploymentGroupName": "advanced-ci-cd-deploy-group",
"applicationName": "advanced-ci-cd-app"
}
Note: Make sure that the CodeDeploy service role has the necessary permissions to access EC2 and execute deployment actions.
5. Create a CodePipeline
Let's put everything together by creating a pipeline that orchestrates the source, build, and deployment stages.
Save the following as pipeline.json (adjust names, regions, and bucket names as needed):
{
"pipeline": {
"name": "advanced-ci-cd-pipeline",
"roleArn": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/CodePipelineServiceRole",
"artifactStore": {
"type": "S3",
"location": "<YOUR_ARTIFACT_BUCKET>"
},
"stages": [
{
"name": "Source",
"actions": [
{
"name": "SourceAction",
"actionTypeId": {
"category": "Source",
"owner": "ThirdParty",
"provider": "GitHub",
"version": "1"
},
"outputArtifacts": [
{
"name": "SourceOutput"
}
],
"configuration": {
"Owner": "<YOUR_GITHUB_USERNAME>",
"Repo": "advanced-ci-cd-repo",
"Branch": "main",
"OAuthToken": "<YOUR_GITHUB_OAUTH_TOKEN>"
},
"runOrder": 1
}
]
},
{
"name": "Build",
"actions": [
{
"name": "BuildAction",
"actionTypeId": {
"category": "Build",
"owner": "AWS",
"provider": "CodeBuild",
"version": "1"
},
"inputArtifacts": [
{
"name": "SourceOutput"
}
],
"outputArtifacts": [
{
"name": "BuildOutput"
}
],
"configuration": {
"ProjectName": "advanced-ci-cd-build"
},
"runOrder": 1
}
]
},
{
"name": "Deploy",
"actions": [
{
"name": "DeployAction",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "CodeDeploy",
"version": "1"
},
"inputArtifacts": [
{
"name": "BuildOutput"
}
],
"configuration": {
"ApplicationName": "advanced-ci-cd-app",
"DeploymentGroupName": "advanced-ci-cd-deploy-group"
},
"runOrder": 1
}
]
}
],
"version": 1
}
}
Create pipeline with:
aws codepipeline create-pipeline --cli-input-json file://pipeline.json
Output:
{
"pipeline": {
"name": "advanced-ci-cd-pipeline",
"version": 1,
"artifactStore": {
"type": "S3",
"location": "my-artifact-bucket"
},
"stages": [
{
"name": "Source",
"actions": [ { ..... } ]
},
{
"name": "Build",
"actions": [ { ..... } ]
},
{
"name": "Deploy",
"actions": [ { ..... } ]
}
]
},
"created": "2024-05-10T15:30:00Z"
}
6. Testing and Monitoring the Pipeline
After creating the pipeline, trigger a test execution by pushing a commit to your GitHub repository. Then, check the pipeline status with:
aws codepipeline get-pipeline-execution --pipeline-name advanced-ci-cd-pipeline --pipeline-execution-id <execution-id>
Output:
{
"pipelineExecution": {
"pipelineName": "advanced-ci-cd-pipeline",
"pipelineExecutionId": "12345678-1234-1234-1234-123456789012",
"status": "Succeeded",
"artifactRevisions": [ ..... ],
"startTime": "2024-05-10T15:31:00Z",
"lastUpdateTime": "2024-05-10T15:33:00Z"
}
}
Additionally, we can set up CloudWatch dashboards and alarms to monitor key metrics such as build duration and deployment status.
Conclusion
In this blog I demonstrated how we can build an advanced CI/CD pipeline on AWS using CLI commands and configuration files. By integrating GitHub, CodeBuild, CodeDeploy (with blue/green deployments), and CodePipeline, we created a fully automated workflow.
Happy Deploying! :)
Top comments (0)