Basic knowledge on Jenkins and Github Actions is required.
Jenkins has been my DevOps team's CI/CD tool of choice for years, however we recently had to say goodbye and switch to GitHub Action Workflows. During the planning stage of our migration, we discovered a lot of redundancy in the Jenkins pipelines, so we devised a strategy to address this by utilizing the DRY concept.
This blog post explains the application of the "Don't Repeat Yourself" (DRY) software development principle, which helps us minimize the effort needed for the migration.
DRY is a key concept in programming that emphasizes the importance of reducing redundancy. Duplicating code is not just ugly it is bad practice because it would make your code hard to maintain. If you need to fix one part on your code, you have to do it on all other parts where it is repeated and often times that can cause issues whenever the fix is not done properly.
Keeping it DRY
Before writing any GitHub workflow, we took a closer look first into our Jenkins pipelines to identify areas we could apply the DRY principle.
Duplicate Stages Across Multiple Pipelines
Here you can see an example of two separate Jenkins pipelines for deployment that accept the same parameter and have very similar stages.
Pipeline A: DB Deployment
pipeline {
agent any
parameters {
string(name: 'JIRA_TICKET', description: '', defaultValue: '')
}
stages {
stage('Get JIRA Ticket Details') {
steps {
script{
echo 'Fetch Ticket Details from JIRA API'
}
}
}
stage('DB Deployment') {
steps {
script{
echo 'Logging in to DB'
echo 'Running Deployment Script'
echo 'Parsing logs for any ORA errors'
}
}
}
stage('JIRA Update') {
steps {
script{
echo 'Update JIRA Ticket with Deployment Result'
}
}
}
}
}
Pipeline B: App Deployment
pipeline {
agent any
parameters {
string(name: 'JIRA_TICKET', description: '', defaultValue: '')
}
stages {
stage('Get JIRA Ticket Details') {
steps {
script{
echo 'Fetch Ticket Details from JIRA API'
}
}
}
stage('App Deployment') {
steps {
script{
echo 'Logging in to Server'
echo 'Running Deployment Script'
}
}
}
stage('Trigger Post Deploy and HealthChecks') {
steps {
script{
echo 'Running post deploy and healthchecks'
}
}
}
stage('JIRA Update') {
steps {
script{
echo 'Update JIRA Ticket with Deployment Result'
}
}
}
}
}
The first and last stages, Stage Get JIRA Ticket Details and Stage JIRA Update, are similar and are quite repeated in other pipelines since we have a dependency on getting JIRA ticket field values to identify what to deploy and also keep track of deployment status.
Pipelines triggered frequently from other pipelines
We also identified which pipelines are triggered from other pipelines frequently, which are usually parameterized pipelines. Taking the above's example, we also have other pipelines that should trigger the same set of steps in Pipeline A.
The Solution
To avoid duplication when we migrate to Github Workflows, below was our solution:
For similar stages across pipelines → Use Composite Actions
For pipelines triggered frequently from other pipelines → Use Reusable Workflows
In our case, we choose these solutions, which Github already offers, to achieve our goals below:
Simplified Migration
Migrating existing Jenkins pipelines often involves translating Jenkinsfile scripts into Github Action Workflows. Composite actions and reusable workflows help manage complexity by isolating and reusing common logicReusability
Mirrors Jenkins's shared libraries or reusable pipelines, easing the transition.Maintainability
Using shared logic in one place simplifies maintenance and troubleshooting.
Using Composite Actions ✨
Composite actions allow you to turn a step or set of steps into an action and reuse it on multiple GitHub workflows. In Jenkins, this is similar to a Jenkins Shared Library.
Example Composite Action
Here is an example composite action created for the Get JIRA Ticket Details stage above.
This composite action handles getting information for a build given a JIRA Ticket number and importing the information fetched from an API as environment variables which can be used multiple times across other workflows.
On composite-repo repository, under a folder called get-jira an action.yaml like below is created:
name: 'Get JIRA Ticket Details'
description: 'Fetch JIRA Ticket details'
inputs:
jira_ticket:
description: JIRA Ticket
required: true
jira_user:
description: JIRA Username
required: true
jira_pass:
description: JIRA Password
required: true
runs:
using: 'composite'
steps:
- name: Get JIRA Ticket Details
id: get_jira
shell: bash
run: |
curl -u ${{ inputs.jira_user }}:${{ inputs.jira_pass }} \
"$jira_api/${{ inputs.jira_ticket }}" > jira.json
summary=$(cat jira.json | jq -r '.fields.summary')
status=$(cat jira.json | jq -r '.fields.status')
artifacts=$(cat jira.json | jq -r '.fields.artifact')
echo "SUMMARY=$summary" >> $GITHUB_ENV
echo "STATUS=$status" >> $GITHUB_ENV
echo "ARTIFACTS=$artifacts" >> $GITHUB_ENV
Here’s how the composite action above is used on a workflow. Note that get-jira is the name of the action and v1 is the branch name or tag on composite-repo repository.
name: Deployment Pipeline
on:
workflow_dispatch:
inputs:
JIRA_TICKET:
description: JIRA Ticket number
required: true
type: string
jobs:
Deploy:
runs-on: ubuntu-latest
steps:
- name: Get JIRA Ticket Fields
uses: composite-repo/get-jira@v1
with:
jira_ticket: ${{ inputs.JIRA_TICKET }}
jira_user: ${{ secrets.JIRA_USER }}
jira_pass: ${{ secrets.JIRA_PASS }}
Now that we have this composite action, we can add and make changes to this and all other workflows which uses this are automatically updated.
We also made other actions we can reuse aside from this one and and we ended up having like a tool box full of tools we can just use on any of our workflows. 🔧 🔨
Lessons Learned when creating Composite Actions 🎉
- A mono repo containing commonly used composite actions with the below structure can be created.
|-action-1
| |-action.yaml
|-action-2
| |-action.yaml
Use branches or tags to version the composite actions for easy maintenance and testing.
Avoid combining very different sets of steps into the same composite action, as it will become complicated to use and may require multiple optional parameters and conditions.
Document actions for easy use. Tools like
npalm/action-docsCLI can be used to automate the creation of documentation.
Using Reusable Workflows ✨
For Jenkins pipelines triggered frequently from other pipelines, we created reusable workflows.
Unlike Composite actions which can be run on steps, reusable workflows are run directly as jobs and can even use the matrix strategy option to trigger it multiple times.
In our case we have one main workflow which triggers multiple DB Deployment workflows for each JIRA ticket and environment, so we created a reusable workflow for that purpose.
Example Reusable Workflow: db_deploy.yaml
name: DB Deployment
on:
workflow_call:
inputs:
JIRA_TICKET:
description: Jira Ticket
required: true
type: string
ENVIRONMENT:
description: Environment
required: true
type: string
jobs:
Oracle-Deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Get JIRA Ticket Fields
uses: composite-repo/get-jira@v1
with:
jira_ticket: ${{ inputs.JIRA_TICKET }}
jira_user: ${{ secrets.JIRA_USER }}
jira_pass: ${{ secrets.JIRA_PASS }}
- name: Get DB Details
id: get-db
uses: composite-repo/get-db-details@v1
with:
environment: ${{ inputs.ENVIRONMENT }}
- name: DB Deployment
uses: composite-repo/db-deploy@v1
with:
dbhost: ${{ steps.get-db.outputs.dbhost }}
dbport: ${{ steps.get-db.outputs.dbport }}
dbuser: ${{ secrets.DB_USER }}
dbpass: ${{ secrets.DB_PASS }}
dbscripts: ${{ env.ARTIFACTS }}
Here’s how the reusable workflow above is used on a workflow. Since we have multiple deployments for multiple JIRA tickets, a reusable workflow is used on a job with matrix strategy.
jobs:
Oracle_Deploy:
strategy:
matrix:
jira: ${{ fromJSON(needs.jobname.outputs.tickets) }}
uses: workflows-repo/.github/workflows/db_deploy.yaml@v1
with:
JIRA_TICKET: ${{ matrix.jira.ticket }}
ENVIRONMENT: ${{ matrix.jira.environment }}
Lessons Learned when creating Reusable Workflows: 🎉
Reusable workflows and composite actions are quite similar. Use reusable workflows to reuse an entire workflow with multiple jobs and steps, and use composite actions for small tasks that can be reused in a step.
Too much reuse can lead to complications. Package related steps and jobs only into one reusable workflow and, for other purposes, create a separate one.
Reusing of workflows is allowed up to 4 levels only. If your main workflow does not need to wait for the triggered reusable workflow to complete, you can use the github CLI
gh workflow run <workflow name>on a step to trigger it using theworkflow_dispatchevent instead.
Conclusion
Composite Actions and Reusable Workflows can be used to avoid duplication and ease the migration from Jenkins to Github Actions.
Top comments (1)
Wow, Pat! This is really helpful! Kudos for making efficiency look so effortless! 🙌🏻