Written by Piotr Popiolek
Going through a production example how to use custom GitHub Action with AWS and other technologies
Introduction
I recently performed a really interesting task for one of our customers that combined several different technologies (GitHub Actions/ AWS/ CDK/ Artifactory/ Node.js). I would like to share it as it is quite a good example of a practical exercise.
The main goal of this project was to rotate the artifactory token daily (every 24 hours) - the customer had a solution in place, but it was outdated and relied on old infrastructure. And just to illustrate the case for someone who does not use an artifactory, it is a centralized solution for keeping and managing all the artifacts, packages, files, images for use in organization or project. To authenticate into the artifactory, we usually provide a username with a token, which should be rotated due to security reasons.
To keep up with customer demands, we had to leverage a new CI/CD tool - GitHub Actions. And because of the format of the task, as adjustments were needed in a few steps, a custom action seemed to be a perfect solution.
In this post we will go through the basics of GitHub Actions, possible connections to AWS, and try to create a custom action.
GitHub Actions - basics
If I had to generally explain what GitHubActions is I would say:
It is an interesting (quite new) CI/CD tool that can be a great fit for companies, which stick to GitHub and do not want to leverage external tools that have their code on GitHub. It is not very expensive (pricing) and is really fast (because of native integration with GitHub). It can work with external workers, which are called self-hosted runners (they can be configured on Cloud or OnPrem); or jobs can be run on managed runners provided by GitHub (with resources 2xCPU, 7xGBs, 14GBs storage) with all common dev packages installed.
Default runners are more than enough for a classic CI/CD workload. What is really worth mentioning is that the response time (start the job) is almost immediate, even if we use the managed runners (for example K8's pods).
Let's not get into deep details about how to write the pipelines and its structure as it is pretty common and well documented (about workflows). Instead, let's just see what a common pipeline format looks like. We will use this template later on to start our custom action.
name: "This-is-a-test-pipeline"
on:
push: #this is info when we want to start it
jobs:
rotateaction: #name of the job
runs-on: ubuntu-latest #it will be run on manage_github_runner
permissions:
id-token: write # needed to interact with GitHub's OIDC
contents: read
steps:
- name: checkoutrepo
uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@master
with:
role-to-assume: arn:aws:iam::${{ env.AWSAccountID}}:role/gha-role
aws-region: eu-west-1
- name: UseOurAction
uses: ./
with:
username: "test_user"
gha_token: ${{ gha_token }}
secret_name: "TEST_SECRET"
In the structure we can see that it's really similar to other styles (like Jenkins). There are jobs, steps, variables and how the pipeline will be triggered. A specific usage for GitHub Actions is "uses." Thanks to that we can leverage existing patterns/actions, which can be used within our jobs. In the first stage "checkoutrepo" we use actions/checkout@v3 which basically does checkout code repository. Existing actions that can be adopted in our pipelines can be viewed here (with all requirements and examples needed for implementation - like Java setup or docker build/push, etc). Of course, we also can write our own action, which we will do in the next steps.
How to connect to AWS?
It gets more interesting when we talk about establishing an AWS connection with GitHub Actions. We could stick to the standard approach and use secrets for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, but the suggested way would be to use an OIDC provider. Thanks to this approach we can leverage the GitHub Identity Provider, create an IAM role on AWS with a specific policy and even set up a duration of the session.
A convenient way of keeping the OIDC as IaaC is to leverage AWS Cloud Development Kit (AWS CDK). AWS CDK is an open-source software development framework to define cloud application resources using familiar programming languages. It is increasingly replacing the use of classic CloudFormation templates.
Thanks to the existing library "aws-cdk-github-oidc" it's really easy to write a construct for a GitHub provider, and the same goes for an IAM role, which can be easily created by aws-cdk-lib/aws-iam. Below there is a sample written in TypeScript.
import { Construct } from 'constructs';
import { GithubActionsRole, GithubActionsIdentityProvider } from 'aws-cdk-github-oidc';
import { ManagedPolicy } from 'aws-cdk-lib/aws-iam';
export class oidc{
constructor(scope: Construct, rolename: string, repo: string, provider: GithubActionsIdentityProvider) {
const accessSSMRole = new GithubActionsRole(scope, rolename, {
provider: provider, // reference into the OIDC provider
owner: 'Test_owner', // your repository owner (organization or user) name
repo: repo, // your repository name (without the owner name)
roleName: gha-role
});
accessSSMRole.addManagedPolicy(ManagedPolicy.fromAwsManagedPolicyName('PolicyNameForTheRole'));
}
}
After the connection to AWS is established, we can move to the custom action.
Let's create a custom action!
The best practice when it comes to creating custom actions is to create them per repo. It allows us to keep release/versioning per action and access to the code is always clear.
When it comes to format and requirements - in the action repository we need to have an action.yml file with the structure as below.
name: OurCustomAction
description: Description of our action
inputs:
username:
required: true
description: Artifactory user to change
gha_token:
required: true
description: Private GitHub Token
secret_name:
required: true
description: Secret name to change
runs:
using: 'node16'
main: 'dist/index.js'
In the inputs section, we have to specify the parameters that are needed to start our action and those variables will be used in the core code.
There are three types of the runs section. We can use JavaScript, Docker or Composite Actions. JavaScript is a great fit as the time for building containers is excluded here and we can immediately access the repo file level (to create/modify files structure) instead of copying it over in docker case. The third option allows us to combine written actions in one place. It's a great fit for reusable workflows/repeated tasks.
After the action file is in place, we can define a draft for index.js - at the beginning we can determine input variables and used libraries as below.
import fetch from 'node-fetch';
import { Headers } from 'node-fetch';
import { Octokit } from 'octokit';
import libsodium from "libsodium-wrappers";
import * as core from '@actions/core'
var AWS = require('aws-sdk');
var ssm = new AWS.SSM({region: 'eu-west-1'});
const username = core.getInput('username')
const gha_token = core.getInput('gha_token');
const secret_name = core.getInput('secret_name');
Thanks to core.getInput we can pass the variables which will be provided by end-users to our action.
In the gha_token we need to provide the GitHub token with sufficient permissions to perform the specific action. We can use a private token, but the preferred way is to leverage the GitHub apps permission. This article perfectly describes in detail all the ways we have around GitHub Actions authentication.
After that we will need three more steps
- Regenerate token for provided username
- Put the re-generated token to AWS SSM
- Put the re-generated token to GitHub Secret
Do not worry about the first step as it depends on the artifactory provider and is not really important in this story. Let's assume we have built a regenerateToken function and it does a job and a new generated-token is called token.
For AWS we would need an updateSSM function, which will pass a changed token to the proper SSM secure parameter. This step was needed to follow customer policy.
function updateSSM(parameter, token){
var params = {
Name: parameter + '/token,
Value: token,
Tier: "Standard",
Type: "SecureString",
Overwrite: true
};
ssm.putParameter(params, function(err, data) {
if (err) console.log(err, err.stack);
else console.log(data);
});
}
And the last step in our code is to create a function to put a new token in the GitHub Actions secret. Why do we do that? Because this token can be easily used in GitHub Actions workflows to access the artifactory for specific projects
GitHub has a really well-documented process of the api (example) and we can use octokit to write our function.
async function createGithubSecret(token) {
const octokit = new Octokit({
auth: gha_token
})
const { data: publicKeyData } = await octokit.request('GET /orgs/<ourOrg>/actions/secrets/public-key')
const publicKey = publicKeyData.key;
const publicKeyId = publicKeyData.key_id;
const keyBytes = Buffer.from(publicKey, 'base64');
const messageBytes = Buffer.from(token);
const encryptedBytes = libsodium.crypto_box_seal(messageBytes, keyBytes);
const encrypted = Buffer.from(encryptedBytes).toString('base64');
await octokit.request('PUT /orgs/{owner}/actions/secrets/{secret_name}', {
owner: <ourOrg>,
secret_name: secret_name,
encrypted_value: encrypted,
key_id: publicKeyId,
})
.catch(error => {
console.error(error);
})
}
Please note that we have to encrypt our secret with the libsodium library before we can update it (this process is also described in the GitHub documentation).
After we call our functions, we are ready to use written custom action.
name: "Use-Our-Custom-Action"
on:
schedule:
- cron: '0 4 */2 * *'
jobs:
rotateaction:
runs-on: ubuntu-latest
permissions:
id-token: write # needed to interact with GitHub's OIDC Token endpoint.
contents: read
steps:
- name: checkoutrepo
uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@master
with:
role-to-assume: arn:aws:iam::${{ env.AWSAccountID}}:role/gha-role
aws-region: eu-west-1
- name: use action
uses: /path/to/action -> /<org>/<repo>@v1 or /<org>/<repo>/@branch
with:
username: "test_user"
gha_token: ${{ gha_token }}
secret_name: "ROTATION_TEST_SECRET"
As we can see in the above example, we can reference the action by providing the full path to the action repository. We can even call it from the branch or specific release, and do not need to call the main branch. We can also see that the action has been scheduled by cron and is started every second day at 4 UTC. We also need to provide three parameters which we marked in the action.yaml file as required.
And with this we have reached the end of our short journey. Hope it was at least a little interesting and you want to see how custom actions work on your own!
Top comments (0)