Use Secret Environment Variables in GitHub Actions

Introduction

Earlier this year, my development team and I were in the midst of building a new open source application to help our users get up and running faster with the hardware the Internet of Things (IoT) company I work for, sells.

I work for an IoT startup called Blues Wireless, whose mission is to make IoT development simpler and easier through the use of Notecards - prepaid cellular devices that can be embedded into any IoT project to transmit sensor data to a secure cloud: Notehub - that can then forward that data on to any public cloud or application.

The idea is that once our users set up their new IoT devices, they can fork our starter repo, which is already designed to accept and display certain sensor data in charts and graphs, and then modify the code as they customize their devices further with new sensors or data visualizations.

Keeping ease of setup and deployment in mind, we built the app using the self-contained Next.js framework, hosted a demo example of the app on Netlify, and used GitHub Actions to control the automated CI/CD workflows (build, test, deploy, etc.).

One tricky thing I encountered when working with GitHub Actions was how to add sensitive environment variables that were used locally to run the app and its integration and end-to-end tests, while still keeping them secret (and out of GitHub's source control).

In this article, I'll teach you how to add "secret" environment variables to a GitHub Actions workflow so steps requiring dynamic, sensitive variables (like automated test suites) can successfully run in the CI/CD pipeline just as they do in a local development environment.

What is GitHub Actions?

Let's step back for just a minute to talk about GitHub Actions in case you haven't had a need to work with them yet.

Back in 2018, GitHub released a new feature called GitHub Actions, designed to give developers the ability to automate their workflows across issues, pull requests, and other events.

In 2019 it introduced full continuous integration / continuous deployment (CI/CD) to GitHub Actions as well.

These tools make it easier to automate how developers build, test, and deploy projects on any platform without the need for a devops expert on a development team. Among the advantage GH Actions has over other CI/CD tools is that if a repo is already in GitHub (which it probably is), it's very straightforward to set up workflows to handle all of these sorts of tasks (and more) in an automated fashion.

If you've ever used another CI/CD platform like Jenkins, CircleCI, TeamCity, or a handful of other tools, GitHub Actions should feel similar to you once you start working with it.

Here's a sample diagram of what a simple CI/CD pipeline might look like.

Continuous integration diagram courtesy of GitHub Resources: CI/CD

Going into all the specifics of how GH Actions works is beyond the scope of this blog, but hopefully I've given you a good enough primer to understand the material we'll cover now.

In the Repo, Set the Environment Variables for GH Actions

The place to begin setting environment variables for GH Actions to use is inside the repo itself.

Identify the Local Environment Variables to Replicate in GitHub

In our particular repo, we have four dynamic environment variables that the app (and its test suites) require to run: HUB_BASE_URL, HUB_AUTH_TOKEN, HUB_PROJECT_UID, and HUB_PRODUCT_UID,

All of these variables relate to unique project values that are discoverable in Notehub, the cloud application where IoT sensor data is delivered via the Notecard.

• HUB_BASE_URL is the URL to reach the Notehub API to fetch data.
• HUB_AUTH_TOKEN is an authorization token required to access data in Notehub.
• HUB_PROJECT_UID is the project ID in Notehub where the sensor data is sent.
• HUB_PRODUCT_UID is the sensor device's unique ID - as one project can have many devices sending data to it.

Although not all of these environment variables are "top secret", since there's just four of them, it made more sense to keep them all grouped together in the repo instead splitting them up into "public" and "private" variables located in separate places in the code base.

Since this is a Next.js application, the way we locally store these sensitive variables that should not be checked in to source control (i.e. GitHub) is through the use of .env files.

For the app, the following sample code controls the environment variables read by Next.js when it starts up locally.

.env.local

HUB_BASE_URL=https://api.notefile.net
HUB_AUTH_TOKEN=[YOUR_AUTH_TOKEN_HERE]
HUB_PROJECT_UID=[YOUR_NOTEHUB_PROJECT_UID_HERE]
HUB_PRODUCT_UID=[YOUR_NOTEHUB_PRODUCT_UID_HERE]

These variables are then accessed inside the application using the process.env syntax. For instance, to get the HUB_PROJECT_UID value inside of the app's running code, it's simply a matter of writing process.env.HUB_PROJECT_UID in the JavaScript file where it's required.

Ok, so we've identified our environment variables that are dynamic and also should not be checked into GitHub, let's let GitHub Actions know it needs to look for them too when it runs its CI workflows.

Add the Environment Variables to the GitHub Actions YAML File

In order for GitHub to know it has a GitHub Action workflow to run for a repo, the following folders should exist at the root of the repo: .github/workflows/.

A YAML file will be contained therein: ours is named CI-PRs-and-main.yml, but you can name it anything that makes sense to you.

Inside of the YAML file, there will be a variety of steps listed out and for the steps requiring these secrets, we'll add the needed environment variables like so:

Note: If you'd like to see the full GitHub Actions file in its repo, you can click the file title below.

CI-PRs-and-main.yml

# more GitHub Actions steps above...
- name: Run unit tests
  env:
    HUB_AUTH_TOKEN: ${{ secrets.HUB_AUTH_TOKEN }}
    HUB_BASE_URL: ${{ secrets.HUB_BASE_URL }}
    HUB_PROJECT_UID: ${{ secrets.HUB_PROJECT_UID }}
    HUB_PRODUCT_UID: ${{ secrets.HUB_PRODUCT_UID }}
  run: yarn test:coverage
- run: yarn lint
- name: Cypress run # our e2e tests
  uses: cypress-io/github-action@v2
  with:
    env: configFile=staging
    start: yarn start
  env:
    HUB_AUTH_TOKEN: ${{ secrets.HUB_AUTH_TOKEN }}
    HUB_BASE_URL: ${{ secrets.HUB_BASE_URL }}
    HUB_PROJECT_UID: ${{ secrets.HUB_PROJECT_UID }}
    HUB_PRODUCT_UID: ${{ secrets.HUB_PRODUCT_UID }}
# more GitHub Actions steps below...

If you notice there are two steps in this YAML file that require the sensitive environment variables: Run unit tests and Cypress run.

Declare the env property and each needed environment variable

To tell GH Actions these custom environment variables should exist, we declare the env property inside the steps that need the variables.

Then indent and list each secret variable needed (and how the repo reads them). For example, since our repo reads the secret variable HUB_AUTH_TOKEN locally, it makes sense to name the env property to be read during the GitHub Actions workflow by the same name.

  env:
    HUB_AUTH_TOKEN:

Set the ${{ secrets.MY_SECRET }} value

After listing out the environment variable names for GitHub Actions, use the syntax ${{ secrets.NAME_OF_SECRET_IN_GITHUB }} to assign a value stored in GitHub Actions by the same name.

I'll go into how to set this value remotely in the next section.

In the GitHub Repo Set Those Sensitive Environment Variables

Now we're ready to move on to setting the secret variables in GitHub.

Access the Repo's Settings Tab

Head to your repo in GitHub that needs the variables and click on the Settings tab.

Inside of Settings , go down the sidebar to the Secrets dropdown and click on it to access the Actions page. The page should look something like this image below.

In my Actions Secrets page, there are already a few Repository Secrets present - this is what we'll be adding to to run in the GH Actions workflow. Click on the New repository secret button in the upper right corner of the screen.

Set New Environment Secrets

A new screen will appear to add new secrets to the repository. Here is where the environment variable names defined in the GitHub Actions YAML will be replicated and their values defined.

Using our example variables from above, we'd define a new variable named HUB_PROJECT_UID as the secret's name in the Name input, then add the actual project's value (as defined in Notehub, in this case) into the Value text area.

This corresponds to the ${{ secrets.HUB_PROJECT_UID }} value in the YAML file.

Click the Add Secret button after double checking the secret name and value are correct. Once the secret's saved it's encrypted in GitHub and you'll be hard pressed to see what its original value was.

Note: If you do realize later a secret is incorrect or needs to be updated, not to worry, secret values can be updated by clicking the Update button next to the secret's name on the main Actions Secrets page.

You still won't be able to see what the old value of the secret was, but there is a path forward to make changes.

Great! The GitHub Actions file knows what environment variables to inject for certain steps of the workflow, those environment variables and their values have been defined in the repo's settings tab, it's time to test it out.

Run Your GitHub Actions Workflow and Watch it Pass

For our repo, the GitHub Actions workflows are designed to run whenever a user pushes code to a pull request or to the main branch, as defined here:

# more GitHub Actions code above
on:
  pull_request:
  push:
    branches:
      - main
# more GitHub Actions code below      

If you have an already existing workflow that failed, simply go to the repo's Actions tab, click on the failing workflow, and click the Re-run all jobs button on the main workflow's screen.

If not, perform whatever action is needed to trigger a new workflow to run.

Either way, this is the end result you should see when the secret environment variables set are successfully read out.

Notice in the image that among all the steps listed out during the course of the workflow, the two that required the sensitive environment variables, Run Unit Tests and Cypress run, both successfully passed.

No sensitive info or secrets revealed in the process.

Conclusion

With the introduction of GitHub Actions a few years ago, GitHub significantly reduced the complexity and expertise needed by development teams to take advantage of the benefits continuous integration / continuous deployment pipelines provide. One thing that's always been a problem though is how to handle environment variables that should be kept secret from the general code base for their sensitive nature - the very kinds of secrets required for apps (and automated test suites) to run.

GitHub figured out a way to handle that though: secrets set in the repo's settings and encrypted as soon as they're set. Instead of having to figure out how to keep secrets safe through the use of .gitignore JSON files, build pipeline vaults, or other, non-intuitive, difficult-to-manage systems, GitHub can simply store the variables and access them in any necessary workflow step using the ${{ secrets.MY_SECRET_HERE }} syntax. Much better!

Check back in a few weeks — I’ll be writing more about JavaScript, React, IoT, or something else related to web development.

If you’d like to make sure you never miss an article I write, sign up for my newsletter here: https://paigeniedringhaus.substack.com

Thanks for reading. I hope this helps you out when building your own GitHub Actions workflows that require sensitive environment variables: CI/CD is the best way of building quality software today, and GitHub is making it easier and easier for everyone to achieve that.

References & Further Resources

• GitHub Actions
• GitHub's CI/CD resources
• Full GitHub project repo
• Blues Wireless website
• Notehub cloud