Anthony Ikeda

Posted on Sep 2, 2020

GitHub Actions, Maven & Artifactory

#githubactions #maven #artifactory #java

I recently started a new Proof of Concept that started with a single repo in GitHub and built using GitHub Actions.

Then I externalised some base classes to share across a couple of repositories and, of course, broke the build because the shared classes were no longer immediately available.

This started my journey to setting up an Artifactory instance and performing proper releases on my Proof of Concept and I wanted to share how to get set up with:

Maven
Artifactory
GitHub Actions

Getting Started

Of course you will need to have a couple of GitHub Repositories and an Artifactory instance set up. The free version of Artifactory works perfectly for this scenario thought remember, there may be other steps required to setup a different dependency management tool.

First let's look at how our system authentication looks:

In order to manage the artifacts (code, jars, bill of materials) we will need to be able to use our Artifactory instance as one of our Maven repositories. This requires ensuring we can authenticate to read from our repository.

We also will need to ensure we have permissions to push to the Artifactory instace.

Lastly in order to tag and push a release in GitHub, we will need to configure Maven with the correct security credentials to do so on your behalf.

As a reference here are the different security variables we need to set up:

Environment var	Secret Value	Description
ARTIFACTORY_USERNAME_REF	'stream-github-actions'	It's not actually used but more a placeholder since we will be using the Authorization Header with the token
ARTIFACTORY_TOKEN_REF	secrets.ARTIFACTORY_TOKEN	The bearer token that has the permissions to read and write to the selected Artifactory repositories.
ARTIFACTORY_TOKEN	secrets.ARTIFACTORY_TOKEN	This token is used by the Artifactory maven plugin and uses the same token that is used to read from Artifactory.
SCM_USERNAME_REF	github-actions	Just a reference point for a committer
SCM_PASSWORD_REF	secrets.GITHUB_TOKEN	Enables controls on the git code for checkout, commit and tagging.

Creating the Artifactory Auth Tokens

There are 2 steps to setting up the Artifactory Auth tokens:

Creating a group and permissions to access repositories
Using the command line to generate the tokens

In your newly created Artifactory instance we want to first create a group that we can assign permissions to.

Access Artifactory and navigate to: Administration → Identity & Access → Groups and click "Create new group"
Give the group a name and click "Save"

Next, we want to create a permission scheme to allocate to the group.

Access Administration → Identity & Access → Permissions and click 'New Permission'
Give the permission a name (e.g. tutorial-cicd)
Click Add Repositories and select the repositories you want to grant access to (libs-release-local, libs-snapshot-local) and click Save
Click the plus sign on the Builds section then select 'Any'
Next click on the Groups tab and click the plus sign
Drag the actions-cicd group to the right and hit Ok.
Now we want to select the Repository permissions so choose: Read, Deploy/Cache
Next we want to allow build permissions so select: Read, Deploy
Click 'Create' to return to the Permission screen
Click 'Save'

Generating the Artifactory Auth Tokens

There is no way to set up auth tokens in the Artifactory UI so we will use the command line to generate them.

First let's set up some environment variables in the terminal:



$ export ART_USERNAME= #name used to log in to your artifactory
$ export ART_PASSWORD= # password to log in to your artifactory
$ export ART_GROUP=tutorial-cicd
export ART_HOSTNAME= # hostname of your artifactory instance

Next let's call your Artifactory API to generate the token:



$ curl -u "${ART_USERNAME}:${ART_PASSWORD}" -XPOST "https://${ART_HOSTNAME}/artifactory/api/security/token" \
  -d "username=stream-github-actions" \
  -d scope="member-of-groups:${ART_GROUPNAME} api:*" \
  -d expires_in=0

The main thing to pay attention to here is the scope:

scope="member-of-groups:${ART_GROUPNAME} api:*"

the member-of-groups ensures that the group we just set up with permissions is allocated to the user stream-github-actions ensuring the token grants the permissions we need.

You'll also note that we have set expires_in=0 which ensures the token does not expire.

The response should be similar to:



{
  "scope" : "member-of-groups:actions_cicd api:*",
  "access_token" : "xxxxxxxxx",
  "expires_in" : 0,
  "token_type" : "Bearer"
}

Make sure you keep the username and token handy as once we add it to GitHub Secrets you'll never see it again!

Creating the GitHub Secrets

Next we have to make sure we can use these values and since this is sensitive data, we will make use of GitHub Secrets to manage them.

Access the GitHub repository for your build and navigate to Settings → Secrets.

Click 'New Secret' and create the following secret:

Secret Name	Secret Value
ARTIFACTORY_TOKEN	The value returned

Maven POM Updates

Time to move on to configuring you maven project!

What needs to be configured:

Maven Profile
- Used to ensure we utilise the right repositories when we build the project in GitHub Actions
SCM setup
- This defines where the code exists so that when we perform a release, maven can connect to the GitHub repo.
Repository credentials for both Artifactory and GitHub in the Maven settings.xml file
The maven-artifactory-plugin
The maven-release-plugin

Set up a Maven Profile

Maven profiles are handy for configuring how you manage dependencies based on certain parameters. You can, for example, have a local setup that pulls from the default repositories you need. You can also ad a repository to your local setup with your own private credentials. However, this file (settings.xml) is not portable.

We will setup the repositories in the project pom.xml and have it activate only when an environment variable (BUILD_ENV) is configured.

In your project pom.xml file we want to first add the following profile:



<profiles>
  <profile>
    <id>github-actions</id>
    <activation>
      <property>
        <name>env.BUILD_ENV</name>
        <value>github-actions</value>
      </property>
    </activation>
    <repositories>
      <repository>
        <id>artifactory</id>
        <name>your-artifactory</name>
        <url>https://your-artifactory.jfrog.io/artifactory/libs-release-local</url>
        <releases>
          <enabled>true</enabled>
        </releases>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </repository>
    </repositories>
    <pluginRepositories>
      <pluginRepository>
        <id>artifactory</id>
        <name>your-artifactory</name>
        <url>https://your-artifactory.jfrog.io/artifactory/libs-release-local</url>
        <releases>
          <enabled>true</enabled>
        </releases>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </pluginRepository>
      <pluginRepository>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
        <id>central</id>
        <name>bintray-plugins</name>
        <url>https://jcenter.bintray.com</url>
      </pluginRepository>
    </pluginRepositories>
  </profile>
</profiles>

So what have we just done?

Above we have created a new profile that will only be active if you have set the BUILD_ENV environment variable to github-actions - we will manage this in our GitHub Actions pipeline.

Next we've defined the repositories to pull dependencies from. Artifactory acts as a proxy for artifacts that are considered standard access. However, it will now search your customer Artifactory for any dependencies you have already released and stored.

Finally we still need to load certain plugins that originate from the BinTray repository (maven-artifactory-plugin) which is why we have added in the the BinTray Plugin repository.

Credentials for you repository will be configured outside the pom.xml in a custom settings.xml file we will set up in our build pipeline.

Plugin Setup

As mentioned there are 2 plugins we need to configure:

The Maven release plugin will enable the release:prepare and the release:perform actions.

In your pom.xml update the <build> section with:



<build>
  <plugins>
    <plugin>
        <artifactId>maven-release-plugin</artifactId>
        <version>3.0.0-M1</version>
        <configuration>
          <tagNameFormat>v@{project.version}</tagNameFormat>
        </configuration>
      </plugin>
      <plugin>
        <groupId>org.jfrog.buildinfo</groupId>
        <artifactId>artifactory-maven-plugin</artifactId>
        <version>2.7.0</version>
        <inherited>false</inherited>
        <executions>
          <execution>
            <id>build-info</id>
            <goals>
              <goal>publish</goal>
            </goals>
            <configuration>
              <publisher>
                <contextUrl>https://your-artifactory.jfrog.io/artifactory</contextUrl>
                <username>stream-github-actions</username>
                <password>${env.ARTIFACTORY_TOKEN}</password>
                <repoKey>libs-release-local</repoKey>
                <snapshotRepoKey>libs-snapshot-local</snapshotRepoKey>
              </publisher>
            </configuration>
          </execution>
        </executions>
      </plugin>
  </plugins>
</build>

You can see in the Artifactory plugin, we are passing the token we generated as the password in via the environment variable. This will only be used for when we push the jar file to our Artifactory instance.

SCM Setup

So that we can perform versioned releases, we next need to configure the SCM provider.

In the pom.xml you want to add the following somewhere near the top of the file:



  <scm>
    <developerConnection>scm:git:https://github.com/your-org/your-repo.git/</developerConnection>
    <url>https://github.com/your-org/your-repo</url>
    <tag>HEAD</tag>
  </scm>

Here we have specified the type of scm provider (scm:git) and the location of the repository. We also want to make sure when we release that maven will tag the HEAD of the branch with the new version.

One last setup item is going to be to configure our server credentials:



<properties>
  ...
  <project.scm.id>github</project.scm.id>
</properties>

This indicates the server configuration that we are going to use in the settings.xml file we will generate in GitHub actions and ensures that we can successfully tag and bump the project version.

Creating our Build Pipeline

Okay it's been a long journey so far but we are close to the finish line. We are now going to take what we've created and build out our build and release pipeline.

What we will be using

First let me outline what we will be doing:

Create a standard GitHub Actions Maven build pipeline
Customizing 2 jobs:
- Build and Unit Test
- Prepare and Perform release

The Build and Unit test job will run on all branches and pull requests, however, our release will only run on the master branch.

Basic Build

Go to your GitHub repository and select Actions and then select Java with Maven - Set up this workflow:

For now, let's just get the build going with the basic setup so let's make some small changes:



name: Java CI with Maven

on:
  push:
    branches: [ "**" ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 11.0.2
        uses: actions/setup-java@v1
        with:
          java-version: 11.0.2
      - name: Build with Maven
        run: mvn -B package --file pom.xml

The first change we will make is to ensure the build runs on all branches, so you can change the on.push.branches to ["**"]

Next I've configured the JDK to build with by setting the java-version to 11.0.2.

For now commit the file and the very first build should run.

Now we will setup the settings.xml file.

Setting up settings

Why do we need this file? Well as mentioned earlier, we need to store credentials somewhere, in this case we will be referencing environment variables to ensure we don't expose any sensitive data.

The action we will use to create the file is the whelk-io/maven-settings-xml-action@v11 This will allow us to create a custom built settings.xml file that maven will use:



name: Java CI with Maven

on:
  push:
    branches: [ "**" ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 11.0.2
        uses: actions/setup-java@v1
        with:
          java-version: 11.0.2
      - name: Setup Maven settings.xml
        uses: whelk-io/maven-settings-xml-action@v11
        with:
          servers: |
            '[
              {
                "id": "artifactory",
                "username": "${env.ARTIFACTORY_USERNAME_REF}",
                "password": "${env.ARTIFACTORY_TOKEN_REF}"
              }
            ]'
     - name: Build with Maven
       env:
          ARTIFACTORY_USERNAME_REF: "stream-github-actions"
          ARTIFACTORY_TOKEN_REF: ${{ secrets.ARTIFACTORY_TOKEN }}
          BUILD_ENV: 'github-actions'
       run: mvn -B package --file pom.xml

As you can see we aren't embedding any sensitive date, just environment variables. When this step runs a newly created settings.xml file will exist with the following content:



<settings>
  <servers>
    <server>
      <id>artifactory</id>
      <username>${env.ARTIFACTORY_USERNAME_REF}</username>
      <password>${env.ARTIFACTORY_TOKEN_REF}</password>
    </server>
  </servers>
</settings>

Now when the build runs, we want to use the injected secrets as these environment variables.

You'll also note we are setting the BUILD_ENV environment variable so that Maven uses the profile we set up with pull artifacts from our own Artifactory server.

You can also see in the maven.yml build file that we are mapping our secrets and values to the environment variables that our pom.xml and settings.xml files are using.

Commit the changes and your build should run and pull the dependencies from your own Artifactory instance.

Doing a release

Now that we have a build running successfully we can apply a release and version our artifact. As mentioned earlier, we will make sure we bump the artifact version and push to Artifactory only when a merge to master is successful.

Most of the building blocks are in place right now and we just need to add a release stage to our main.yml actions file:



name: Java CI with Maven

on:
  push:
    branches: [ "**" ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 11.0.2
        uses: actions/setup-java@v1
        with:
          java-version: 11.0.2
      - name: Setup Maven settings.xml
        uses: whelk-io/maven-settings-xml-action@v11
        with:
          servers: |
            '[
              {
                "id": "artifactory",
                "username": "${env.ARTIFACTORY_USERNAME_REF}",
                "password": "${env.ARTIFACTORY_TOKEN_REF}"
              }
            ]'
     - name: Build with Maven
       env:
          ARTIFACTORY_USERNAME_REF: "stream-github-actions"
          ARTIFACTORY_TOKEN_REF: ${{ secrets.ARTIFACTORY_TOKEN }}
          BUILD_ENV: 'github-actions'
       run: mvn -B package --file pom.xml

  release:
    runs-on: ubuntu-latest
    needs: build
    if: github.ref == 'refs/heads/master'

    steps:
      - uses: actions/checkout@v2
      - name: Setup JDK 11
        uses: actions/setup-java@v1.3.0
        with:
          java-version: 11.0.2

      - name: maven-settings-xml-action
        uses: whelk-io/maven-settings-xml-action@v11
        with:
          servers: '
            [
              {
                "id": "github",
                "username": "${env.SCM_USERNAME_REF}",
                "password": "${env.SCM_TOKEN_REF}"
              },
              {
                "id": "artifactory",
                "username": "${env.ARTIFACTORY_USERNAME_REF}",
                "password": "${env.ARTIFACTORY_TOKEN_REF}"
              }
            ]'
      - name: Configure git
        run: |
          git config --global committer.email "noreply@github.com"
          git config --global committer.name "GitHub"
          git config --global author.email "${GITHUB_ACTOR}@users.noreply.github.com"
          git config --global author.name "${GITHUB_ACTOR}"
      - name: Prepare the release
        env:
          SCM_USERNAME_REF: 'github-actions'
          SCM_TOKEN_REF: ${{ secrets.GITHUB_TOKEN }}
          ARTIFACTORY_USERNAME_REF: "stream-github-actions"
          ARTIFACTORY_TOKEN_REF: ${{ secrets.ARTIFACTORY_TOKEN }}
          BUILD_ENV: 'github-actions'
        run: |
          ./mvnw --batch-mode release:prepare
      - name: Push the release
        env:
          SCM_USERNAME_REF: 'github-actions'
          SCM_TOKEN_REF: ${{ secrets.GITHUB_TOKEN }}
          ARTIFACTORY_USERNAME_REF: "stream-github-actions"
          ARTIFACTORY_TOKEN_REF: ${{ secrets.ARTIFACTORY_TOKEN }}
          ARTIFACTORY_TOKEN:  ${{ secrets.ARTIFACTORY_TOKEN }}
          BUILD_ENV: 'github-actions'
        run: ./mvnw --batch-mode release:perform
      - name: Rollback if failure
        if: ${{ failure() }}
        run: ./mvnw --batch-mode release:rollback

There are 6 steps we are introducing here:

Setup our JDK we want to build with
Create our settings.xml file but this time with a new server entry for GitHub
Configure a Git username for commits
The Prepare release step
The Perform release step
If the release process fails, we want to roll back the steps that set up the release.

To prevent the release job from running before we've successfully built and unit tested the project, we've included a needs: build directive. This ensures that unless the build job passes, the release job won't run.

We are also validating that we are only running the release on the master branch as indicated by if: github.ref == 'refs/heads/master'

In order to authenticate GitHub, we've added a new server to our settings.xml file. If you recall we added a property to the pom.xml file <project.scm.id>github</project.scm.id> which tells maven to use the GitHub server configuration with the id of 'github'.

Also note we are using reserved secret for the credentials for GitHub: secret.GITHUB_TOKEN. This is a reserved secret and enables you to use it for any actions you need to perform on the repository.

Once you've committed your work to master, the build should run, your release will run and you will find a new version tag and the artifact uploaded to your Artifactory instance.

Top comments (6)

Karl Heinz Marbaise • Sep 3 '20

Is there a good reason why you defined the repositories via a profile in your pom file and not within settings.xml ?

Anthony Ikeda • Sep 3 '20

For this article the profile was a write once and forget, also I was able to isolate the profile while testing locally from may main configuration.

You can place the profile in the settings.xml file using the whelk-io/maven-settings-xml-action@v11 plugin, however, the different touchpoints of credentials and mappings (between the pom.xml, settings.xml and main.yml build file) just made it easier putting the profile in the pom.xml.