Table of Contents
- Table of Contents
- Undocumented parameters for config.sh
- Automated token retrieval
- Automated runner removal
- Conclusion
In the last post we built a docker image which can launch an actions runner. The image definition up to the end of that post is here.
There are a few issues with the setup in that image.
- We have a few hacks in the
entrypoint.sh
script for configuring the runner - e.g. passing the name via anecho
. - Anytime you want to start a new runner you need to get the latest token from the UI.
- You need to manually unregister runners which you want to remove.
I'm going to tackle these three issues now. To help, we should dive into the actions runner source code.
Undocumented parameters for config.sh
I was pretty sure the help section for config.sh
was missing some flags. All it prints is:
$ ./config.sh --help
Commands:,
./config.sh Configures the runner
./config.sh remove Unconfigures the runner
./run.sh Runs the runner interactively. Does not require any options.
Options:
--version Prints the runner version
--commit Prints the runner commit
--help Prints the help for each command
To configure a runner we need the following four things at least:
- The url to the repo to register the runner against, this has a
--url
option. - The token this has a
--token
option. - The name of the runner, we cheat in our entrypoint.sh by echoing a name into the execution of
config.sh
. This sets the first prompt as the value we echo. - The work directory, here we also cheat with
echo
and because we echo a single value this defaults to_work
.
./config.sh
calls out to a .net
binary called Runner.Listener, the source code can be found here. What can we find in this?
Diving into the source
Our aim is to find any flags which are not documented in ./config --help
that may help us automate configuration. The source code is large enough that blindly searching through would be tedious, so what information do we have that can narrow down our search?
When we run config.sh
it defaults the work folder to _work:
$ ./config.sh --url https://github.com/${OWNER}/${REPO} --token ${TOKEN}
...
Enter name of work folder: [press Enter for _work]
...
So, their must be a string in the code with the value _work
. If we search in the actions-runner repo, we find it in a few places. The most promising is Runner.Common Constants.cs.
This file contains quite a few interesting constants which may be of use later on! For our current goal, there is the CommandLine Args class.
public static class Args
{
public static readonly string Auth = "auth";
public static readonly string MonitorSocketAddress = "monitorsocketaddress";
public static readonly string Name = "name";
public static readonly string Pool = "pool";
public static readonly string StartupType = "startuptype";
public static readonly string Url = "url";
public static readonly string UserName = "username";
public static readonly string WindowsLogonAccount = "windowslogonaccount";
public static readonly string Work = "work";
// Secret args. Must be added to the "Secrets" getter as well.
public static readonly string Token = "token";
public static readonly string WindowsLogonPassword = "windowslogonpassword";
public static string[] Secrets => new[]
{
Token,
WindowsLogonPassword,
};
}
This looks promising. It mentions two args we already use - --token
and --url
. Let's see if we can set name and the work dir similarly:
$ ./config.sh --url https://github.com/${OWNER}/${REPO} \
--token ${TOKEN} \
--name my-runner \
--work _work
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
√ Runner successfully added
√ Runner connection is good
# Runner settings
√ Settings Saved.
It works! Now there are no prompts. We can update our docker image's entrypoint.sh
from the previous post to take this into account:
#!/usr/bin/env bash
OWNER=$1
REPO=$2
TOKEN=$3
NAME=$4
./config.sh \
--url https://github.com/${OWNER}/${REPO} \
--token ${TOKEN} \
--name ${NAME} \
--work _work
./run.sh
See here for the full code up to this point including the Dockerfile.
Automated token retrieval
Currently anytime we want to run a new action we need to login to the UI and retrieve a new token. Recently github release an API to generate this token, see the self hosted runners API docs. We can take advantage of this in our entrypoint.sh
script.
Generating a personal access token
To generate an actions registration token we first need to generate a Personal Access Token that we can use to call the github API. I'll refer to these as PAT's from here on in so as not to confuse them with the registration token, as they are both tokens of a different type!
To generate a PAT, go to Settings in your account:
Then Developer Settings in the left panel:
Go to Personal Access Tokens:
Finally click the Generate a new token button:
Give your token a name that associates it with what it's doing. I've called it actions-runner-registration
. I am not 100% sure the exact scopes that it needs, setting all repo
scopes works however so we can do this for now.
Scroll to the bottom and click Generate Token.
Make sure to copy the token on the next page and store it in a safe place! This is the only chance you get to see it, and if you do not you will have to generate a new one.
Automatically generating a registration token
Now we have our PAT, we can use the API to generate an actions runner registration token. The docs for the different methods of auth with the github API can be found here. Let's keep his simple and use curl
.
$ curl -XPOST \
-H "authorization: token ${YOUR_PAT}" \ # ${YOUR_PAT} is the PAT we generated above
https://api.github.com/repos/${OWNER}/${REPO}/actions/runners/registration-token
{
"token": "*******",
"expires_at": "2020-02-02T16:21:21.410+00:00"
}
It works! With this we can update our entrypoint.sh
script again.
#!/usr/bin/env bash
OWNER=$1
REPO=$2
PAT=$3
NAME=$4
token=$(curl -s -XPOST \
-H "authorization: token ${PAT}" \
https://api.github.com/repos/wayofthepie/gh-app-test/actions/runners/registration-token |\
jq -r .token)
./config.sh \
--url https://github.com/${OWNER}/${REPO} \
--token ${token} \
--name ${NAME} \
--work _work
./run.sh
If you have never used the jq
command before, jq .token
means take the token field out of the given json. The json returned from the curl
call has two fields, token and expires_at. If we used jq
without the -r
flag it would return the token wrapped in quotes, we want it without quotes. So the full command is jq -r .token
.
We need to use curl
and jq now, neither of these are in our docker image so we need to also update our Dockerfile.
FROM ubuntu
ENV RUNNER_VERSION=2.164.0
RUN useradd -m actions \
&& apt-get update \
&& apt-get install -y \
wget \
# add curl and jq
curl \
jq
RUN cd /home/actions && mkdir actions-runner && cd actions-runner \
&& wget https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
WORKDIR /home/actions/actions-runner
RUN chown -R actions ~actions && /home/actions/actions-runner/bin/installdependencies.sh
USER actions
COPY entrypoint.sh .
ENTRYPOINT ["./entrypoint.sh"]
The source for the image up to this point can be found here.
Let's build it and run, see if it works:
$ docker build -t actions-image .
Sending build context to Docker daemon 81.92kB
Step 1/9 : FROM ubuntu
---> 775349758637
...
Step 9/9 : ENTRYPOINT ["./entrypoint.sh"]
---> Using cache
---> 0dff3c9bac89
Successfully built 0dff3c9bac89
Successfully tagged actions-image:latest
$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
A runner exists with the same name
Would you like to replace the existing runner? (Y/N) [press Enter for N]
Ah an issue! We registered a runner called my-runner previously in this post and never removed it. We can manually remove for now and try again:
$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
√ Runner successfully added
√ Runner connection is good
# Runner settings
√ Settings Saved.
√ Connected to GitHub
2020-02-02 15:42:22Z: Listening for Jobs
It works! Now we don't have to generate a token manually each time we want to start a new runner. One major issue remains, the fact you have to manually remove runners.
Automated runner removal
When we kill the container running an actions runner it should automatically unregister itself. To manually unregister we use the ./config remove
command. Let's add some small changes to our entrypoint so it will do this automatically.
#!/usr/bin/env bash
OWNER=$1
REPO=$2
PAT=$3
NAME=$4
cleanup() {
token=$(curl -s -XPOST -H "authorization: token ${PAT}" \
https://api.github.com/repos/${OWNER}/${REPO}/actions/runners/registration-token |\
jq -r .token)
./config.sh remove --token $token
}
token=$(curl -s -XPOST \
-H "authorization: token ${PAT}" \
https://api.github.com/repos/wayofthepie/gh-app-test/actions/runners/registration-token |\
jq -r .token)
./config.sh \
--url https://github.com/${OWNER}/${REPO} \
--token ${token} \
--name ${NAME} \
--work _work
./run.sh
cleanup
The cleanup function is run at the end of the script, it gets a new registration token and called ./config.sh remove --token ${TOKEN}
to unregister the runner. The code to this point can be found here. Does it all work?
$ docker build -t actions-image .
Sending build context to Docker daemon 88.06kB
Step 1/9 : FROM ubuntu
---> 775349758637
...
Successfully built 1de80d7b74d6
Successfully tagged actions-image:latest
$ docker run -ti --rm actions-image ${OWNER} ${REPO} ${PAT} my-runner
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
√ Runner successfully added
√ Runner connection is good
# Runner settings
√ Settings Saved.
√ Connected to GitHub
2020-02-02 16:05:31Z: Listening for Jobs
^CExiting...
# Runner removal
√ Runner removed successfully
√ Removed .credentials
√ Removed .runner
I used CTRL-C to cancel and it ran the cleanup!
Conclusion
We can now easily register multiple actions runners on the fly and have the automatically clean up when we are finished with them. However, we still need to launch them manually. What if we could launch a runner per commit, have them run a single build, and disappear? That's the dream. We'll star working towards that in the next post.
Top comments (0)