Configuring Gitlab CI and Private Go Modules

Last year I talked about private repositories and Gitlab CI as well as our story when we migrated to modules, during that migration (and because of the way we organize our projects) we were forced to start appending .git to our private modules and to use the replace directive in go.mod to explicitly indicate Go the exact location of that repository.

Something like:

require (
    private.gitlab.instance/project/team/service-name v1.0.0
)

replace (
    private.gitlab.instance/project/team/service-name => private.gitlab.instance/project/team/service-name.git v1.0.0
)

That workflow was cumbersome, not idiomatic, and made upgrading our own internal dependencies harder than normal.

However after investigating even more, I realized there was a nicer alternative which involved using:

• Gitlab's Personal Access Tokens, and
• .netrc

This required 4 changes:

1. Using a Personal Access Token,
2. Updating .gitlab-ci.yml to properly use the new token,
3. Updating our Dockerfile to define the .netrc configuration, and
4. Remove replace directives in go.mod.

Using a Personal Access Token

We specifically need a new token with the following scopes:

• read_api
• repository_access

For security concerns this token will be defined as a new CI/CD environment variable, for that we will define two new variables:

• GO_MODULES_USER representing the Gitlab username and
• GO_MODULES_PERSONAL_ACCESS_TOKEN representing the token we just created.

The reason behind defining a new token instead of using CI_JOB_TOKEN is that this token does not have enough permissions for accessing the Gitlab API used to determine the right repository when calling go mod <xyz> behind the scenes.

Updating .gitlab-ci.yml to properly use the new token

Replacing the docker build instruction to pass in the new variables should do it, so from something like this:

docker build \
  --build-arg CI_JOB_TOKEN

We could change it to:

docker build \
  --build-arg GO_MODULES_USER
  --build-arg GO_MODULES_PERSONAL_ACCESS_TOKEN

Updating our Dockerfile to define the .netrc configuration

Our Dockerfile will be changed to something like:

FROM golang:1.15.0-alpine3.12

ARG GO_MODULES_USER
ARG GO_MODULES_PERSONAL_ACCESS_TOKEN

WORKDIR /project-name/

RUN go env -w GOPRIVATE="private.gitlab.instance" && \
    echo -e "machine private.gitlab.instance\nlogin ${GO_MODULES_USER}\npassword ${GO_MODULES_PERSONAL_ACCESS_TOKEN}" > ~/.netrc

COPY ["go.mod", "go.sum", "./"]
RUN go mod download

COPY . .

#-

FROM golang:1.15.0-alpine3.12

WORKDIR /project-name/

ENV PATH=/go/bin/:$PATH

COPY --from=0 /project-name/ /project-name/
COPY --from=0 /go/ /go/

Thanks to the multi-stage build we know for sure the credentials are not stored in the final Docker image being built

Remove replace directives in go.mod.

Finally the last step would be get rid of those .git replace directives:

require (
    private.gitlab.instance/project/team/service-name v1.0.0
)

And with that we are back to the well known go get/mod workflow!