Your Rust project on production with GitLab CI and Google App Engine

If you want to release your Rust project on Google App Engine, you can use GitLab CI to help you with this task. Through this article I will guide you through the whole process.

Google App Engine

First, create an account at cloud.google.com/appengine. You will get $300 in free credits to use with your account during the first 90 days.

Then, go to console.cloud.google.com/appengine and create a new project.

You can change Project name and Project ID according to your needs.

Once your project is ready, you will see the following screen.

Now, create an application and select a region for your app.

And select the programming language and environment. Rust is not listed, select Other instead and Flexible as the environment.

A service account is required for GitLab to have access to your Google App Engine account.

Go to console.cloud.google.com/iam-admin/serviceaccounts/create for creating a new service accounts.

Change Service account name and Service account ID according to your needs.

Add the following roles to the service account you're creating:

• App Engine Admin
• App Engine Deployer
• App Engine Service Admin
• Service Account User
• Storage Admin
• Cloud Build Editor

After that, you will be redirected to the Service accounts page. There you will see the service account you created.

Click on it and go to the Keys section and create a new key. This will generate a JSON file, download it. Later, you will copy the content of this file.

Enable the App Engine Admin API. Go to console.developers.google.com/apis/api/appengine.googleapis.com/overview?project=rust-gitlab-project. Replacing rust-gitlab-project from the URL with the name of your project.

Finally, go to console.cloud.google.com/storage/browser and add Storage Object Creator and Storage Object Viewer permissions to your service account for the buckets listed there:

• PROJECT-ID.appspot.com
• staging.PROJECT-ID.appspot.com

GitLab CI

Create a repository for your project and sync your code.

Create a Dockerfile in your repository with the following content:

FROM rustlang/rust:nightly AS builder
WORKDIR /app

ENV ROCKET_ENV=prod

# Avoid having to install/build all dependencies by copying
# the Cargo files and making a dummy src/main.rs
COPY Cargo.toml .
RUN mkdir src && echo "fn main() {}" > src/main.rs
COPY src/lib.rs src/lib.rs
RUN cargo build --release

# We need to touch our real main.rs file or
# else docker will use the cached one.
COPY src src
RUN touch src/main.rs
RUN cargo build --release

# Size optimization
RUN strip target/release/server

# Start building the final image
# Ideally would use `scratch` as a base image, but this would require
# fully static compilation by linking against musl instead of glibc
# (i.e. emk/rust-musl-builder) but it's a bit restrictive.
# This isn't a 5mb image, but 100mb is still better than 2gb ;)
FROM ubuntu
WORKDIR /app
COPY --from=builder /app/target/release/server .
COPY Rocket.toml Rocket.toml
COPY static static
COPY templates templates
EXPOSE 8080
ENV ROCKET_ENV=stage
ENTRYPOINT ["./server"]

The Docker image used for releasing the app is built through multi-stage builds. At first, the app is compiled using the Docker image of the Nightly version of Rust. You can change it to stable depending on the crates you're using for your project.

After compilation, you get the binary you need for running your app on production. This is done to avoid copying all the files generated after running cargo build --release, specially the ones located in the target directory.

In the second stage, the official Docker image of Ubuntu is used. The binary generated before is copied, along with the files required for running your app. Now you have the Docker image that would be used for releasing on Google App Engine.

Check this repository on GitHub where you can see the files that must be in your repository for publishing on Google App Engine.

Then, create the configuration file for Google App Engine, app.yaml, with the following content:

runtime: custom
env: flex
# Warning! skip_files doesn't work for custom runtime
skip_files:
    - ^target.*$

Before creating the GitLab CI configuration file, .gitlab-ci.yml, go to Settings ⇾ CI/CD and add two variables:

• PROJECT_ID. Type the ID of your Google App Engine project in the Value field.

• SERVICE_ACCOUNT. In the Value field, copy the content of the JSON file you downloaded previously.

And finally, create the .gitlab-ci.yml file in your repository, with the following content, according to this article:

image: google/cloud-sdk:alpine

deploy_production:
  stage: deploy
  environment: Production
  only:
  - master
  script:
  - echo $SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
  - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
  - gcloud config set app/cloud_build_timeout 1600
  - gcloud --quiet --project $PROJECT_ID app deploy app.yaml

deploy_staging:
  stage: deploy
  environment: Staging
  only:
  - staging
  script:
  - echo $SERVICE_ACCOUNT > /tmp/$CI_PIPELINE_ID.json
  - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
  - gcloud config set app/cloud_build_timeout 1600
  - gcloud --quiet --project $PROJECT_ID app deploy staging-app.yaml

after_script:
- rm /tmp/$CI_PIPELINE_ID.json

Set app/cloud_build_timeout to 1600 as compilation could take more time than the default value.

Once the file is created, the pipeline will be executed and after a few minutes, your app will be on production at PROJECT_ID.uc.r.appspot.com.

Check rust-gitlab-project.uc.r.appspot.com for a demo and gitlab.com/mattdark/rust-gae-demo for the code.