GitLab, Azure, OpenTofu, and NO secrets!

Believe it or not, you can deploy any resource using Terraform from the GitLab pipeline to the Azure cloud without any secrets!

Imagine... No secrets, no maintenance, no KeyVaults, no hassle.

Let me show you how!

The GitLab project

To demonstrate the technique I will use this project gitlab-azure-oidc-opentofu.

What does it do? It creates a Resource Group in one of the Azure Subscriptions. Is it too simple? Yes, it is. However, the main goal is to show how to set up an OpenID Connect (OIDC) authentication.

Everything was set up and running. But as you read this, the Azure subscription has probably been canceled. Just in case somebody manages to merge terraform code that pushes 100 nodes Azure Kubernetes cluster :).

And one more thing to mention.

The OIDC technology works perfectly well with Terraform. In this project, however, I chose to use OpenTofu. It is integrated into the component that is supported by GitLab.

So, the pipeline in this project might be a good example of using the OpenTofu component along with GitLab Terraform state service.

Let's start from the end...

Terraform Code

Here is the full providers.tf file.

Have a look at the terraform code, shall we?

provider "azurerm" {
  features {}

  tenant_id       = var.tenant_id
  subscription_id = var.subscription_id
  client_id       = var.client_id

  use_oidc        = true
  oidc_token      = var.oidc_token
}

All these provider settings are used for authentication. And the two last lines make Terraform (OpenTofu) use OIDC!

Where are these variables set up and what values are used? Great question!

Gitlab pipeline code

The variables are set up within the pipeline code! Here is the link to the pipeline.

There are several details that I would like to highlight.

OIDC token

Yes. I lied in the title. There is a secret used in this technology. However, this secret is orchestrated by technology itself. OpenID Connect provides your code with a temporary OIDC token. This is precisely what is supposed to be put into the oidc_token variable. But how?

Simple!

Every job that requires authentication has these several lines:

...
  id_tokens:
    TF_VAR_oidc_token:
      aud: https://gitlab.com
...

These lines create an environment variable TF_VAR_oidc_token and fill it with the token value.

And that is all you need! From the Terraform code perspective. OK. Almost all you need.

Since the environment variable is set as TF_VAR_<something>, Terraform (OpenTofu) will take its value and set an internal variable with the <something> name. It will be oidc_token in our case.

And this is exactly what our Terraform code expects!

Other Terraform (OpenTofu) variables

Right. It is not enough to set up only a token. The code requires tenant_id, subscription_id, and client_id. Where were these set up?

Simple. These are global variables of the pipeline.

variables:
...
  TF_VAR_client_id: $AZURE_CLIENT_ID
  TF_VAR_tenant_id: $AZURE_TENANT_ID
  TF_VAR_subscription_id: $AZURE_SUBSCRIPTION_ID
...

And here I use the same TF_VAR_<something> notation. So, Terraform (OpenTofu) will take anything that goes after 'TF_VAR_' and create an internal variable with the <something> name.

For example, Terraform (OpenTofu) will take

TF_VAR_client_id: $AZURE_CLIENT_ID

line, create variable client_id and fill it with the value of $AZURE_CLIENT_ID variable.

Great! But, what are $AZURE_CLIENT_ID, $AZURE_TENANT_ID, $AZURE_SUBSCRIPTION_ID?

A very good question. Again! Ten points to Gryffindor! But I am from Slytherin! OK. Minus ten points to Slytherin!

First, let me explain where these variables are set up. And then where their values are taken from.

GitLab CI/CD Variables

These are GitLab CI/CD Variables. To create them, you need to use the Settings / CI/CI / Variables interface of GitLab.

You probably have another couple of questions.

• Why do you use CI/CD Variables?

That is not a real question. There is a concept of separation of
data and logic. It's about 75 years old. :)

Yes, it is possible to put these IDs directly into the code. That makes the code unique, and less flexible and limits the usage of the code.

• And why don't you create them in TF_VAR_<something> notation?

That is another concept. Never use global variables in a subroutine. :)

It was possible to set up CI/CD Variables with TF_VAR_<something> notation. However, it would make these variables hidden. One should guess the source of data. Now everything is visible and it is much simpler to understand the code.

The Azure or rather Entra ID side

Let us dive into the meaning and source of the $AZURE_CLIENT_ID, $AZURE_TENANT_ID, $AZURE_SUBSCRIPTION_ID variables.

I believe Subscription ID and Tenant ID are self-explanatory, right? Just in case, the Subscription ID is visible almost on every interface of portal.azure.com.

Tenant ID is slightly harder to find out. The quickest way is to search for 'Entra ID' in the portal, open 'Entra ID' and you will see it immediately:

Don't close it yet.

What is the $AZURE_CLIENT_ID?

It is an identifier of the Service Principal (or App Registration) you created to authenticate your pipeline.

If you did not, here is how to do it. Open your Entra ID portal, find the App Registrations blade, and press New Registration.

Then provide a meaningful name and press 'Register'.

In my case, the name is sp_azure_subscription_contributor. You can always find your Service Principal in the App Registrations interface of Entra ID.

This is the place where you can find out the Client ID of your Service Principal.

How and where do I provide the Service Principal with access to the Azure subscription?

You created the identity and it must have permission to access your subscription. How to do it? Simple.

Open your subscription in the portal, open the Access control (IAM) blade, and Add role assignment:

Then choose the Contributor Role and, find and choose your newly created Service Principal and assign it.

So, for now, we have everything except the last and most important OIDC ingredient.

The Federated Identity Credentials is Open ID Connect magic

And there is no magic at all. You will need to create the Federated Credentials to allow your pipeline to access your Service Principal credentials without a shared secret.

Open your App Registrations information about your Service Principal. Open the Certificates & Secrets blade. Click on the Federated credentials tab, and press + Add credential

Here is the 'magic sauce':

You can enter whatever you want in the name or description. But it is crucial to provide the correct information in the next fields:

Issuer:

https://gitlab.com

Subject identifier:

project_path:<gitlab_user_name>/<gitlab_project_name>:ref_type:branch:ref:<pipeline_branch_name>

OR

project_path:<gitlab_project_group>/<gitlab_project_name>:ref_type:branch:ref:<pipeline_branch_name>

In my case, it is:

project_path:rokicool/gitlab-azure-oidc-opentofu:ref_type:branch:ref:main

Audience:

https://gitlab.com

And that is all. Here is the Resource Group created by the demo pipeline:

Useful links

GitLab Official Azure AD Federated Identity Credentials Documentation
GitLab OpenTofu Component