How to Impersonate a Service Account Using Bigquery Client Library

If you are not familiar with Service Accounts in Google Cloud, here's a short text explaining it:

A service account is a special kind of account typically used by an application or compute workload, such as a Compute Engine instance, rather than a person. A service account is identified by its email address, which is unique to the account.

The most common way to make an application act like a service account is by connecting the service account to the resource where the application is running. For instance, you can link a service account to a Compute Engine instance so that the applications running on that instance can act as the service account. After that, you can give the service account special permissions (IAM roles) so that it, and the applications on the instance, can use Google Cloud resources.

In some scenarios such as multi-tentant deployments where you need to have more strict control permissions for each organisation or customer it may make sense to tailor down the permissions, there are multiple ways of dealing with it, but recently upon facing that scenario, I used a feature from Google Cloud called Service Account impersonation to isolate each organisation resources access controls.

When an authenticated principal, such as a user or another service account, authenticates as a service account to gain the service account's permissions, it's called impersonating the service account. Impersonating a service account lets an authenticated principal access whatever the service account can access. Only authenticated principals with the appropriate permissions can impersonate service accounts.

It's also a quite nice feature since it allows you to use a short-lived token flow as stated in this part of Google Cloud documentation:

Quite common scenario if you don't want to have our engineering team downloading service accounts and potentially exposing those credentials. See Service account impersonation for more details.

How to use it within BigQuery Client Library

There are several ways of doing Service Account impersonation and many samples out there, but at the time this post was written I didn't find sample code showing how to do it using BigQuery client library, so after digging a little bit and some tests here is a working version of it:

Packages used:
pip install google-cloud-bigquery
pip install google-auth

Sample code:

from google import auth
from google.auth import impersonated_credentials
from google.cloud import bigquery


# Set scopes, usually using the global cloud-platform is enough since the actual persmissions 
# will be set at the Service Account level.
target_scopes = ["https://www.googleapis.com/auth/cloud-platform"]

source_credentials, project = auth.default()
creds = impersonated_credentials.Credentials(
    source_credentials=source_credentials,
    target_principal="[MY_SERVICE_ACCOUNT_ID]@[MYGCP_PROJECT_ID].iam.gserviceaccount.com",
    target_scopes=target_scopes,
)
client = bigquery.Client(credentials=creds, project=project, location=settings.region)

# Then run any additional commands with the impesonated auth scope
# client.query(...

Hope this helps!