We use Git for everything now, from code source to organization, history, and even for Kubernetes Cluster Management (aka GitOps).
But, there is still something not widely adopted... managing our secrets in Git. Some tools like HashiCorp Vault, Google Secret Management, or AWS Secret Manager provide us a solution to manage our secrets in a dedicated system, but they are still not in sync with our source code.
We will see here, thanks to Mozilla SOPS how to integrate our secrets management directly in Git.
SOPS: Secrets OPerationS
SOPS is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP (demo)
Binaries and packages of the latest stable release are available at https://github.com/getsops/sops/releases.
For the adventurous, unstable features are available in the main branch, which you can install from source:
$ mkdir -p $GOPATH/src/github.com/getsops/sops/
$ git clone https://github.com/getsops/sops.git $GOPATH/src/github.com/getsops/sops/
$ cd $GOPATH/src/github.com/getsops/sops/
$ make install
(requires Go >= 1.19)
If you don't have Go installed, set it up with:
$ {apt,yum,brew} install golang
$ echo 'export GOPATH=~/go' >> ~/.bashrc
$ source ~/.bashrc
$ mkdir $GOPATH
Or whatever variation of the above fits your system and shell.
To use SOPS as a library, take a look at the decrypt package.
…
DISCLAIMER: I've previously written an article on the same subject about a project named kubesec
specialized in Kubernetes Secret. The project seems to be stopped and Mozilla SOPS is a better alternative right now, because it can manage every kind of secrets, not only Kubernetes ones.
Sops Installation
Sops is very simple to install, like every golang
application, you just have to download the binary for your specific Operating System (Linux, Mac, Windows) directly from the release page on GitHub.
By the way, you can install it thanks to brew
on Mac & Linux (sops formuale).
Use case
What we will try to achieve is to store secrets in Git but with restrictions on "who can access what".
For example, we have 4 environments, dev_a
, dev_b
, int
, and prod
and 3 team members, Alice
, Bobby
, and Devon
.
We want to restrict secrets access with the following requirements:
-
dev_a
secrets should be only accessible byAlice
. -
dev_b
secrets should be only accessible byBobby
. -
int
secrets should be accessible byAlice
,Bobby
, andDevon
. -
prod
secrets should be accessible only byDevon
.
Each of them already has configured their GPG key pairs. If you need to set them up, you can follow the official GitLab documentation 🦊 about this.
In this example, secrets are just plain old env
files.
dev_a
configuration
Alice will generate a file containing a secret:
Thanks to the command:
sops -pgp 5844C613B763F4374BAB2D2FC735658AB38BF93A -e dev_a.env > dev_a.encrypted.env
Alice
has encrypted the file dev_a.env
and stored the result in dev_a.encrypted.env
. She is the only one able to decrypt it.
Let's see, if we run this as Bobby
:
dev_b
and prod
configurations are similar to the one created by Alice
. Then, Bobby
and Devon
will respectively create dev_b
and prod
.
int
configuration
In this configuration, we would like every developers to be able to read this file. But, only developers from the project and not everyone with access to the git repository... so we still have to encrypt this file.
To do so, Devon
will execute the following commands:
Devon
has to create the secret with the command
sops --pgp 57E6DA39E907744429FB07871141FE9F63986243,\
5844C613B763F4374BAB2D2FC735658AB38BF93A,\
AE0D6FD0242FF896BE1E376B62E1E77388753B8E \
-e int.env > int.encrypted.env`
```
This command contains every public key ids, comma sparated.
We can check that both `Alice` and `Bobby` can decrypt the `int.encrypted.env` file:
```shell
alice $ sops -d int.encrypted.env
secret=5Tz2QNxki789YFDa
```
```shell
bobby $ sops -d int.encrypted.env
secret=5Tz2QNxki789YFDa
```
All the `*.encrypted.env` files are now stored in `Git` and can be managed like any other resources, with history and diff in commits. Only those defined during encryption can read them edit them.
You can find the source code of this article, files, and scripts in this [GitLab repository](https://gitlab.com/davinkevin/sops-blog-post).
I hope this will help you to use `Git` & `SOPS` to manage your secrets.
Top comments (2)
I make the case, here, that that is a feature.
Take it from someone who has lead the charge on this kind of thing before, yup, there's a lot of history down that road.
If you have someone crowing about how they don't need to worry about
etcd
backups, because they can restore their entire application from.yaml
files, shouldn't that raise an eyebrow, or maybe even some questions? Like, "What about sensitive data or identification is needed by your applications?"Not to mention that kind of thing sort of screws up the way source control and version control is supposed to work.
Am I going to
git bisect
and get stuck with old, hopefully expired versions of credentials, too?shakes his cane at you
mutters incomprehensibly
ansible-vault is an alternative