GitOps is a paradigm that puts Git at the heart of building and operating cloud native applications by using Git as the single source of truth and empowers developers to perform what used to fall under IT operations. This post is part a blog post series covering GitOps and Kubernetes security.

Kubernetes - A GitOps Companion

Kubernetes, as the new application server, leverages a “declarative” approach when it comes to building cloud native application, which means that application configuration is guaranteed by a set of facts instead of by a set of instructions. With application’s declarations versioned in Git, we have a single source of truth, our apps can be easily deployed and rolled back to and from Kubernetes, and when disaster strikes, your cluster’s infrastructure can also be reproduced.

With Git at the center of the delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes.

Is GitOps right for you?

The fresh approach that GitOps + Kubernetes brings into the application delivery lifecycle is undeniably different, increasing engineering velocity, as well as simplifies building the CI+CD pipelines themselves. The question of whether a GitOps ‘Pull’ approach is a better fit than ‘Push’ approach is really a matter of engineering & operational culture of an organization, as well as almost a theological question of whether engineering are accountable for security and what visibility into this application server blackbox, security teams require.

GitOps & Security

1. GitOps changes are synched into the cluster only through the cluster git repository users. The repository is secured at the same level of the git user accounts. A compromised user account in with permissions to push into the cluster git repo can introduce changes that will result in a data breach, service disruption or anything in between. There must be additional guardrails that GitOps infrastructure must implement in the form of whitelisting in order to have a cluster side guardrails.

2. GitOps tools like Flux, ArgoCD and alike are practically running with cluster god permissions - and are persistent in the cluster. Kubernetes Dashboard, which is considered to be a high risk cluster, is often times removed from the cluster.
   ‘Push’ based CD pipelines, such as Spinnaker, Jenkins and alike are external to the cluster, invoke on-demand, and introduce automation-driven changes into the cluster.

   

   Example: Flux RBAC Permission - Cluster God * * * * *

3. GitOps tools like Flux, ArgoCD and alike require cluster external access, represented as domain names (github.com, bitbucket.org, gitlab.com,..) which means that the Kubernetes native policies are not suitable to implement to segment those highly privileged in-cluster components.

4. Application secrets in a GitOps era requires a Kubernetes external secret provider. For example Hashicorp Vault, AWS KMS, Azure Vault and alike. Alternatively, teams can revert into Git Secrets which means committing secrets into git in their encrypted form, and decrypting the secrets before application consumption.

Conclusion

Continuous integration/continuous development (CI/CD) with the Kubernetes ecosystem does have a variety of tools to choose from and organizations should use the tools that are best suited for their specific use cases and culture. Glueing all the pieces together is not trivial. Integrating security and consuming security insights by various stakeholders is an equally challenging task to achieve. GitOps simplifies this in some aspects, but complicates in other aspects.

Stay tuned for Part 2.

Join our upcoming webinar on April 22nd: GitOps Best Practices for Continuous Deployment and Progressive Security.