How to deploy Kubernetes Secrets with AWS Secrets Manager

In Kubernetes, external secrets refer to managing sensitive information, such as API keys, database passwords, or other credentials, outside of the Kubernetes cluster and then securely injecting them into the cluster when needed.

This approach is crucial for several reasons:

• Security: Storing sensitive information directly in Kubernetes manifests or configuration files is a security risk. External secret helps to reduce the risk by keeping the data separate from the application code and configuration which reduces the chance of accidental exposure or misuse of credentials.

• Separation of Concerns: Externalizing secrets allows us to separate application code and configuration which is a major concern. It allows developers to focus on writing code without worrying about handling sensitive data. The operations team can manage secrets separately by applying the best practices without impacting application logic.

• Centralized Management: External secrets facilitate centralized management of sensitive information. This means secrets can be stored, rotated, and audited in a centralized system outside the Kubernetes cluster. Centralized management simplifies the task of maintaining and updating credentials without the need to modify application code or configurations.

• Dynamic Secrets: Some external secret management systems support dynamic secrets, which are credentials that are generated on-demand and have a limited lifespan. This enhances security by minimizing the exposure window for sensitive information. Kubernetes workloads can request dynamic secrets as needed, reducing the risk of unauthorized access.

• Integration with Secret Management Tools: External secrets can be integrated with different secret management tools like HashiCorp Vault, AWS Secrets Manager, etc. These tools provide advanced security and features such as encryption, access controls, and audit trails.

• Compliance: Many organizations need to adhere to specific compliance standards that mandate secure handling of sensitive information. Externalizing secrets and leveraging external secret management tools can help meet these compliance requirements.

Some common third-party secrets management tools include:

• HashiCorp Vault: A tool for managing secrets and protecting sensitive data.

• AWS Secrets Manager: A service for managing sensitive information used by AWS services and applications.

• Azure Key Vault: A cloud service for securely storing and managing secrets.

• Google Secret Manager: Managed service provided by Google for storing secrets and confidential data.

In this blog, we will dive into AWS Secrets Manager and deploy secrets in Kubernetes using External Secrets Operator.

Deploy secrets from AWS Secret Manager in Kubernetes using Devtron

Prerequisite: Create Secrets in AWS Secret Manager

To add secrets in the AWS secret manager, do the following steps:

1. Go to the AWS secret manager console

2. Click Store a new secret

3. Add and save your secret

Deploy secrets in Kubernetes

Step-1: Deploy External Secret Operator (ESO)

External Secrets Operator is a Kubernetes operator that integrates external secret management systems like AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault, and many more. The operator reads information from external APIs and automatically injects the values into a Kubernetes Secret.

Helm chart link: https://charts.external-secrets.io

Step-2: Deploy the aws auth creds in secret name aws-secret-auth

Create a Kubernetes secret in the namespace in which the application is to be deployed using base64 encoded AWS access-key and secret-access-key. You can use Devtron's generic-helm-chart for it.

Step-3: Create a secret for your application

Go to the App Configuration >> Secrets and click on Add Secret

Select  AWS Secrets Manager under External Secret Operator (ESO) from the drop-down. You can also see all other options available, and if there are requirement of any other third-party secret which is not available as of now, that can also be supported as long as it is supported by ESO.

• AWS SecretManager

• Google Secret Manager

• Hashi Corp Vault

Step-4: Configure the secret

To configure the external secret that will be fetched from the AWS Secret Manager for your application, you will need to provide specific details using the following key-value pairs:

If you don't want to authenticate using access-key and secret-access-key, attach the SecretManagerReadWrite policy to your node and then the system should automatically fetch the secrets from the secret manager and deploy it into your Kubernetes cluster.

Once the policies are attached, you can change the SecretStore configs as specified below.

Feel free to check out the official documentation of External Secrets Operator to find out how authentication can be done. https://external-secrets.io/latest/provider/aws-secrets-manager/#aws-authentication

Interested in mastering the use of HashiCorp Vault within Kubernetes? Dive into our insightful blog to grasp how to deploy the Hashicorp Vault and integrate the fetched secrets into your Kubernetes applications.

Conclusion

As we wrap up, it's clear that unlocking the vault and harnessing the power of external secrets in Kubernetes is not just about securing your applications—it's about future-proofing your infrastructure, enabling innovation, and staying ahead of emerging threats.

Our exploration across tools like AWS Secrets Manager and HashiCorp Vault aims to bring order to the often turbulent world of secret management. Investing in these secure secret management practices today is safeguarding your assets while fostering an environment prepared for consistent growth and success.

If you have any questions feel free to reach out to us. Our thriving Discord Community is just a click away!