Centralizing IAM Access Key Used by IAM Machine User Using AWS Secrets Manager

In Amazon Web Services (AWS) environment, if We want to interact with AWS API, we can use AWS API key/AWS credentials for programmatic access which usually consist of AWS Access Key ID and AWS Secret Access Key. Usually this API key associated with AWS Identity and Access Management (IAM) such as IAM user. IAM user also can be classified to IAM user for human-to-machine interaction and IAM user for machine-to-machine interaction. Example usage of machine-to-machine interaction is when We created IAM machine user for CI/CD integration, then We save API key credential of IAM machine user in CI/CD engine environment variable so that the engine can interact with AWS. For example when it is used to push container image from CI/CD to AWS.

Sometimes it is common that those API key need to share with developers/engineers that in need and has privileged to access the key. For example Infrastructure Team want to share API key information with one of developer in service product team A. If the API key just shared or sent via csv file, probably there will be a problem in the future. For example that developer forgot where the API key is saved, or if the developer/engineer has resigned and forgot to share that API key with other team member or that API key might be shared or leaked to unprivileged user or bad actor and will open a security hole.

In this blogpost, I want to share some idea or solution regarding problem that mentioned above. The solution is We keep the API key used by IAM machine user in centralized way using AWS service called AWS Secrets Manager.

Scenario

• Configuration of storing API key will be initialized with Infrastructure as Code (IaC) method using Terraform
• Terraform configuration is using remote backend via Terraform Cloud

Directory Structure

In this article, the file structure used by Terraform codes is described below:

.
└── environment-name/
    └── secrets-manager/
        └── region-name/
            ├── main.tf
            ├── resources.tf
            └── variables.tf

Configuration

1. I started with creating file main.tf. In this file, I declared terraform configuration that refers to terraform cloud (app.terraform.io) as a remote backend. I also declared the organization and workspace used by Terraform code. Next in this file, I declared the provider used by Terraform code which is Hashicorp/AWS, and versions related to it. Last, I declared the AWS region used by Terraform code to refer to the variable aws_region that is configured in file variables.tf which I will explain later.
   

   ###########################
   # Terraform Configuration #
   ###########################
   
   terraform {
     backend "remote" {
       hostname     = "app.terraform.io"
       organization = "my-organization"
   
       workspaces {
         name = "my-secretsmanager-workspace"
       }
     }
   }
   
   terraform {
     required_providers {
       aws = {
         source  = "hashicorp/aws"
         version = "~> 4.48.0"
       }
     }
   
     required_version = ">= 1.3.0"
   }
   
   provider "aws" {
     region = var.aws_region
   }
   

2. Next, I defined variables used by the Terraform in file variables.tf.
   

   variable "aws_region" {
     type        = string
     description = "AWS Region"
     default     = "ap-southeast-1"
   }
   
   variable "waiting_deletion_period" {
     type        = number
     description = "Waiting period in days before secret deleted. Set to 0 for force destroy"
     default     = 0
   }
   

3. Refer to Terraform AWS provider document regarding resource aws_secretsmanager_secret_version. If We want to create secret with key-value pairs, we can get those value with several methods and one in common is from variable. So that, still in file variables.tf, I created variable that consist key-value pairs used by IAM machine user. And because AWS_SECRET_ACCESS_KEY contain sensitive value, I did not type the real value in the code but I changed manually from AWS Web Console to prevent repository contain any sensitive value. Example:
   

   variable "my_machine_credentials" {
     type        = map(string)
     description = "Key value pairs consist of AWS credentials used by my machine user"
     default = {
       "AWS_ACCESS_KEY_ID"     = "AKIAWE1AB2CD34FG5I767"
       "AWS_SECRET_ACCESS_KEY" = "change_me"
     }
   }
   

4. Next in file resources.tf, I declared secret with value, which value is referred from variable that described in point number 3.
   

   ###########
   # Secrets #
   ###########
   
   resource "aws_secretsmanager_secret" "dummy_machine_sbx" {
     name                    = "dummy-machine-sbx"
     recovery_window_in_days = var.waiting_deletion_period
   
     tags = {
       "Name"                 = "dummy-machine-sbx"
       "my:environment"       = "sandbox"
       "my:owner"             = "Luthfi"
     }
   }
   
   resource "aws_secretsmanager_secret_version" "dummy_machine_sbx" {
     secret_id     = aws_secretsmanager_secret.dummy_machine_sbx.id
     secret_string = jsonencode(var.dummy_machine_user_sbx_credentials)
   }
   

5. Next commit and push terraform codes to the repository. After that run Terraform plan and Terraform apply from Terraform Cloud workspace so that infrastructures or resources can be provisioned. This activity will not be explained in detail and I will only show the simulation. If you are interested in how to configure Terraform Cloud workspace and how to run Terraform plans and apply for the provision of AWS resources, please check out my other blog post here.

   

6. After resource provisioning via terraform has been finished, then we can verify resources have been successfully created via the AWS web console. Go to Secrets Manager service page, then go to menu Secrets. Verify secret that defined in Terraform has been successfully created.

   

7. Next go to details configuration of the secret by clicking secret name mentioned in previous step. In this page, We can see details configuration of secret such as: secret name, secret ARN, encryption key used by secret and secret tags.

   

   

8. In this blogpost, I don’t use automatic rotation for the secret.

   

9. To verify the value of sercet, click Retrieve secret value button as seen in point number 7. Then click Edit button to change the value of AWS_SECRET_ACCESS_KEY.

   

10. Change the value of AWS_SECRET_ACCESS_KEY then click Save.

    

11. Verify value has been updated.

    

12. Next please always make sure that only privileged user that can access the secret.

    

Conclusion

So We have reached the last section of this article. There are some key takeaways that I want to point out:

• To prevent some potential issue regarding security hole and shared secret of API key used by IAM machine, We can keep the API key in centralized way using AWS Secrets Manager.
• Secret configuration in AWS Secrets Manager can be declared using the Infrastructure as Code (IaC) method. So that We can have a state file of infrastructure that has been created.

Please check my GitHub repository to see source code example used in this blogpost.

Please comment if you have any suggestions, critiques, or thoughts.

Hope this article will benefit you. Thank you!