The Dummies Guide to AWS KMS.

Introduction¶

This is the article you have been waiting for to gain knowledge on security, encryption, cipher text. All these topics puts a scare in a lot of us. You have always wished if somehow these topics just don't cross your road. Just like the black cat.

I am going to explain you these topics with interesting visuals, charts and info graphics. This will help you to hold a conversation with your colleague when such scary topics are being discussed.

In Short, if you want to score few marks of these topics in AWS Certification exams. If you want to have a conversation with your team mates next time, when these topics come up. This is the article you have been waiting for.

AWS provides a managed service which does a lot of heavy lifting for us. This service is like Rubeus Hagrid, the guide to novice Harry porter, though the woods of The Forbidden Forest.

This service provides a mechanism to encrypt and decrypt data, 2 of the most important task in encryption. It also helps us with envelope encryption. It can also generate the data keys for you.

This service is everything you wished for when you wanted to make your application secure and your data encrypted. This service also automatically integrates with lots of other AWS service, making it easier to encrypt the data.

Lets jump right in.. to the mystical world of AWS KMS.

What is encryption?¶

Before you get into the journey to understanding the AWS provided service. You should understand what is encryption in its simplest form.

As mentioned by Wikipedia, Encryption is

Encryption is the process of encoding information. This process converts the original representation of the information, known as plaintext , into an alternative form known as ciphertext.

You can consider it like this, there is a black box, into which we pass a plain text and get an encrypted text, which no one can understand. This helps in keeping the data secure, if we delete the plain text now, no one can decode the encrypted text back.

The process to get back the original plaintext differs on the basis of the encryption used.

There are two types of encryption used.

• Symmetric Encryption
• Asymmetric Encryption

The name derives from the fact that if they are using the same key to encrypt or decrypt the data.

Symmetric Encryption¶

In symmetric encryption, we use the same key to encrypt and decrypt the data.

Consider your home lock, it opens and closes with the same key. This is same as symmetric encryption.

Asymmetric Encryption¶

In Asymmetric Encryption, we have to encrypt the data with a public key which is known to everyone, but the decrypt happens with a specific private key known only the authorized person. In this case, both the keys are different.

Consider the bank locker, to open the locker, you need your key, and the bankers key. You cannot open with just one key. Asymmetric encryption is like the bank locker.

AWS KMS Definition¶

As per AWS, a KMS is

AWS Key Management Service (AWS KMS) is a managed service that makes it easy for you to create and control customer master keys (CMKs), the encryption keys used to encrypt your data.

This is the most apt definition for AWS KMS you will ever find. Trust me, I have searched. AWS KMS creates and controls the customer master keys (CMKs) only. For you to understand this is the master key which you have, and it can open any lock for you. This is the skeleton key for you.

If you notice carefully in the definition, it does not mention about the storing of keys, it just mentioned create and control. As you go further in this article you will understand why is that the case.

You might be getting an inkling, why is AWS KMS important after reading about the definition, encryption and its type. You can store a lot of data on AWS, using various services, namely AWS S3, RDS, EBS, etc. To keep the enterprise data on public platforms in a plain text form is not advisable.

So let's jump into the world of AWS KMS, by learning about some key concepts.

Customer Master Keys (CMKs)¶

Customer Master Keys (CMKs) are the core of AWS KMS. This is a logical representation of a master key. It is assumed that a CMKs have these metadata.

• Key ID
• Creation Date
• Description
• Key State
• Key Materials

You cannot see all these metadata upfront. CMKs are used to encrypt or decrypt up to 4KB of data. As there are different types of encryption, we have corresponding types of CMKs.

• Symmetric CMKs
  • A 256 byte key, which is used for both encryption and decryption.
• Asymmetric CMKs
  • represent the RSA Key Pair.

The Symmetric CMKs keys, and the private key of Asymmetric CMKs never leave the AWS KMS unencrypted.

The CMKs provide APIs for various programming languages, like Boto3 for Python, which can use the AWS KMS Api's to get the task done, in place of the actual physical key materials which is not visible in AWS KMS.

As a user we only have to option to delete the CMKs but not modify any of the key metadata.

AWS KMS supports 3 different type of CMKs.

• Customer Managed CMKs
• AWS Managed CMKs
• AWS Owned CMKs

AWS services manage to use the above 3 types of CMKs in different manner, some use only the Customer Managed CMKs, some use only AWS Managed CMKs or AWS Owned CMKs, and some gives the flexibility of all 3.

Customer Managed CMKs¶

These are the CMKs which are created, owned and managed by you. These are under your full control.

You can find the customer managed CMKs in the AWS Console. Just remember that Customer Managed CMKs has a monthly fee even in the free tier. A word of advice, after learning about customer managed CMKs, please delete the keys, not disable it.

AWS Managed CMKs¶

These are the CMKs which are automatically used by some AWS services when we decide to encrypt the data. These are completely in control of AWS, and you as a user have no control on these CMKs.

AWS Managed CMKs are free to use in free tier. You may have to pay for excess usage.

Data Keys¶

The Keys used to encrypt data, is called a Data Keys. CMKs are used to generate, encrypt and decrypt the data keys. Please note, we are using CMKs to generate, encrypt and decrypt the data keys and not the data itself.

AWS KMS does not take the responsibility of storing, managing or tracking your data keys.

Create a data key¶

When you use the AWS KMS APIs, for encryption, the generation of data keys happens automatically. You can explicitly create a data key by calling the API, GenerateDataKey.

The API returns 2 things.

• Plain Text copy of the data key.
• The data Key encrypted with the CMKs.

Encrypt a data key¶

AWS KMS does not have a mechanism for you to encrypt the data key, You can use the OpenSSL library to encrypt the data with Data key, or use the AWS Encryption SDKs.

The plain text data key generated above, can be used to encrypt the data using the OpenSSL Library or the encryption SDKs. The encrypted data can now be stored safely, but delete the plain text data key as soon as possible.

Decrypt a data key¶

Now you have stored the encrypted data, at some point you want to decrypt the data, You can use the decrypt API, which will decrypt the data key, and returns the plain text data key. This plain text data key can now be used to decrypt the data.

Envelope encryption¶

By definition, Envelope Encryption is,

Envelope Encryption is a practice where the data is encrypted with a key (data key), and then the key (data key) in turn is again encrypted.

To understand the above statement more clearly, you can take the analogy of when you leave your house for a long time, you lock your important document and valuable in a safety lock. You then keep the keys to this safety lockers inside another room or lock. Finally, you lock the front of your door and leave.

In above case the data (your valuable) & your key (safety lockers) are in the same place (your home), but you have the master key to the house (CMKs). This helps you leaving your house in peace.

In AWS KMS, Customer Master Keys (CMKs) are the master keys, which never leave AWS KMS unencrypted.

You can create a chain of such encryption cycle, and the reason why envelope encryption works is for this reason.

• Encrypting Data is a slow process, but encrypting just the data key is a fast process.
• Encryption of data keys, gives an added layer of security for you, and it helps you storing the encrypted data keys along with the data.
• Different encryption algorithms can be used to encrypt multiple keys in each layer of the envelop.

Create A CMKs on AWS¶

You are now well versed with the major concepts in AWS, now lets dive into how to create a CMKs and its usage.

One thing you should note that a lot of old blogs mentions the AWS KMS as an option under AWS IAM, it is not the case now. AWS KMS is an independent service.

The first step in AWS KMS is to create a CMKs. Creation of CMKs is a five step process.

1. Configure Key ⚙️
2. Add Label 🏷️
3. Define key administrative 🚶
4. Define Key usage permissions 👨‍💻
5. Review 🔍

Lets get you a AWS KMS CMKs.

Create CMKs | Step 01 | Configure Key ⚙️¶

Here is the screen for your reference.

You learned about Symmetric encryption & Asymmetric encryption, in the topic above What is encryption?.

You have chosen one of those options for your own CMKs, lets keep is simple and choose Symmetric encryption, as you will use the same key to encrypt and decrypt data.

The default Advanced Option is selected as KMS, you can keep the same. The other 2 option are for giving your own Keys to encrypt the data or take it from CloudHSM.

Create CMKs | Step 02 | Add Label 🏷️¶

Let me take you through this option one at a time.

You have to give an Alias first, this is the name you will use to reference this key in your code, or for any other service. Give a good name and a description of the Key. You cannot go to the next step without giving the Alias name.

The next option is to give a Tag name, you can provide a key/value pair which you can easily reference in code or in service to identify the key in addition to the name you provided.

Create CMKs | Step 03 | Define key administrative 🚶¶

Once you are done with naming your keys, you have to find out off all the IAM user's or Roles who can be the Admin of these keys.

AWS KMS CMKs administrators are those people who can administer the keys, but they cannot use the key for cryptographic operation, which make it important for the next step.

Select the Administrator User or role, who will not be using it for cryptographic operation, and lets go to the next step.

Create CMKs | Step 04 | Define Key usage permissions 👨‍💻¶

You have the administrators of your AWS KMS CMKs, now lets get one of the IAM role or user be selected as the user of the key. These selections enable this user with two extra policies.

1. Policy to use the CMK directly.
2. Use the CMKs with the AWS Services.

Create CMKs | Step 05 | Review 🔍¶

The final step for you in most of the AWS configuration is Review. Check all the details, and see the key policy document. What ever we discussed in the last two steps will now make sense to you.

You analyze the key policy mentioned below, and understand what are the special permission given to the administrators and the user.

{
    "Id": "key-consolepolicy-3",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow access for Key Administrators",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/XYZ"
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:TagResource",
                "kms:UntagResource",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/ABC"
            },
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Allow attachment of persistent resources",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:user/ABC"
            },
            "Action": [
                "kms:CreateGrant",
                "kms:ListGrants",
                "kms:RevokeGrant"
            ],
            "Resource": "*",
            "Condition": {
                "Bool": {
                    "kms:GrantIsForAWSResource": "true"
                }
            }
        }
    ]
}

• administrators
  • The administrators is XYZ in the above policy, check carefully what you see.
  • These administrators are given the Actions to most of the AWS KMS administrators an allow effect.
  • This administrators do not have any encryption, decryption or generate key action associated with it.
• user.
  • The user is ABC in the above policy,
  • The user as you see is granted the Action of encrypt, decrypt, generate and other key related uses with an allow effect.
  • The user also has an addition allow effective on the AWS KMS Grants, which is mostly used by AWS Services when encrypting data at rest.

This makes you the owner of a CMK. Please keep this in mind, the AWS KMS CMKs are charged, so if you are finished learning delete the key immediately, and I would like to remind you, do not disable the key, delete the key.

AWS KMS API¶

When you have your CMKs configured, you obviously want to use it. You should wait a bit before using it. Let me explain to you a few of the AWS KMS APIs.

There are many APIs in AWS KMS, the full list of AWS KMS api's are mentioned here. We will not go through all of them, but the three most important once, which you can use in a fun manner in the next section.

• GenerateDataKey - Returns a plain text and a cipher text version of a data key.
• Encrypt - Encrypt the plain text using a CMKs
• Decrypt - Decrypt the cipher text which was encrypted with the Encrypt API.

The name is self describing but still let me explain you the little fundamental about the APIs.

GenerateDataKey¶

When you are using an SDKs, you generally do not need to use the GenerateDataKey, because you can use the Key Id generated above.

This API generates a symmetric data key for you, which you can use for client side encryption.

It generates two keys for you.

• Plain text copy of the data key
• encrypted copy of the data key with the CMKs.

Once the client side encryption of data is done and stored, the plain text copy of the data key can be deleted and the encrypted copy of the data key can be kept along with the encrypted data.

GenerateDataKey always returns a unique data key on each call.

There are different variations of the same API, which you can consider to use.

• GenerateDataKeyWithoutPlaintext - Which will only generate the encrypted copy of the data key.
• GenerateDataKeyPair - Generates the Asymmetric data key.

Encrypt¶

Once you have the Key id generated, you can use the encrypt API to encrypt your data. This API just encrypts the plain text data into ciphertext using the CMKs we generated.

This API has two primary use cases handled.

• Encrypt small amount of data, like a database password.
• It can be used to move encrypted data from one Region to another.

Decrypt¶

The name itself tells you, it will decrypt the data we have encrypted till now. We do not need to pass the Key Id in the API, if we are symmetric encryption, which is our case anyway.

AWS KMS API in action with Boto3¶

Finally the code which uses the Boto3 SDK to do all the good things for you.

import boto3
kms = boto3.client('kms')
key_id = 'alias/test' # this should be present in the KMS
database_password = 'Lorem-ipsum-dolor-sit-amet.'
result = kms.encrypt(KeyId=key_id, Plaintext=database_password) # result will now have these fields
# ChiphertextBlob - encrypted data
encrypted_password = result['ChiphertextBlob']
decrypt_result = kms.decrypt(ChiphertextBlob=encrypted_password) # will have the password decrypted
decrypt_result['Plaintext']

• key_id = 'alias/test'
  • this is the key id you created the label., you have to add alias/ before the actual label.
• result = kms.encrypt(KeyId=key_id, Plaintext=database_password)
  • You will call the encrypt() API, passing the key id, and the text which has to be encrypted.
  • This API is only used to encrypt small amount of data.
• encrypted_password = result['ChiphertextBlob']
  • You can retrieve the encrypted password which is mapped with a key value of ChiphertextBlob in the result.
• decrypt_result = kms.decrypt(ChiphertextBlob=encrypted_password)
  • You can decrypt, the encrypted_password using the decrypt api.
  • You were using symmetric encryption, this is the reason you are not passing the key id.
• decrypt_result['Plaintext']
  • You can retrieve the plain text password back from the key value of Plaintext

The above is a very simplistic example of the use of Boto3 for encrypt & decrypt APIs. You can explore further once you have completed the AWS certification. For now the understanding of these three important APIs is enough.

Conclusion¶

You might have always wondered about cloud computing, How does the data in the cloud is secured? Though there are multiple service are involved in AWS to provide that data security.

You came to know about one of the building blocks in that data security infrastructure.

You understood that encryption is the way of encoding information with a key so that you can transmit data securely. You also found that you can use the same or a different key for encrypting and decrypting.

AWS KMS is a service which provides a way to create Customer master Keys (CMKs), which can be made by you, or AWS or owned by AWS. This key is the master keys, which you can use the data key or small plain text data.

When encrypting large chunks of data, we cannot use the CMKs, but we need something called data keys, which can be generated using the CMKs we created. You can use these data keys to encrypt the data, and deleted the plain text data key. You can store the encrypted data keys with the data.

AWS KMS provide different APIs to for you to generate the data key. AWS KMS does not give the option to encrypt and decrypt the data, You have to use the OpenSSL library, or the AWS Encryption SDKs. AWS KMS does not own the responsibility of storing the data key, you are the owner to store it.

In encryption you also understood envelope encryption, where you can use multiple keys to encrypt the data. You can use the combination of symmetric and asymmetric encryption to encrypt the data.

You followed the 5 step process to create an AWS KMS CMKs, You also found out about the three AWS KMS API, GenerateDatakey, Encrypt and Decrypt. You also used the AWS Boto3 SDK for Python to create a sample understanding of the process to encrypt and decrypt data.

Overall you have now a fair understanding of the AWS KMS. You can further enhance the reading of AWS KMS, by following the various articles provided in reference.

Reference¶

• Photo by Ales Nesetril on Unsplash
• AWS KMS​ - Key Management Service
• AWS KMS - Encrypt & Decrypt DEMO | KMS pricing | KMS Key Rotation (Part 2)
• What is AWS Key Management Service?
• Encryption
• What types of encryption are there?
• AWS Key Management Service concepts
• Using grants
• What is the purpose of kms:GenerateDataKey in AWS?

Let me know if you used the AWS Boto3 SDK to encrypt and decrypt data. How was your experience, if you could not encrypt or decrypt the data, please give the error in the comment and I will try to help you.