Protect Client Data Using MongoDB Field Level Encryption With AWS KMS & AWS Lambda

This article is written by Devadyuti Das (Senior Solutions Architect - AWS) and Babu Srinivasan (Senior Partner Solutions Architect - MongoDB)

As the cloud becomes the preferred environment for an increasing number of customers, their system components — including applications, compute resources, storage, networks, and databases — are now operating in a more distributed manner. So in a way, customer data is moving continuously through network, servers, and storage from its origin to target. In this scenario, security and protection of the data is paramount and, hence, encrypting the data is of utmost importance. Encryption is the process of converting data into a coded form to prevent unauthorized access, making it significant in safeguarding sensitive information. Client-side encryption seeks to eliminate the potential for data to be viewed/intercepted by any third party on the above chain of data flow. Client-side encryption ensures that data and files that are stored in the cloud can only be viewed on the client side of the exchange.

In this blog, we'll explore how to enable client-side field level encryption (CSFLE), a feature that lets you encrypt data within your application before it is transmitted to MongoDB on AWS. By enabling CSFLE, you ensure that no MongoDB product can access your data in an unencrypted form.

Let’s look at the key services we are going to use to achieve this.

MongoDB Atlas is a multi-cloud database service from MongoDB. It streamlines the deployment and management of your databases, providing the flexibility required to develop robust and efficient global applications on your preferred cloud providers. MongoDB Atlas facilitates the on-demand deployment and management of databases, ensuring they are available whenever and wherever you need them.

AWS KMS allows you to create, manage, and control cryptographic keys across your applications and AWS services. By combining AWS Key Management System with the MongoDB encrypted storage engine, which automatically encrypts all cluster storage and snapshot volumes at rest, you can add an extra layer of security.

AWS Lambda is a compute service that executes your code in response to events and automatically manages the compute resources, facilitating the rapid development of modern, production-ready, serverless applications. By connecting to a MongoDB Atlas cluster, you can quickly begin building your functional code in any popular programming language.

Let’s look from a high level at what we are going to build from the following architecture.

Solution architecture

The above figure illustrates the architecture diagram for implementing MongoDB CSFLE on AWS infrastructure. The steps outlined below correspond to each numbered section in the diagram:

1. Set up an AWS account.
2. Configure a MongoDB Atlas cluster.
3. Create an AWS KMS master key.
4. Create APIs to enable client-side encryption using AWS Lambda.
5. Create a web application to test CSFLE.

Let’s get started!

Prerequisites

• AWS account
• MongoDB Atlas cluster
• ReactJS

Step 1: Set up an AWS Account

Follow the step by step guide from the AWS docs to sign up for an AWS account and create an administrative user. If you have an existing AWS account and administrative user, please proceed to Step 2.

Step 2: Configure MongoDB Atlas cluster

Sign up for a MongoDB account by following the step by step guide. Then, set up and configure a MongoDB Atlas cluster on AWS.

For this blog, I have used eu-west-1 as the chosen region on AWS. Make sure you use a single region while following the instructions.

Complete the steps of creating an account, deploying a free cluster, creating a database user, and configuring network access.

During the ‘Manage the IP Access List’ step, we need to allow our code to access the MongoDB Atlas cluster. To do that, go to ‘Network Access’ and click +Add IP Address. For testing our blog, we want our service/application to access the MongoDB database from AWS. We’ll ‘Allow access from Anywhere’ with a time limit of ‘6 Hours’. Confirm once you choose the appropriate options. Please note that this configuration (0.0.0.0/0) is only for demo purposes and is not recommended for any production or production-like environments.

After creating a MongoDB Atlas cluster, copy the whole connection string for MongoDB driver connectivity (see figure below) and keep it safe to use at a later stage. If you forgot to capture this, you can go to Database -> Connect, click Connecting with MongoDB for VS Code, and copy the connection string.

The above figure shows the screenshot of the connect button on the MongoDB Atlas console.

The above screenshot shows the MongoDB connect for VS Code.

The above screenshot shows the connection string for the cluster.

Step 3: Create AWS KMS master key

Step 3a:
Create a new symmetric key in your preferred region (for this blog, we’ll use eu-west-1) by following the official AWS documentation on creating symmetric KMS keys. The key you create is your Customer Master Key. Choose a name and description that helps you identify it; these fields do not affect the functionality or configuration of your key.

In the Usage Permissions step of the key generation process, apply the following default key policy to allow Identity and Access Management (IAM) policies to grant access to your Customer Master Key. Replace the placeholder for the account principal with your 12-digit AWS account number, which can be found in the top right corner of your AWS console.

{
   "Id": "key-consolepolicy",
   "Version": "2012-10-17",
   "Statement": [
       {
           "Sid": "Enable IAM User Permissions",
           "Effect": "Allow",
           "Principal": {
              "AWS": "arn:aws:iam::<Your AWS account number>:root"
           },
           "Action": "kms:*",
           "Resource": "*"
       }
   ]
}

Record the Amazon Resource Name (ARN) and Region of your Customer Master Key. You will use these in later steps of this guide.

Step 3b:

Now, we will create an IAM user and grant the user permission to use the recently created KMS master key.
Navigate to the AWS IAM Console.

Create a new programmatic IAM user in the AWS Management Console by following the official AWS documentation on adding a user. This IAM user will serve as a service account for your CSFLE-enabled application. Your application will authenticate with AWS KMS using this IAM user to encrypt and decrypt your Data Encryption Keys (DEKs) with your Customer Master Key.

Ensure you record the following IAM credentials in the final step of creating your IAM user: Access key ID, Secret access key. You have one opportunity to record these credentials. If you do not record these credentials during this step, you must create another IAM user.

Step 3c:
Grant your IAM user kms:Encrypt and kms:Decrypt permissions for your remote master key.

The new client IAM user should not have administrative permissions for the master key. To keep your data secure, follow the principle of least privilege.

The following inline policy allows an IAM user to encrypt and decrypt with the Customer Master Key with the least privileges possible:

NOTE: Remote Master Key ARN
The following policy requires the ARN of the AWS KMS key you generated in this step of the guide.

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [ 
               "kms:Decrypt", 
               "kms:Encrypt"
           ]
           "Resource": "<Amazon Resource Name (ARN) of your KMS master key"
       }
   ]
}

To apply the preceding policy to your IAM user, follow the adding IAM identity permissions guide in the AWS documentation.

Authenticate with IAM roles in production
When deploying your CSFLE-enabled application to a production environment, authenticate your application by using an IAM role instead of an IAM user.

To learn more about IAM roles, see the following pages in the official AWS documentation:

• IAM roles
• When to create an IAM role (instead of a user)

Step 4: Create APIs to enable client-side encryption

As for this solution, we want to enable client-side field level encryption completely automated, and the same has a dependency on Automatic Encryption Shared Library of MongoDB. This library is distributed specific to the operating system requirement for the client. In this scenario, we are going to use AWS Lambda in JavaScript for our client, so we’ll need to set up our development environment in Linux-x86_x64 (which is the supported runtime for AWS Lambda).

We have a few choices, like using AWS Cloud 9 or another Linux-based IDE. In this blog, we’ll learn how to set up your AWS Cloud 9 environment for AWS Lambda. The complete development environment set up — storing the MongoDB connection string and AWS KMS resource details, and building and deploying the code — requires a number of tasks to be completed, so we will use an existing GitHub repository to simplify. Please follow the instructions for csfle-service and once completed, come back to this blog to proceed.

After you have successfully deployed the APIs in AWS, let’s try to understand what we have done in the CSFLE service. The code for CSFLE APIs are inside the /mongodb-aws-csfle/csfle-service folder.

In CSFLE service, our objective is to create three APIs for customer profile information with some sensitive personal data to encrypt from the client (e.g., account information, email, date of birth, and phone number).

Let’s take a look at each file and what that is intended to achieve.

1. handler.js: This file holds the main Lambda functions for client-side field level encryption for MongoDB. There are three functions which will be delivered through API gateway.

saveCustomerCSFLE — API function to encrypt customer information like account details, date of birth, email, phone number, etc. before saving or updating in MongoDB using an encryption key

getCustomerWithKey — API function to retrieve customer information with a valid client encryption key

Note: Both the above functions need to initialise a MongoDB client instance with an encryption option parameter.

getCustomerNoKey — API function to retrieve customer information without a valid client encryption key

getCustomerSchema — Defines a schema for customers on what type of encryption will be applied for which specific fields for a customer. In my example, I have encrypted ‘date of birth’, ‘email’, ‘phone number’, and ‘account details’. Also, please note if you try to read a customer document on an attribute which is encrypted in your find method, use the ‘Deterministic’ algorithm and not ‘Random’.

1. secret.js: This contains a module function to call AWS Secrets Manager to fetch the secrets we stored for the MongoDB connection string and KMS key/provider details.

2. encrypt.js: This file holds all the key functions needed for client-side field level encryption as follows.

getKeyManagementProviderDetails — Fetches the KMSProvider and KMS master key details

getDataEncryptionKey — Generates or fetches the data encryption key in/from the keyVault collection in MongoDB based on the existence of the data encryption key

createUniqueIndex — Creates a unique index on keyAltNames attribute

createDataEncryptionKey — Generates the data encryption key in keyVault collection in MongoDB

getEncryptionOption — Collates all additional encryption options including the MongoDB shared encryption library for the MongoClient to execute CSFLE operations

1. mdb.js: This file holds the template for all MongoDB NodeJS driver client functions including creating a MongoClient instance and subsequent CRUD operations like insert, update, and find document(s).

serverless.yaml: This file holds the deployment descriptor for the whole serverless stack, including the Amazon API gateway endpoint definitions along with corresponding AWS Lambda functions.

Environment variables — like the MongoDB collection name, Automated Crypto Shared Library path, NodeJS version, etc.

Permission for API gateway to invoke the Lambda function, decrypt or encrypt using the AWS KMS encryption key and reading AWS Secrets Manager via AWS Identity and Access Management (IAM) policies

Optionally — Amazon Route 53 domain mapping for the API endpoints using Amazon CloudFront distribution

Step 5: Create a web application to test CSFLE

To test the above APIs we have deployed, we’ll create a simple React single page app(SPA). Get the instructions for the prerequisites of creating a Cognito authentication, setting up a development environment, and running the React web app. Once completed, please come back to continue reading this blog.

*Register a new user in CSFLE test web app
*

Open your browser and navigate to http://localhost:3000/.

The above screenshot shows the landing screen of the application.

Click sign in and complete the New User Sign-up using email, password, and confirmation code.

The above screenshot shows the signup/login page.

The above screenshot shows the registration page of the application.

The above screenshot shows the signup confirmation code page.

Once sign-up is completed, you will be redirected to the home page in a signed in state.

*Save a customer document with CSFLE
*

Click on Save Customer (CSFLE).

The above screenshot shows the landing screen of the application.

Complete dummy data for first name, last name, email, date of birth, and phone number and click Save Changes. You will receive an alert that the application is successfully updated.

The above screenshot shows the customer details updated screen of the application.

*Validate the saved document in MongoDB Atlas
*

Go to cloud.mongodb.com and log in to your MongoDB Atlas cluster. From ‘Browse Collections’, navigate to the ‘customer’ collection under the CSFLE database and you will able to see the document you just saved. The encrypted fields are masked with an asterisk (*).

The above screenshot shows the customer details in the MongoDB Atlas cluster.

Validate the saved document from the web app using the key

Open a browser and navigate to https://localhost:3000.

Click on Get Customer (With Key).

The above screenshot shows the landing screen of the application.

You should be able to see the information visible on the page with sensitive fields unencrypted.

The above screenshot show the customer details display screen.

*Validate the saved document from the web app without key
*

Open a browser and navigate to https://localhost:3000.

Click on Get Customer (No Key).

You should be able to see the information visible on the page with the sensitive fields encrypted.

The above screenshot displays the sensitive data in an encrypted format.

Conclusion

In this blog, we have dived deeper into the technical aspects of encrypting data in a client application before we send it over the network to MongoDB Atlas on AWS. However, the best way to learn new technologies and concepts is hands-on, so we recommend going through the GitHub repository thoroughly step by step and learn how to configure and run a dev environment to test client-side field level encryption.
If you’re interested to learn more about how you can use MongoDB Atlas on AWS to drive solutions and innovation for your customers, please read our MongoDB-AWS blogs.