What is AWS CDK?
AWS CDK is an opensource software development framework to create and deploy the cloud infrastructure using your familiar languages like Python. AWS CDK make the infrastructure management fun and easy 🕺
Manual infrastructure Deployment
Let’s have a look into the process of deploying infrastructure manually. Typically we use the management console to create and deploy resources but sometimes we use CLI tools. For instance, to create a S3 bucket you log into the AWS account, click Create Bucket
and fill the form. In addition to creation of resources, you have to manually create the bucket policy, ACL and IAM policy in order to integrate to other components in the application. This is fine to begin with and when the infrastructure requirements are minimal. When it comes to enterprise level infrastructure management where you have to manage multiple applications with multiple stacks that has many resources the manual process of deployment is not reliable nor consistent. It is error prone and maintenance of application infrastructure becomes nightmare.
From this example, we can understand that deploying and managing infrastructure manually for a fairly complex application is not a viable solution.
Infrastructure as Code (IaC)
Infrastructure as Code is the process of provisioning and managing the resources programmatically through machine readable definition files applying the same rigor as the application code development. For AWS the Infrastructure as Code is implemented using AWS CloudFormation. With AWS CloudFormation the application resources can be provisioned by writing the definition of resources in a cloud formation template either in YAML or JSON. For example, the following YAML template describes AWS S3 bucket to create.
AWSTemplateFormatVersion: 2010-09-09
Resources:
S3Bucket:
Type: 'AWS::S3::Bucket'
Properties:
AccessControl: PublicRead
WebsiteConfiguration:
IndexDocument: index.html
ErrorDocument: error.html
DeletionPolicy: Retain
BucketPolicy:
Type: 'AWS::S3::BucketPolicy'
Properties:
PolicyDocument:
Id: MyPolicy
Version: 2012-10-17
Statement:
- Sid: PublicReadForGetBucketObjects
Effect: Allow
Principal: '*'
Action: 's3:GetObject'
Resource: !Join
- ''
- - 'arn:aws:s3:::'
- !Ref S3Bucket
- /*
Bucket: !Ref S3Bucket
Cloud formation YAML template
AWS CloudFormation takes this template and will create, update and delete resources on your AWS account according to the description in the template.
Benefits of IaC:
- Visibility - The template act as singe source of reference for the resources. Web console is not required.
- Stability - Version controlled and rollback to previous stable version incase new changes break the application.
- Scalability - Reusable and scales horizontally seamlessly.
How CDK works?
CDK is built on top of the AWS CloudFormation service and uses it as the engine for provisioning resources. AWS CDK uses the familiarity and expressive power of favorite programming languages for modelling your applications. The AWS CDK supports TypeScript, JavaScript, Python, Java, C#/.Net, and (in developer preview) Go. Developers can use their favorite programming languages to define reusable components called Constructs and compose several Constructs into Stacks and Apps.
Image credit: https://docs.aws.amazon.com/cdk/v2/guide/home.html
CDK Constructs
Constructs are basic building blocks of AWS CDK apps. Constructs are high level components that preconfigure cloud resources with proven defaults to make the infrastructure provisioning easier. AWS CDK provides a library of constructs that cover many AWS features and services. Developers can define the application infrastructure at high level using these constructs. Constructs are adjustable and composable and one can easily change the parameters and build custom constructs.
AWS Constructs library
The CDK constructs library includes all the resources that are available in AWS. For example, s3.Bucket
class represents an AWS S3 bucket and dynamodb.Table
represents an AWS DynamoDB table.
There are three level of constructs in this library,
- L1 Constructs are the low level constructs that directly represents the resources in AWS CloudFormation. It is also called CFN Resources.
- L2 Constructs are the next higher level constructs that are similar in function to L1 constructs but provide defaults, boiler plate and glue logic.
-
L3 Constructs which are called as patterns are designed to accomplish common tasks in AWS For example,
aws-apigateway.LambdaRestApi
construct represents an API Gateway backed by AWS Lambda function.
from aws_cdk import App, Stack
import aws_cdk.aws_s3 as s3
from constructs import Construct
class HelloCdkStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)
s3.Bucket(self, "MyFirstBucket", versioned=True)
app = App()
HelloCdkStack(app, "HelloCdkStack")
CDK Stacks and Apps
The smallest unit of deployment in AWS CDK is called a stack. All of the resources defined within the scope of a stack is deployed as a single unit. You can define any number of stacks inside your AWS CDK app.
As an example, here we declare a stack class MyFirstStack
that includes a single Amazon S3 bucket. However, this only declares a stack and the stack have to defined in some scope in order to be deployed.
class MyFirstStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs):
super().__init__(scope, id, **kwargs)
s3.Bucket(self, "MyFirstBucket")
app = App()
MyFirstStack(app, 'stack1')
MySecondStack(app, 'stack2')
app.synth()
CDK Toolkit
The AWS CDK Toolkit known as cdk
is the command line tool is the primary tool to provision and manage AWS CDK app. It executes the app, generates and deploys the AWS CloudFormation templates.
Setup
- AWS CLI - Make sure you have the latest version of the AWS CLI installed on your system.
- AWS Account and User - You’ll need access to an AWS account. Create IAM user with programmatic access and attach AdministratorAccess policy to the IAM user.
- Node.js - The AWS CDK uses Node.js (>= 10.13.0, except for versions 13.0.0 - 13.6.0).
- VSCode - IDE
- Python - 3.6 or later.
- AWS CDK Toolkit
npm install -g aws-cdk
cdk --version
CDK commands
Command | Function |
---|---|
cdk ls | Lists the stacks in the app |
cdk synt | Synthesizes and prints the CloudFormation template for the specified stack(s) |
cdk bootstrap | Deploys the CDK Toolkit staging stack; see Bootstrapping |
cdk deploy | Deploys the specified stack(s) |
cdk destroy | Destroys the specified stack(s) |
cdk diff | Compares the specified stack with the deployed stack or a local CloudFormation template |
cdk init | Creates a new CDK project in the current directory from a specified template |
Your first CDK project
Create project directory 📁
mkdir cdk_workshop
cd cdk_workshop
Use cdk init
to create a new Python CDK projects
cdk init sample-app --language python
Activating the Virtualenv. The init script created bunch of code to get started and also created a virtual environment. Activate the virtual environment.
source .venv/bin/activate
Install the Python modules.
pip install -r requirements.txt
Project directory
The application entry point is app.py.
#!/usr/bin/env python3
import aws_cdk as cdk
from cdk_workshop.cdk_workshop_stack import CdkWorkshopStack
app = cdk.App()
CdkWorkshopStack(app, "cdk-workshop")
app.synth()
The constructs are defined in cdk_workshop/cdk_workshop_stack.py
from constructs import Construct
from aws_cdk import (
Duration,
Stack,
aws_sqs as sqs,
aws_sns as sns,
aws_sns_subscriptions as subs,
)
class CdkWorkshopStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)
queue = sqs.Queue(
self, "CdkWorkshopQueue",
visibility_timeout=Duration.seconds(300),
)
topic = sns.Topic(
self, "CdkWorkshopTopic"
)
topic.add_subscription(subs.SqsSubscription(queue))
Synthesize cloud formation template from the CDK app
cdk synth
This will output the following template
Resources:
CdkworkshopQueue18864164:
Type: AWS::SQS::Queue
Properties:
VisibilityTimeout: 300
Metadata:
aws:cdk:path: cdkworkshop/CdkworkshopQueue/Resource
CdkworkshopQueuePolicy78D5BF45:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Statement:
- Action: sqs:SendMessage
Condition:
ArnEquals:
aws:SourceArn:
Ref: CdkworkshopTopic58CFDD3D
Effect: Allow
Principal:
Service: sns.amazonaws.com
Resource:
Fn::GetAtt:
- CdkworkshopQueue18864164
- Arn
Version: "2012-10-17"
Queues:
- Ref: CdkworkshopQueue18864164
Metadata:
aws:cdk:path: cdkworkshop/CdkworkshopQueue/Policy/Resource
CdkworkshopQueuecdkworkshopCdkworkshopTopic7642CC2FCF70B637:
Type: AWS::SNS::Subscription
Properties:
Protocol: sqs
TopicArn:
Ref: CdkworkshopTopic58CFDD3D
Endpoint:
Fn::GetAtt:
- CdkworkshopQueue18864164
- Arn
Metadata:
aws:cdk:path: cdkworkshop/CdkworkshopQueue/cdkworkshopCdkworkshopTopic7642CC2F/Resource
CdkworkshopTopic58CFDD3D:
Type: AWS::SNS::Topic
Metadata:
aws:cdk:path: cdkworkshop/CdkworkshopTopic/Resource
CDKMetadata:
Type: AWS::CDK::Metadata
Properties:
Modules: aws-cdk=1.18.0,jsii-runtime=Python/3.7.3
The template contains the following resources:
- AWS::SQS::Queue - queue
- AWS::SNS::Topic - topic
- AWS::SNS::Subscription - the subscription between the queue and the topic
- AWS::SQS::QueuePolicy - the IAM policy which allows this topic to send messages to the queue
Bootstrapping the environment 💼
For the first time you deploy CDK app into an environment or region, a bootstrap stack has to be installed which includes the needed resources for toolkit operations.
cdk bootstrap
Deploy 🚀
cdk deploy
In the CloudFormation console, you will see that the stack has been created successfully.
cdk destroy
How CDK improve developers experience?
- Infrastructure and code in one place
- Easy to define IAM policy - minimal code and minimal permissions
- Refactoring code
- Snapshot test - Compare the changes to previously successful state
- Fine grained assertion tests - Test specific aspects of cloud formation templates
- Validation tests - Validate rules and checks
Conclusion
AWS CDK is a game changer in Infrastructure as Code. It is a complete developer friendly tool to provision and manage all your cloud infrastructure resources.
References
- https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-aws/infrastructure-as-code.html
- https://docs.aws.amazon.com/cdk/api/v1/python/index.html
- https://docs.aws.amazon.com/cdk/v2/guide/home.html
- https://cdkworkshop.com/
Author
Bismillah Kani is a passionate and multi-faceted AI Scientist and Cloud Architect with a passion for deep learning, machine learning, and artificial intelligence. As an AWS certified professional he is constantly striving for excellence and innovation in the field of AI and Cloud.
Top comments (0)