CloudFormation stack creation using Python

CloudFormation stack can be created from AWS Console, AWS CLI or using many other ways. We can also automate the creation of the CloudFormation stack using AWS CLI, CodePipeline etc. In this section of my blog, I am going to introduce how to use an AWS SDK for Python to automate the CloudFormation Stack creation. Here I am assuming that you know the basic Python and have understanding of the AWS SDK for Python. I would suggest to test the below scripts with small stack first and further you can customise the Python script for your own requirements.

Follow the steps below to automate the Cloudformation stack creation.

1. Make sure you have the latest Python installed on your box where you are intending to run the Python Script. In case of Mac it comes with default installed Python. I have upgraded it to the latest version.
   

   rimpaljohal@MacBook-Pro Blog-Contents % Python --version
   Python 3.9.6
   
   

2. Get hold of the AWS SDK for Python and installed it on the box where you are going to execute the Python Script.

   AWS SDK for Python (Boto3)

3. You need two files for automate creation of CloudFormation stack.
   CFNStackCreation.py --> Your Python Script
   Parameter.json --> Your Parameter file

4. Keep these two files in the same directory path from where you are going to execute the Python script. For example
   

    rimpaljohal@MacBook-Pro Rimpal % ls CFNStackCreation.py Parameter.json 
   CFNStackCreation.py Parameter.json
   

5. Python Script below for reference
   

   #-- Import modules
   import sys
   import os.path
   import json
   import time
   import boto3
   
    #-- Functions
   def check_status( cfn_client_ss, cfn_stack_name_ss ):
   stacks_ss = cfn_client_ss.describe_stacks(StackName=cfn_stack_name_ss)["Stacks"]
   stack_ss_val = stacks_ss[0]
   status_cur_ss = stack_ss_val["StackStatus"]
   print ("Current status of stack " + stack_ss_val["StackName"] + ": " + status_cur_ss)
   for ln_loop in range(1, 9999):
       if "IN_PROGRESS" in status_cur_ss:
           print ("\rWaiting for status update(" + str(ln_loop) + ")...",)
           time.sleep(5) # pause 5 seconds
   
           try:
               stacks_ss = cfn_client_ss.describe_stacks(StackName=cfn_stack_name_ss)["Stacks"]
           except:
               print (" ")
               print ("Stack " + stack_ss_val["StackName"] + " no longer exists")
               status_cur_ss = "STACK_DELETED"
               break
   
           stack_ss_val = stacks_ss[0]
   
           if stack_ss_val["StackStatus"] != status_cur_ss:
               status_cur_ss = stack_ss_val["StackStatus"]
               print (" ")
               print ("Updated status of stack " + stack_ss_val["StackName"] + ": " + status_cur_ss)
       else:
           break
   
   return status_cur_ss
   #-- End Functions
   
    #-- Main program
   def main(access_key_ss, secret_key_ss, param_file_ss):
   
   #-- Confirm parameters file exists
   if os.path.isfile(param_file_ss):
       json_data_ss=open(param_file_ss).read()
   else:
       print ("Parameters file: " + param_file_ss + " is invalid!")
       print (" ")
       sys.exit(3)
   
   print ("Parameters file: " + param_file_ss)
   parameters_data_ss = json.loads(json_data_ss)
   region_ss = parameters_data_ss["RegionId"]
   
   #-- Connect to AWS region specified in parameters file
   print ("Connecting to region: " + region_ss)
   cfn_client_ss = boto3.client('cloudformation', region_ss, aws_access_key_id=access_key_ss, aws_secret_access_key=secret_key_ss)
   
   #-- Store parameters from file into local variables
   cfn_stack_name_ss = parameters_data_ss["StackName"]
   print ("You are deploying stack: " + cfn_stack_name_ss)
   #-- Check if this stack name already exists
   stack_list_ss = cfn_client_ss.describe_stacks()["Stacks"]
   stack_exists_ss = False
   for stack_ss_cf in stack_list_ss:
       if cfn_stack_name_ss == stack_ss_cf["StackName"]:
           print ("Stack " + cfn_stack_name_ss + " already exists.")
           stack_exists_ss = True
   
   #-- If the stack already exists then delete it first
   if stack_exists_ss:
       user_response = input ("Do you want to delete or update the stack")
       print ("Calling Delete Stack API for " + cfn_stack_name_ss)
       cfn_client_ss.delete_stack(StackName=cfn_stack_name_ss)
   
       #-- Check the status of the stack deletion
       check_status(cfn_client_ss, cfn_stack_name_ss)
   
   print (" ")
   print ("Loading parameters from parameters file:")
   fetch_stack_parameters_ss = []
   for key_ss in parameters_data_ss.keys():
       if key_ss == "TemplateUrl":
           template_url_ss = parameters_data_ss["TemplateUrl"]
       elif key_ss == "StackName" or key_ss == "RegionId":
           # -- Do not send as parameters
           print (key_ss + " - "+ parameters_data_ss[key_ss] + " (not sent as parameter)")
       else:
           print (key_ss + " - "+ parameters_data_ss[key_ss])
           fetch_stack_parameters_ss.append({"ParameterKey": key_ss, "ParameterValue": parameters_data_ss[key_ss]})
   
   #-- Call CloudFormation API to create the stack   TemplateBody='', 
   print (" ")
   print ("Calling CREATE_STACK method to create: " + cfn_stack_name_ss)
   
   status_cur_ss = ""
   
   result_ss = cfn_client_ss.create_stack(StackName=cfn_stack_name_ss, DisableRollback=True, TemplateURL=template_url_ss, Parameters=fetch_stack_parameters_ss, Capabilities=["CAPABILITY_NAMED_IAM"])
   print ("Output from API call: ")
   print (result_ss)
   print (" ")
   
   #-- Check the status of the stack creation
   status_cur_ss = check_status( cfn_client_ss, cfn_stack_name_ss )
   
   if status_cur_ss == "CREATE_COMPLETE":
       print ("Stack " + cfn_stack_name_ss + " created successfully.")
   else:
       print ("Failed to create stack " + cfn_stack_name_ss)
       sys.exit(1)
   
   #-- Call Main program
   if __name__ == "__main__":
   if len(sys.argv) < 4:
       print ("%s:  Error: %s\n" % (sys.argv[0], "Not enough command options given"))
       print ("Argument 1 (required): AWS Access Key ")
       print ("Argument 2 (required): AWS Secret Access Key ")
       print ("Argument 3 (required): Stack Parameters JSON file ")
       print (" ")
       sys.exit(3)
   else:
       access_key_ss = sys.argv[1]
       secret_key_ss = sys.argv[2]
       param_file_ss = sys.argv[3]
   
   main(access_key_ss, secret_key_ss, param_file_ss)
   
   

6. Parameter json file below for reference. This is the template parameter json file and one can customise it as per the cloudformation build requirement. I am calling master stack here through this parameter file. Master stack is having the nested stacks to create VPC, Subnets(private and public, Database Subnets), application ec2 instances with autoscaling and Load Balancer.
   

    {
   "RegionId": "us-east-1",
   "TemplateUrl": "https://blogbucket.s3.amazonaws.com/masterstackv1.yml",
   "StackName": "nonprodstack",
   "EnvironmentType": "dev",
   "VpcCIDR": "10.192.0.0/16",
   "PublicSubnet1CIDR": "10.192.10.0/24",
   "PublicSubnet2CIDR": "10.192.11.0/24",
   "PublicSubnet3CIDR": "10.192.12.0/24",
   "PrivateSubnet1CIDR": "10.192.20.0/24",
   "PrivateSubnet2CIDR": "10.192.22.0/24",
   "PrivateSubnet3CIDR": "10.192.24.0/24",
   "DatabaseSubnet1CIDR": "10.192.21.0/24",
   "DatabaseSubnet2CIDR": "10.192.23.0/24",
   "DatabaseSubnet3CIDR": "10.192.25.0/24",
   "LatestAmiID": "/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2",
   "ALBAccessCIDR": "0.0.0.0/0",
   "ServerInstanceType": "t2.micro",
   "WebAsgMax": "4",
   "WebAsgMin": "3"
   }
   

7. Once the required two files(Python script and the Parameter file) are saved in one directory then open the terminal and navigate/go to the directory which is having two mentioned files.

8. Execute the python scripts as given below
   

      rimpaljohal@MacBook-Pro Rimpal % python CFNStackCreation.py AU************* SL*********************** Parameter.json
   

   Here you need to give AWS Access Key and AWS Secret Access Key of your AWS account.

9. As the script progress, you can see the progress on the Mac terminal.

Above defined steps will help you to deploy the CloudFormation stack using python script.

Happy Coding!!