DEV Community

Cover image for Challenge: Display EC2 instances from all regions using Python and Boto3
AWS Community Builders profile image Sri
Sri for AWS Community Builders

Posted on

Challenge: Display EC2 instances from all regions using Python and Boto3

#aws #boto3 #ec2 #lambda

This was an idea from the AWS Certified Global Community to create challenges by breaking a certain part of code, so the users can learn by fixing the code.

I have written this second challenge for the AWS Certified Global Community and have also used it to mentor some of my mentees.
I thought that it might be a good idea to share this with the broader community as well.

Boto3 is the name of the Python SDK for AWS. It allows you to directly create, update, and delete AWS resources from your Python scripts.

I agree that we have EC2 Global View, which shows what we are trying to achieve. However this is more of a programming challenge on how to code using Python and Boto3.

Let's start the challenge!

Pre-Requisites:

  • Lambda with Python 3.9
  • Lambda Execution role with access to EC2 and CloudWatch access (IAM)
  • It is preferable to have at least one EC2 instance in any of the regions (not mandatory though)

Step 1. Create an IAM Policy:

You could either let Lambda create an execution role and add the AmazonEC2ReadOnlyAccess policy or you could create the following IAM policy and a role.

Note: Replace [Region], [LambdaFunctionName] and [AccountID] in the Policy

IAM Policy with the following access:

AWSLambdaBasicExecution

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:[Region]:[AccountID]:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

AmazonEC2ReadOnlyAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:Describe*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:ListMetrics",
                "cloudwatch:GetMetricStatistics",
                "cloudwatch:Describe*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:Describe*",
            "Resource": "*"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Step 2. Create a role for Lambda and attach the policy created in Step 1

Trusted entities:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode

Step 3. Broken Code: Create a Lambda function with the following code and execute the function.

Note: Please increase the Lambda Function Timeout under General Configuration from 3 sec to 2 min or more as required.

import json

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:

        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']

        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()

        for reservation in response["Instances"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }
Enter fullscreen mode Exit fullscreen mode

Note: DO NOT forget to deploy the Lambda function every time you make a change.

First Error: We get the following error when we execute the code❓

Response
{
  "errorMessage": "name 'boto3' is not defined",
  "errorType": "NameError",
  "requestId": "6926f216-e0e8-4f25-bb16-761ada972e3f",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 8, in lambda_handler\n    ec2 = boto3.client('ec2')\n"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Hint 1:
The first one is easy to solve by executing the code in Lambda, the hint appears in the error message.

Second Error: We get the following error after fixing the first one❓

Response
{
  "errorMessage": "'Instances'",
  "errorType": "KeyError",
  "requestId": "c78a6211-1d73-4d72-b83a-4c51bfd4e181",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 21, in lambda_handler\n    for reservation in response[\"Instances\"]:\n"
  ]
}
Enter fullscreen mode Exit fullscreen mode

Hint 2:

Please refer to the Response Syntax of describe_instances(**kwargs) at Boto3 Documentation

Step 4. Final Code: ✅

import json
import boto3

ec2list = []

def lambda_handler(event, context):

    # Get list of regions
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions().get('Regions',[] )

    # Iterate over regions
    for region in regions:

        print ("* Checking region  --   %s " % region['RegionName'])
        reg=region['RegionName']

        client = boto3.client('ec2', region_name=reg)
        response = client.describe_instances()

        for reservation in response["Reservations"]:
            for instance in reservation["Instances"]:
                print ("  ---- Instance %s in %s" % (instance['InstanceId'], region['RegionName']))
                ec2list.append(instance['InstanceId'])

    return {
        "statusCode": 200,
        "body": ec2list
    }
Enter fullscreen mode Exit fullscreen mode

Output

{
  "statusCode": 200,
  "body": [
    "I-[instanceid]"
  ]
}

START RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Version: $LATEST
* Checking region  --   eu-north-1
* Checking region  --   ap-south-1
* Checking region  --   eu-west-3
* Checking region  --   eu-west-2
* Checking region  --   eu-west-1
* Checking region  --   ap-northeast-3
* Checking region  --   ap-northeast-2
* Checking region  --   ap-northeast-1
* Checking region  --   sa-east-1
* Checking region  --   ca-central-1
* Checking region  --   ap-southeast-1
* Checking region  --   ap-southeast-2
* Checking region  --   eu-central-1
* Checking region  --   us-east-1
---- Instance [instanceid] in us-east-1
* Checking region  --   us-east-2
* Checking region  --   us-west-1
* Checking region  --   us-west-2
END RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd
REPORT RequestId: f6a1c381-51e2-43c9-8490-a62b8a6394dd Duration: 15667.85 ms Billed Duration: 15668 ms Memory Size: 128 MB Max Memory Used: 90 MB Init Duration: 246.67 ms
Enter fullscreen mode Exit fullscreen mode

Note: The actual instance id is replaced with [instanceid] in the output.

So what have I done to break the code

  • I removed import boto3 at line no 2
  • Replaced response["Reservations"]: with response["Instances"]: at line no 21

Hope you liked the challenge! Please let me know your thoughts in the comments section.

Tip: Please do check out the reference link as you will learn something about global services too with Boto3. 👇

See you next time 👋

References:

I learnt this from stackoverflow when I asked a question about Retrieve list of buckets from all regions using boto3

Hope you have also learnt something about retrieving S3 buckets from all regions using Boto3.

Top comments (0)

Read next

ianbrumby profile image

Finally got some time to play with the new JSONata and Variables support for Step Functions, and I have to say, it is massive improvement. Check out my latest blog post, where I walk through a simple example of how easy it is to handle pagination now

Ian -

094459 profile image

Amazon Q Developer Tips: No.20 Amazon Q Developer Agents - /review

Ricardo Sueiras -

briansuarezsantiago profile image

Terraform vs AWS CDK: ¿Qué herramienta de infraestructura como código es mejor para tu proyecto?

Brian -

sudo_anuj profile image

Scaling to Zero with Amazon Aurora Serverless v2

Anuj Tyagi -