How to schedule RDS instances (start/stop) easily

#aws #rds #database #serverless

A potential game changer in AWS is to schedule when your RDS should be running since every cent that we can save is important. AWS provides some solutions also, but they create new dependencies since they use AWS Dynamo DB, which I wouldn't say is the best idea. That's why I created based on some tutorials a new one based on one Lambda only. Let's check it:

Step 1. Create an IAM policy and role for Lambda

Go to the IAM console, under Access Management in the navigation pane, and choose Policies. Then, click Create policy.

Go to the JSON tab, and enter the following policy code:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
              "rds:DescribeDBClusterParameters",
              "rds:StartDBCluster",
              "rds:StopDBCluster",
              "rds:DescribeDBEngineVersions",
              "rds:DescribeGlobalClusters",
              "rds:DescribePendingMaintenanceActions",
              "rds:DescribeDBLogFiles",
              "rds:StopDBInstance",
              "rds:StartDBInstance",
              "rds:DescribeReservedDBInstancesOfferings",
              "rds:DescribeReservedDBInstances",
              "rds:ListTagsForResource",
              "rds:DescribeValidDBInstanceModifications",
              "rds:DescribeDBInstances",
              "rds:DescribeSourceRegions",
              "rds:DescribeDBClusterEndpoints",
              "rds:DescribeDBClusters",
              "rds:DescribeDBClusterParameterGroups",
              "rds:DescribeOptionGroups"
          ],
          "Resource": "*"
      }
  ]
}

Next, click Review Policy. In the Create Policy section choose a name like rdsstopstart.

After this, choose the option Create policy.

Now that you have the policy ready, you need to create the IAM role. In the navigation pane, choose Roles followed by clicking Create role.

For Select type of trusted entity, choose AWS service. Next, For Common use cases, choose Lambda. Then, choose Next: Permissions.

In the new screen search for and select the policy you created before rdsstopstart.

Go to the last screen and for the Role name, enter rdsLambda. Then click Create role.

Step 2. Create your Lambda function to stop the database

For this section, you need to create one Lambda function that will stop and start the databases. On the Lambda console, choose Functions in the navigation pane and select Create function.

For Function name, enter rds-scheduler.
For Runtime, choose Python 3.12+.
For Execution role, select Use an existing role.
For Existing role, choose the role you created (rdsLambda).

After this, choose to Create function. On the function details page, go to the function code and replace the sample code with the following one:

import boto3
import os

def lambda_handler(event, context):
    region = os.environ['REGION']
    key = os.environ['KEY']
    value = os.environ['VALUE']

    action = event.get('action')

    if action not in ['start', 'stop']:
        print('Invalid action specified. Please provide either "start" or "stop".')
        return

    client = boto3.client('rds', region_name=region)

    def handle_rds_instances():
        response = client.describe_db_instances()
        v_read_replica = []

        for i in response['DBInstances']:
            read_replica = i['ReadReplicaDBInstanceIdentifiers']
            v_read_replica.extend(read_replica)

        for i in response['DBInstances']:
            if i['Engine'] not in ['aurora-mysql', 'aurora-postgresql']:
                if i['DBInstanceIdentifier'] not in v_read_replica and len(i['ReadReplicaDBInstanceIdentifiers']) == 0:
                    arn = i['DBInstanceArn']
                    resp2 = client.list_tags_for_resource(ResourceName=arn)

                    if 0 == len(resp2['TagList']):
                        print('DB Instance {0} is not part of autoshutdown'.format(i['DBInstanceIdentifier']))
                    else:
                        for tag in resp2['TagList']:
                            if tag['Key'] == key and tag['Value'] == value:
                                if action == 'start':
                                    start_instance(i)
                                elif action == 'stop':
                                    stop_instance(i)
                            elif tag['Key'] != key and tag['Value'] != value:
                                print('DB instance {0} is not part of autoshutdown'.format(i['DBInstanceIdentifier']))
                            elif len(tag['Key']) == 0 or len(tag['Value']) == 0:
                                print('DB Instance {0} is not part of autoShutdown'.format(i['DBInstanceIdentifier']))
                elif i['DBInstanceIdentifier'] in v_read_replica:
                    print('DB Instance {0} is a Read Replica.'.format(i['DBInstanceIdentifier']))
                else:
                    print('DB Instance {0} has a read replica. Cannot shutdown & start a database with Read Replica'.format(i['DBInstanceIdentifier']))
            else:
                for tag in resp2['TagList']:
                    if tag['Key'] == key and tag['Value'] == value and i['MultiAZ']:
                        if action == 'start':
                            start_aurora_cluster(i['DBInstanceArn'])
                        else:
                            stop_aurora_cluster(i['DBInstanceArn'])

    def start_instance(instance):
        if instance['DBInstanceStatus'] == 'available':
            print('{0} DB instance is already available'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'stopped':
            client.start_db_instance(DBInstanceIdentifier=instance['DBInstanceIdentifier'])
            print('Started DB Instance {0}'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'starting':
            print('DB Instance {0} is already in starting state'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'stopping':
            print('DB Instance {0} is in stopping state. Please wait before starting'.format(instance['DBInstanceIdentifier']))

    def stop_instance(instance):
        if instance['DBInstanceStatus'] == 'available':
            client.stop_db_instance(DBInstanceIdentifier=instance['DBInstanceIdentifier'])
            print('Stopped DB instance {0}'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'stopped':
            print('DB Instance {0} is already stopped'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'starting':
            print('DB Instance {0} is in starting state. Please stop the cluster after starting is complete'.format(instance['DBInstanceIdentifier']))
        elif instance['DBInstanceStatus'] == 'stopping':
            print('DB Instance {0} is already in stopping state.'.format(instance['DBInstanceIdentifier']))

    def stop_aurora_cluster(cluster_arn):
        """Stops an Aurora cluster identified by ARN if tagged for shutdown."""
        client.stop_db_cluster(DBClusterIdentifier=cluster_arn)
        print(f"Stopped Aurora Cluster with ARN: {cluster_arn}")

    def start_aurora_cluster(cluster_arn):
        """Stops an Aurora cluster identified by ARN if tagged for shutdown."""
        client.start_db_cluster(DBClusterIdentifier=cluster_arn)
        print(f"Started Aurora Cluster with ARN: {cluster_arn}")

    handle_rds_instances()

Then click Save.

The function needs a couple of parameters that you can configure in the Configuration tab. You need to navigate to the Environment variables section. The variables are:

KEY: it's a tag name that you would choose like SNOOZE or DEV-TEST.
REGION: the DB location like us-east-1 or eu-west-1.
VALUE: the tag value like Auto-Shutdown or true.

Then, go to the Test tab

And replace the Event JSON with one of the following options:

{ "action": "start" } => We will share this option in EventBridge to start the tagged DBs.
{ "action": "stop" } => We will share this option in EventBridge to stop the tagged DBs.

Navigate to your RDSs and tag your DB:

Then click on Test. If all is successful you will see a screen like the following one:

Step 3. Create your Amazon EventBridge Schedulers

Amazon EventBridge schedulers will trigger the Lambda function to start or stop the databases based on the schedules you defined. You will need to create two, one to start and one to stop the DBs.

In the Schedule pattern, you define the time choosing a Recurring schedule:

Then, you click Next. In the Target Detail, you choose AWS Lambda:

In the Invoke section, you choose your Lambda function like rds-scheduler:

And also you configure your Payload with:

{ "action": "start" } => We will share this option in EventBridge to start the tagged DBs.
{ "action": "stop" } => We will share this option in EventBridge to stop the tagged DBs.

After that click and choose Skip to review and create schedule. On the new page, click Create schedule.

After this, you can repeat the process for the stop scheduler. This is all that you need to schedule your RDSs.

Note:
Thanks to Yesh Tanamala, Sharath Lingareddy, and Varun Mahajan for starting this process that I improved.