aws-auto-cleanup
Using aws-auto-cleanup to keep an AWS test account neat & tidy (and minimal cost!)
Functional Requirements
- Reduce operational run-time of resources used for a testing\development, or for temporary spin-up of resources
- Reduce cost\operational expenses
- Ability to whitelist AWS resources that need to be retained
Operating Cost
< $2.00/mo for the following:
- AWS::Events::Rule
- AWS::Lambda::Function
- AWS::Lambda::Permission
- AWS::Logs::LogGroup
- AWS::IAM::Role
- AWS::S3::Bucket
- AWS::S3::Bucket
- AWS::DynamoDB::Table
- AWS::DynamoDB::Table
Node | Type | Unit | Usage | Total |
---|---|---|---|---|
auto-cleanup-settings-prod (DynamoDB-Table) | provisioned read | $0.00013 per hour for units of read capacity | 1 units per month | $0.10 |
auto-cleanup-settings-prod (DynamoDB-Table) | provisioned write | $0.00065 per hour for units of write capacity | 1 units per month | $0.48 |
auto-cleanup-settings-prod (DynamoDB-Table) | storage | $0.25 per GB-month | 1 GB | $0.25 |
auto-cleanup-whitelist-prod (DynamoDB-Table) | provisioned read | $0.00013 per hour for units of read capacity | 1 units per month | $0.10 |
auto-cleanup-whitelist-prod (DynamoDB-Table) | provisioned write | $0.00065 per hour for units of write capacity | 1 units per month | $0.48 |
auto-cleanup-whitelist-prod (DynamoDB-Table) | storage | $0.25 per GB-month | 1 GB | $0.25 |
auto-cleanup-prod (Lambda-Function) | 1000 | 128MB * AvgTime * Invocations per month (Usage) | 10000 invocations | $0.02 |
ServerlessDeploymentBucket (S3-Bucket) | Standard | $0.023 per GB - first 50 TB / month of storage used | 10 GB | $0.23 |
auto-cleanup-prod-resourcetreebucket-troydieter (S3-Bucket) | Standard | $0.023 per GB - first 50 TB / month of storage used | 10 GB | $0.23 |
Dynamo Storage | Discount | First 25GB Free (-$0.25 per GB) | -2 | ($0.50) |
Dynamo Backup | Point In Time Recovery | $0.20 per GB-month (Continuous backups) | 0 | $0.00 |
Dynamo Backup | OnDemand | $0.10 per GB-month (On Demand) | 0 | $0.00 |
Dynamo Network Outbound Traffic | Transfer | $0.09 per GB (Data Transfer Out) | 1 | $0.09 |
Lambda Service | Number of invocations | Invocation call for a Lambda function | 10000 | $0.00 |
Diagram
Stack
-
Serverless Stack
Framework Core: 2.1.1 Plugin: 4.0.4 SDK: 2.3.2 Components: 3.1.3
- AWS CLI
- NPM
Preferred configuration
After you clone the aws-auto-cleanup repository in the next step, you will have the opportunity to change the parameters of the serverless-stack deployment. A few notable changes:
- Change the serverless.yml file line 4, to your company name or project name.
-
Within the
auto_cleanup/data/auto-cleanup-settings.json
file, you will find the default parameters for the function. I have (obviously) found that the TTL (time-to-live) of 7 days to be too low. I have raised that to 120 days. Example:"ttl": { "N": "120" }
Deploy
Following the directions listed on the README, over at the aws-auto-cleanup Github page!
Useful commands
serverless deploy --region region-example --aws-profile profile-example
serverless invoke --function AutoCleanup --region region-example --aws-profile profile-example --type Event
Outputs
- Under CloudWatch, you'll see under Logs > Log Groups the following log group (if you kept the default Lambda function name):
/aws/lambda/auto-cleanup-prod
With a log stream of the latest Serverless invocation (or scheduled Lambda execution):
| 1600638849212 | [INFO] S3 Bucket
'outbound-email-send-dev-serverlessdeploymentbucke-46346' was created
234 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849212 | [INFO] S3 Bucket
'outbound-email-send-dev-serverlessdeploymentbucke-46346' was created
234 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849212 | [INFO] S3 Bucket
'outbound-email-send-hand-serverlessdeploymentbuck-46346' was created
234 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849212 | [INFO] S3 Bucket
'outbound-email-send-hand-serverlessdeploymentbuck-46346' was created
234 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849212 | [INFO] S3 Bucket
'46346-ai-chat-bot-dev-serverlessdeploymentbucket-46346' was created
230 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849268 | [INFO] S3 Bucket '46346-artifacts' was created 505
days ago and has been deleted. (s3_cleanup.py, buckets(), line 168)
| | 1600638849269 | [INFO] S3 Bucket '46346-cpds-infra' was created
494 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849269 | [INFO] S3 Bucket '46346-sree52-testbkt' was
created 430 days ago and has been deleted. (s3_cleanup.py, buckets(),
line 168)
| | 1600638849269 | [INFO] S3 Bucket 'python-myservice-bucket' was
created 158 days ago and has been deleted. (s3_cleanup.py, buckets(),
line 168)
| | 1600638849269 | [INFO] S3 Bucket 'python-poc-deployment-bucket'
was created 166 days ago and has been deleted. (s3_cleanup.py,
buckets(), line 168)
| | 1600638849269 | [INFO] S3 Bucket 's3-to-sns' was created 264 days
ago and has been deleted. (s3_cleanup.py, buckets(), line 168)
| | 1600638849269 | [INFO] S3 Bucket
'serverless-telegram-bot-serverlessdeploymentbuck-46346' was created
236 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849269 | [INFO] S3 Bucket 'swagger-bucket-1' was created
199 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
| | 1600638849269 | [INFO] S3 Bucket 'terraform-bucket-lokesh' was
created 243 days ago and has been deleted. (s3_cleanup.py, buckets(),
line 168)
| | 1600638849270 | [INFO] S3 Bucket
'twilio-voice-test-dev-serverlessdeploymentbucket-46346' was created
230 days ago and has been deleted. (s3_cleanup.py, buckets(), line
168)
Performing clean-up
-
Change the
dry_run
item in the DynamoDB table (auto-cleanup-settings-prod
- if you kept the default settings name) to false to perform clean-up:{ "dry_run" : { "BOOL" : false }}
- Invoke using:
serverless invoke --function AutoCleanup --region region-example --aws-profile profile-example --type Event
Recap
aws-auto-cleanup helps to keep multi-region AWS accounts clean. Resources are either white-listed, or deleted after specified amount of days. This reduces monthly expenses and conflicting resources within the same region!
Top comments (0)