AWS Cost Optimization #2-Techniques

Cost Optimization is an activity which is aimed at driving down the business spending and the cost. I recently published an article Cost Optimization #1: AWS EBS Volume type IO1 or IO2? where I mentioned that I will be talking about more such ways to optimize your AWS cost. Through this article, I will share different tips and techniques which can be adopted to minimize operation costs for compute resources on AWS.

Cost Optimization Techniques

1. Stop/Scale down the resources that are not in use
The biggest benefit of cloud computing is on-demand pricing. Therefore, the most effective way to reduce your compute costs is to turn-off those resources that are not in use to stop their billing. This will require you to prepare a schedule and then automate the stop and start of instances according to that schedule. Below are some of the ways to achieve this:

A. AWS Instance Scheduler
This is by far one of the most easiest ways to schedule stop and start of the EC2 and RDS instances. AWS built this solution on top of Cloudwatch Events, Lambda and Dynamo DB and provided a cloudformation template to simplify the deployment. I will provide you with couple of documentation which will help you get started - Automated Deployment and an example.

The only effort required from the user is to implement a schedule in DynamoDB and tag those instances which you would like to stop and start on a schedule. Although you might not be familiar with DynamoDB but the documentation surely makes it easy and here is a sample schedule which will be your reference point to form your own custom schedule.

Important points to remember

• Do not use the solution for instances in Autoscaling group as the moment instances are stopped, Autoscaling group will terminate them and launches a replacement instance.

• Cost of running the solution - Below content is picked up from Instance Scheduler Documentation:

  As of the date of publication, the cost for running this solution with default settings in the US East (N. Virginia) Region is approximately $5.00 per month in AWS Lambda charges, or less if you have Lambda free tier monthly usage credit. This is independent of the number of Amazon EC2 instances you are running. The optional custom Amazon CloudWatch metric, if enabled, will cost an additional $0.90 per month per schedule or scheduled service.

B. Lambda to stop and start EC2 instances
Another way to perform the scheduled stop and start of EC2 instances is to make use of AWS Lambda and Cloudwatch Events. The solution is explained in this article along with the implementation. On a high level, you will have to define the instances in the lambda function and then use cloudwatch events rule cron expression to trigger the lambda function. You can define instances in the lambda function using comma as the separator as shown below:

instances = ['i-12345cb6de4f78g9h', 'i-08ce9b2d7eccf6d26']

Important points to remember

• It cannot be used for instances placed in Autoscaling group for the reasons explained above.

• Cost of running the solution - You will be paying price for executing the lambda function and it is documented here. Please ensure to select the correct region to check the prices.

C. Schedule scaling for EC2 instances under autoscaling group
This feature of EC2 Autoscaling groups can be used to achieve the same effect as stopping the instances but with a slight difference, here the instances will be terminated by reducing the desired count of your Autoscaling groups on a schedule start time. Once the schedule ends, the desired count will reach to the original value, thus launching new instances to fulfil desired capacity. To understand more about the solution, you can refer to this documentation. The steps are documented nicely and the only points to note here are:

• You will have to create scheduled action for each Autoscaling group
• If you attempt to schedule an activity at a time when another scaling activity is already scheduled, the call is rejected with an error message noting the conflict.
• No additional costs are associated with this feature.

D. Scale the capacity down using scaling policies
It is a similar approach as above but the difference is you do not have to schedule your termination of instances. You will be using dynamic scaling up and down your EC2 instances based on their utilization. For instance, you can scale up when the CPU utilization of the instances above 70% or scale down when the CPU utilization reaches 20%. The difference in this approach is that you can scale down anytime when the system is not being used which will help you to reduce your compute costs. Dynamic scaling is very nicely explained here along with 3 scaling policies which can be used:

• Target Tracking Policy - Here you can configure a standard cloudwatch metric for utilization or you can also create a custom metric like memory utilization for scaling up and down according to your preferences.

• Step and Simple Scaling Policies - These policies require you to define low and high thresholds as well as you would like to increase or decrease instances. Read the documentation to learn more about it.

2. Free AWS services for Cost Optimization
AWS focus heavily on the success of their customers and this is the reason they have built services which can not only optimize costs but also analyze if the infrastructure is optimized, secure and built according to the best practices. Some of the services which I am aware of are listed below:

A. AWS Compute Optimizer
AWS Compute Optimizer is a free service and you can enroll in an individual account or all of the member accounts in your organization. It reports whether your EC2 and Autoscaling groups are optimal, and generates optimization recommendations to reduce the cost and improve the performance of your workloads by analyzing the historical utilization metrics. You can use the recommendations to switch the current instance type running in your account to the recommended instance type to realize the performance and cost benefit. To get started with this service, please follow this documentation

This service can really prove beneficial as it can save a lot of manual time to investigate your past performance to determine if your instance is over-provisioned or under-provisioned and this effort can increase exponentially when there are hundreds or thousands of EC2 Instances/Autoscaling groups in an account or multiple accounts. Therefore, just by using the service (even though not implementing the recommendations), you will still be saving man days worth of effort for performance measurement.

B. Trusted Advisor
AWS Trusted Advisor is an online tool which performs fixed checks to optimize your AWS infrastructure, increase security and performance, reduce your overall costs, and monitor service limits and you do not require any support plan to view the results of these checks. You will get additional checks besides the above ones under Trusted advisor if you have (or will) subscribe to any of the support plan. The biggest benefit of this service is that it checks various resources like RDS, Redshift, Elasticache, Route 53, ElasticSearch and so on for cost optimization. To get started with this service, just follow this page or click on the Services button at the top of your AWS Console and search for Trusted Advisor to open it up.

I will stop here for the moment so as not to make this article too long. I have other tips and techniques which will be presented in future articles. To give you a preview of what is to come in future articles, refer to the below list:

• Tags to identify cost
• Tips to reduce EC2 costs
• AWS Budgets
• Optimize EBS snapshots retention

If you have any comments on the above methods or would like to know more, please do put your comments/feedback as I am doing this series to help people out there achieve real savings, especially during the pandemic.