Automate AWS Cost & Usage report using Event Bridge, Lambda, SES, S3 & AWS Cost Explorer API

Do you often wait for a report on cost usages for your cloud resources to get a view of overall cost?

AWS cost explorer API provides a way to automate the creation of cost & usage report that can be invoked on demand or can be scheduled using other AWS services so that a formatted report can be dropped in your inbox on a regular cadence (weekly, monthly etc.)

Let's see how to automate AWS cost an usage report using Event Bridge scheduler, Lambda and AWS Cost Explorer API.

I will create the cost & usage report using Lambda function, will share it via email to the verified email, will post it to a S3 bucket for record keeping or further processing by other services and will schedule this via event bridge scheduler.

In the realm of cloud computing, resource provisioning is effortlessly streamlined, offering invaluable agility to businesses for a myriad of use cases. This includes accelerating solution delivery, enhancing agility in infrastructure, computing, and scalable storage, as well as facilitating backup, integration, and more. Nevertheless, this convenience comes at a cost; each initiated resource contributes to the overall expense of cloud usage. Therefore, it is imperative to regularly scrutinize both costs and resource utilization, optimizing them in accordance with the specific needs of the business.

Let's review our use cases:

• Generate a monthly cost and usage report for each AWS service.
• Email the generated report to the assigned email recipient.
• Post the generated report in JSON format to an S3 bucket.
• Schedule this report using the EventBridge scheduler.

I am generating my lambda function using AWS SAM, however similar can be created using AWS Console. I like to use AWS SAM wherever possible as it provides me the flexibility to test the function without first deploying to AWS cloud.

Here is the architecture diagram for our use case.

Let's see the steps to create this automated cost & usage report:

Create a S3 Bucket

Since our business use case requires storing the cost & usage report data in JSON format in S3 bucket, first we will need to create a S3 bucket. My s3 bucket is as below, but you can name the bucket as you wish.

Create a SAM template

I will create a SAM template for the lambda function that will contain the code to invoke cost explorer API for my account. Lambda function can be created without the SAM template however, I prefer to use Infra as Code approach since that allow for easy recreation of cloud resources. Here is the SAM template for the lambda function.

This SAM template is defining a Lambda function with timeout and few other parameters. I have some environment variables also defined as those variables provide flexibly of updating s3 bucket, region, report and account at runtime.

Create a Lambda Function

Lambda function is the core of this automated solution. This function will have the code for the business requirement of creating the cost & usage report for AWS cloud resources being consumed. This lambda function also has sub-functions to generate email report and uploading the JSON data in s3 bucket. Let's look at the code behind it.

Create Event Bridge Scheduler

Event Bridge schedule is another AWS service that I will use to complete the use case for automating the cost and usage report.

This scheduler can take the scheduling as CRON job pattern. I am creating a CRON job to generate the report at 12am on the first day of the month.

Last step is to integrate this schedule with the lambda function created in the earlier step. This integration can be performed using the AWS console.

Once this integration is completed, entire solution to generate automated cost & usage report is in place. Once schedule has invoked the function on a defined schedule, you will see the report posted in the S3 bucket. An example is shown below:

If you download and open this report, you will see the data in JSON format. An example as below:

A sub-function being invoked in the Lambda is using another AWS service to send this report via email to a pre-determined email.

"That's it! You now have a solution that provides an automated way of receiving the cost and usage report on a monthly basis via email. Additionally, a JSON format report is posted in an S3 bucket for storage and further processing. A schedule has been created using EventBridge, which will invoke this Lambda function on a monthly basis. This monthly notification of summarized costs by service meets the needs of our business use case and provides an automated method to generate and receive these reports!"

Thanks for reading!

Click here to get to YouTube video for this solution.

https://www.youtube.com/watch?v=pZeJzgjsehE

𝒢𝒾𝓇𝒾𝓈𝒽 ℬ𝒽𝒶𝓉𝒾𝒶
𝘈𝘞𝘚 𝘊𝘦𝘳𝘵𝘪𝘧𝘪𝘦𝘥 𝘚𝘰𝘭𝘶𝘵𝘪𝘰𝘯 𝘈𝘳𝘤𝘩𝘪𝘵𝘦𝘤𝘵 & 𝘋𝘦𝘷𝘦𝘭𝘰𝘱𝘦𝘳 𝘈𝘴𝘴𝘰𝘤𝘪𝘢𝘵𝘦
𝘊𝘭𝘰𝘶𝘥 𝘛𝘦𝘤𝘩𝘯𝘰𝘭𝘰𝘨𝘺 𝘌𝘯𝘵𝘩𝘶𝘴𝘪𝘢𝘴𝘵