AWS Lambda has a surprise learning curve. You create a new function, write your code, and it executes. Easy, right? Then you discover just how deep the rabbit hole goes. Deluged by so many topics, it's hard to know where to go next.
By this point, you've started using Lambda, but have now fallen off that mountain. In this post, we'll break down five steps to get you moving along your Lambda journey.
1) Improving your Code
AWS Lambda seems simple enough - write whatever code you want, and it runs it. That's true, but not all code is created equal, and there are some important ways to elevate your code in Lambda functions.
Foundational to using AWS Lambda is understanding the execution environment it uses so we that can understand behaviour like cold starts, and how our code impacts them.
The Best Practices for AWS Lambda is an excellent read, but to highlight the most significant points about code:
Remove your logic from your handler
Keeping your code as simple as possible removes headaches. When we start writing Lambda functions, they get lengthy, and we may keep them in the handler function.
This complicates our testing since our unit testing is inextricably linked to how our function is invoked. Use your handler function to control interactions with other solution components, while your business logic is held in separate functions within your code.
import requests
def check_inventory(check):
response = request.get('http://apiurl/inventory')
if check in response.text:
return True
else:
return False
def lambda_handler(event, context):
data = check_inventory(event['itemId'])
return {
'statusCode': 200,
'body': json.dumps(data)
}
In this case, the logic for checking our inventory can be tested easily, regardless of how it's integrated into Lambda. Likewise, our handler focuses solely on how to pass the information to and from the business logic, making it easier to maintain. This is a very basic example, while AWS Chief Evangelist EMEA, Danilo Poccia has a brilliantly succinct gist, GreetingsAPI.js, that shows a more complex integration
Configuration outside code
Trying to handle configuration inside of our code is always a horrible mess. How many times have you made code that looks something like this?
# apiUrl = "https://prod.my-tenant.customersystem.net/v2/res/item"
apiUrl = "http://127.0.0.1:3000/mock"
Or even worse...
apiToken = "QmFzZTY0IGlzbid0IHJlYWwgZW5jcnlwdGlvbg=="
Okay yeah, me too... ✋ Fortunately, AWS gives us a few options to get around this with Lambda.
Environment Variables are familiar to most developers and work identically between your local machine and AWS. But for anything secure, or that needs to be used across multiple Lambda functions, it's worth checking out AWS Systems Manager Parameter Store, AWS Secrets Manager, or AWS AppConfig. It removes many future headaches, and helps your security team sleep at night.
Leverage environment reuse
When a new instance of your function is spun up, the Init phase will run any static code outside of your functions.
If you need to perform any long-running tasks before handling requests, or instantiated before receiving requests, this can make your code much more efficient. Make the most of this where you can, making things quicker for your end users.
import pymysql
try:
connection = pymsql.connect(<DetailsGoHere>)
except pymysql.MySQLError as e:
logger.error(e)
sys.exit()
def lambda_handler(event,context):
with connection.cursor() as cursor:
cursor.execute("SELECT * FROM USERS WHERE userId='" + event['uid'] + "'")
data = connection.fetchone()
return data
Ensure that data leaks don't occur between invocations when handling stateful data. And even though something has been instantiated, there should still be checks, especially for database connections, to ensure they're still alive.
AWS Serverless Hero Jeremy Daly's article on Reusing Database Connections in Lambda from 2018 covers the concept well, although new options are available for database connection management.
2) Leveraging CI/CD and Frameworks
If you're still editing and saving your code live in the AWS Console, chances are you've run into this situation:
It's 2023, so you already know why you should be using a CI/CD Pipeline. But you can spend more time building and troubleshooting your pipeline than actually writing valuable code. And that's just for deploying one simple Lambda function.
Fortunately, we have many frameworks and tools to help us build and deploy our serverless solutions. Each Framework has different strengths, but all aim to reduce the friction between development and deployment.
- AWS SAM (Serverless Application Model) smooths the deployment of serverless applications for AWS
- Wing has built an entire language around cloud-native primitives that compiles it into another provisioning provider (currently only Terraform)
- Serverless Framework is the original framework for building serverless apps, and still a solid choice
- Ampt is currently in private beta, but is going deep into building a fully integrated developer platform. I'm excited about what this means for developer experience!
If you're familiar with CloudFormation, I'd suggest AWS SAM as a starting point. Once Ampt exits private beta, I will start to experiment with that.
3) Easy Observability with PowerTools
We're all guilty of using print('boop') or console.log('hi!') for debugging. This doesn't scale well in Production though. Implementing custom logging for the myriad of outputs, plus generating and handling custom metrics, involves a whole mess of extra work. Not to mention supporting distributed tracing.
Enter Lambda PowerTools - a solution that takes only a few minutes to install in your code, and makes it incredibly easy to handle your logging, metrics, and distributed tracing, all familiarly handled in your language of choice.
Observability is the difference between your solution being a big box of production mysteries and a brilliant symphony of many working parts where you can understand and address issues proactively. This is the single most straightforward and significant uplift you can give a serverless application in AWS.
Highly recommend checking out the Tutorial for Powertools for Python, which covers both Powertools, plus some AWS SAM as well!
4) Tweaking your Deployment
"Any sufficiently advanced technology is indistinguishable from magic" (Arthur C. Clarke). Lambda is an advanced and powerful technology, and its magic has created some common misconceptions.
- Lambda will scale your functions infinitely
- Lambda execution environments will start immediately
- Lambda changes everything immediately when you update your code
- Lambda will use as many resources as it needs
Let's take a closer look at how we can configure our Lambda functions.
Function Scaling with Reserved Concurrency
Lambda does have a scaling limit. By default, that number is 1,000 concurrent executions per account. Since all Lambda functions in your account share that concurrency limit, you can reserve a maximum amount for each function with Reserved Concurrency.
Reserved concurrent can also be used to set those upper limits to prevent overwhelming downstream services that your Lambda function may be sending requests to, like API's. After all, your local coffee shop with a single barista could handle infinite orders. That is if you are okay with waiting an infinite amount of time for your coffee.
You can request quota increases for your account's concurrency limits - but it's worth considering other limitations first.
Cold Starts and Provisioned Concurrency
We know from our exploration of the execution environment that Lambda doesn't start new execution environments immediately - it takes time to spin them up through their Init phase. Provisioned Concurrency is how we can ensure that a minimum number is ready for use at any time.
It might be tempting just to use Provisioned Concurrency to work around cold starts, but there is one huge drawback: it's not cheap. Having execution environments on standby incurs an ongoing cost of about 25%. This sounds cheap but it adds up quickly.
A Lambda Function with the standard 512MB memory and 10 provisioned concurrency instances running adds up to about $54 USD per month. That's without even adding the costs of each request and their running compute time. It's always worth optimizing before jumping to Provisioned Concurrency.
Managing Changes with Versions and Aliases
One of the great things about Lambda is quickly updating your code and having it reflected in your Production compute environment. That's great, but often you need a bit more control. This is where versions and aliases come in.
Lambda has built-in versioning capabilities. When doing live editing, you're working with the $LATEST version. Creating versions lets you lock your code to a specific state, which can be executed independently by its unique ARN.
Aliases take this to the next level, where you can have a dynamic reference to a particular function version. If you had an alias for prod, this could be the target of your invocations from your API Gateway. When you've finished testing a new version, you can just update the alias to point to the updated version instead of updating your API Gateway.
Tuning your Resource Configuration
You've probably seen the concept of a GB-second mentioned in the Lambda pricing. Essentially, you get charged based on how long your Lambda function runs, and how many resources you give it; and despite being named a "GB-second", the memory also directly relates to the apportioned CPU and network resources.
The more memory you allocate to your function, the faster it works. But the more memory you allocate, your cost also increases. So it's worth finding a sweet spot between memory allocation and function duration. As a general rule:
- If you're performing lots of calculations, allocate more memory
- If you're waiting on external calls, allocate less memory
The AWS Documentation's Memory and Computing Power page is a good starting point. To avoid configuring it manually, it's worth checking out AWS Lambda Power Tuning, which will help you find the sweet spot.
5) Architecting for serverless
By this point, your AWS Lambda skills have matured enormously. But there's still much to learn in the world of serverless computing. There are a lot of places to get started, so I'd suggest checking out some of these awesome videos from AWS re:Invent 2022
- Get started building your first serverless, event-driven application (SVS209) by Emily Shea
- Building next-gen applications with event-driven architectures (API311-R) by Eric Johnson
- Building Serverlesspresso: Creating event-driven architectures (SVS312) by James Beswick
These give you a much deeper exposure to the development and architecture of serverless solutions. Since you already know Lambda, it's far less about the code, and more about the patterns and architectures.
Wrapping-up
The learning with Lambda does not stop here. There are still many areas of the service and development to be further explored! You can find answers to almost any problem between the Developers Guide and Operators Guide. AWS has made it truly possible to become an expert at their services by being able to navigate their documentation, and Lambda is no exception.
To dive deeper, check out some of the fantastic content on the AWS Events YouTube Channel, with dozens of incredible talks from re:Invent across all serverless topics. You don't need to spend thousands of dollars to learn from industry leaders, and they are truly worth exploring.
Lastly, check out more of the content published by members of the AWS Community! You can bet that whatever you're building, someone has done something similar before.
Keep safe, and happy clouding!
Top comments (0)