Quite often I hear people complain about Lambda cold starts. Frequently this results in hours of discussion and a time-consuming quest for an answer. For me, Lambda cold starts are just a non-issue or at least they should be. Allow me to explain why that is.
The Law of large numbers
Context is everything. If your Serverless application only has to deal with one customer every eight hours, then yes, a Lambda cold start can hit you hard. At least it’s not hard to imagine that it wouldn’t go by unnoticed… On the other hand, if you run a service that handles on average a thousand concurrent users, then most likely a cold start will get lost in the crowd. I guess it won’t even be visible in your 95 percentile statistic. So please, bear in mind the context of your service when making architectural decisions in favour of Lambdas.
The Law of Instrument
Talking about the law of large numbers brings us seamlessly to the second topic: The Law of Instrument.
Although I love to use Lambdas, it’s not a Golden Hammer. If cold starts are hurting you badly -especially in case of a low request count- then maybe Lambda is not the right tool for the job. I agree, most likely it is the cheapest option. But in defence to price, I would argue that your time is too valuable to invest in a project where the extra penny for an ECS Fargate Container or a t3a.nano EC2 instance is a deal-breaker. Furthermore, in a setup where the number of requests is low, there’s probably little need to scale. Which makes that another benefit of Lambda can be left out of the equation as well.
Cold Starts are entangled with Lambdas
At this point, I already should have convinced you that cold starts are just entangled with Lambda. As with every technology, there are both advantages and disadvantages tied to it. It’s the job of a Software Architect to take every aspect of a technology into consideration and to pick the best tool for the job. It’s a trade-off, Lambdas offer you a lot of power and scalability at a very low price. However, this comes at the cost of a slower response every now and then…
What about Provisioned Concurrency?
Since December last year, it’s possible to enable provisioned Concurrency for Lambdas. It certainly has added value for some use-cases. On the other hand, you will pay some extra using it and if you cross the configured threshold you will feel the same pain as before.
Even now, with provisioned concurrency available, cold starts are just part of Lambda. I still believe that in the majority of use-cases cold starts should not have a noticeable impact on your application. Otherwise, it should be part of the trade-off you did when you picked Lambda as the best tool for the job.
Choice of Programming Language
To wrap up I want to spend an extra minute on the ever ongoing debate about the impact of a programming language regarding Lambda cold starts. What annoys me in this debate is the fact that it is beside the point.
In the cold start cases I’ve seen, the latency overhead added by the choice of language stands in great contrast to the latency added by the Lambda source code itself. Even one roundtrip to query AWS SSM Parameter Store will have an impact a few factors higher than picking the fastest language option. I don’t want to drill any deeper in this debate as this is not the core problem, I think more value can be found elsewhere.
Top comments (3)
Nice article Gert!
I do agree to some degree... cold starts aren't often a huge issue.
One thing that does bother me about serverless is how much operations are involved.
FaaS is often misrepresented as really easy to operate etc. But, in reality, there's an awful lot of tinkering and configuring to do i.e provisioned concurrency!
I totally agree. FaaS could be very challenging in every aspect. I stumbled upon quite some FaaS spaghettis already :-)
Depending on the type of workload you have to write, you could also have a look at Cloudflare Workers. You'll be limited on what you can do, but you'll get near zero coldstarts ! Their approach is very clever, if you ask me.