I've been using AWS and their products for the better part of about 7 years now. It's been the default go-to for both my professional projects I've been a part of as well as my own personal ones.
Regardless of the size or scale of the project, the underlying compute architecture or the target audience, I have always found one common denominator that I will always reach for: API gateway.
So I thought I'd put together some thoughts, ramblings and my opinions on why I value it so much, and maybe why you should too.
💡 What?
Just for a quick overview for anyone not familiar, an API gateway is a layer that sits in between your client(s) and backend services - a reverse proxy. You can think of it as the front door to your backend.
AWS' API gateway is essentially their managed-service offering of this.
🤸♀️ Flexibility
Platform architecture evolves over time and as new services as constantly being developed and becoming the new, shiny things that suit business needs, one of the things that has remained fairly consistent is the client-server interactions.
If our front door remains the same and the API contract is solid, we can swap out the backend without having to worry about the client.
AWS API gateway offers a plethora of various integration types, allowing you to evolve your architecture over time and tailor it to the features you are building. For example, if you have a feature that was traditionally synchronous and it is shifting more to an event-driven approach, this is something that can be achieved from a client-server interaction perspective without a lot of effort.
Whether your backend is deployed in a series of AWS Lambda functions, ECS tasks or EC2 instances, API gateway can integrate with all of them and endpoints defined on the gateway can be swapped between these with relative ease. We can also mix n' match the endpoints integrations as well, so some could be hitting Lambda functions and others could be hitting internal load balancers within a VPC - like I said, fully flexible.
For me, this is a massive advantage as I can start with building my backend out using one approach and as the platform evolves and requirements change, whether that is performance related, load related or something else, it can be swapped out for something more suitable - and not even "entire services" need to be migrated at once, it can be taken one endpoint/use case at a time.
📈 Scalability
I've always found scalability something that is often considered at either too early a stage in a platforms lifecycle or too late a stage. Too early and you might be considered to be over-engineering something, too late and you are considered to have a mountain of tech debt and migration issues.
The beauty of API gateway is the initial setup is lightweight and once it's in place, you don't really have to worry about scalability. Multiple clients can hit it all at once (web application, native clients, M2M interactions), these can be separated by API keys, usage plans attached to the keys amongst others if you want to have a lower-level of control.
Since API gateway is a managed service, it will scale based on the inbound traffic that is hitting your APIs - as close to "set and forget" as you probably want!
Configuration rules can also be put in place to throttle and limit certain interactions coming from different clients, meaning you have less to maintain on your downstream application scaling size as it can be more predictable.
🔐 Authentication & security
Since API gateway is going to be your front door, you'll probably want to authenticate folks and validate whether you want them to come through the front door before they have the opportunity to do so.
User authentication can be performed at the gateway layer, through features called custom authorizers. Again, these can be configured at an endpoint-level, meaning that different endpoints can validate different authentication tokens (e.g. normal user, M2M interaction)
These custom authorizers can interface with other services such as AWS Cognito, meaning you don't need any additional custom logic to validate your authentication tokens, or you can implement your own custom authorizer in a lambda function that decodes and validates JWTs (or any other form of authentication token) in the event that you using a different IdPaaS. These will get invoked prior to passing the request downstream and will gate keep whether that occurs or not.
This also means that you can avoid any unnecessary traffic hitting your down stream services in the event that it is unwanted traffic. Ultimately, this means that in the event that you have an authenticated API that is the target of a DDoS or brute-force attack, your API gateway can consume the heavier traffic rather than your downstream backend services taking the hit and potentially degrading their performance. Resulting in everyone being happier, especially your credit card.
Additionally, in the event of the above, you only have a single API to wrap/attach a WAF too, making it incredibly easy to configure and update based on your needs. So you can react quicker if it was a bad actor hitting different endpoints that may have been traditionally spread across multiple services, you have a single place to secure and tailor the WAF rules to help mitigate the attack.
🧠 Simplified Application Logic
If using the aforementioned custom authorizers for handling authentication, another major benefit is the fact that your downstream application logic doesn't need to worry about validating or deserializing authentication tokens - either less database load if validating internally or less HTTP calls to third-party services to perform said validation.
It means your authentication middleware can be super lightweight and easy to maintain, especially when writing your unit tests and only having to mock some JWT sub claims that might come upstream from the API gateway integration passthrough.
And who doesn't love simplifying their unit tests? Right? Right.
📚 Story time
On a past project, I was responsible for helping migrate our existing authentication mechanism (that 10m users used) to a new, third-party IdPaaS provider. At the time prior to migrating, our APIs were a mix of public and private, but all deployed in a similar way exposed through a few public load balancers. We had a series of downstream services behind these LBs that all pointed to the same database for validating user authentication, critically meaning that we had duplicated logic across services that essentially verified the same things under the hood - this, as you can imagine, was a nightmare to maintain. We weren't using API gateway.
So when the decision was made to move to another service for handling user authentication, we agreed it was a good time to implement API gateway to begin handling all of our publicly exposed APIs, not only as our front door for the traffic, but also to handle our user authentication.
(I'm sure you can see where I'm going with this, having read the above paragraphs of why I value API gateway!)
We implemented custom authorizers via AWS Lambda functions that we attached to the endpoints/resources defined on the gateway that validated the requests coming inbound before passing them downstream. Suddenly, this had the knock-on effect of:
- Simplifying application logic - our authentication middleware no longer needed to interface with the database, it just needed to consume the HTTP headers/sub claims from the decoded JWT.
- Unit test complexity was reduced
- The ability to break out functionality to other services or infrastructure was unlocked, we didn't have to balance the pros/cons of new functionality (and most likely duplicating the same authentication logic again!).
At the time, we also had instances of getting brute-forced from attackers and despite playing the delicate game of adjusting WAF rules, some of the traffic still hit downstream services, placing load onto both the application and the underlying database. With implementing the migration to API gateway, this also had the knock-on effect of:
- Reduced load on the LBs and database themselves from the unauthenticated, brute-force requests
- Underlying services didn't have to invoke autoscaling policies as often as the CPU and memory load was reduced on the instances.
- Ultimately, this helps reduce infrastructure cost as compute isn't being unnecessarily consumed.
Whilst all of these were great benefits of updating to use AWS API gateway, we did have the downside of coordinating force-upgrades and deprecation strategies for the native clients that hit the old APIs, whereas if we had have been using API gateway from the get-go, this transition would have been totally seamless.
Despite this downside and effort required to ensure a smooth migration, it was a totally invaluable shift for the future of the platform.
✍️ Conclusion
I've really only scratched the surface here with some of the things that can be achieved using API gateway - not even mentioning the fact that they can be provisioned as WebSocket gateways, individual usage keys tied to API keys for tiering of access, API versioning and the different deployment techniques that can be used.
The benefits of starting to build your APIs with gateway means that it unlocks the ability for all of the points I've talked about above, and if you haven't began with it, the path to migration isn't hard if you have a solid plan in place for the consumers of said API - and the benefits far outweigh the effort required in my opinion.
So despite whether I'm building a small personal project or implementing something that is enterprise levels of scale, chances are I'm always going to reach for API gateway - and I think you should too.
Read more about AWS API gateway here on the official AWS documentation.
Top comments (0)