Amplify at Scale - Beyond the First Users

I want to take a second to talk about why Amplify exists, the problem it's solving, and what that can mean for scale.

At the end of the day the goal should be to focus as much time as possible on producing product, no matter what that is. Which means we need to automate or abstract away anything we can. The closer to the surface you can work on something, the less time you spend working on things that are not the actual product or service itself.

A great example of this is the No Code revolution. Services like Webflow and Draftbit are abstracting the need to know how to code to make beautiful and engaging websites and mobile applications. However, there are times when you need to do more than what you currently can do with a No Code solution. That's where Low Code solutions step in to pick up the slack.

A Low Code solution is a tool/framework/etc that allows you to focus as much as possible on business logic and spend minimal time working on technical code, or code that isn't actually solving a business problem. Things like setting up APIs and authentication, or setting up data management on the client are technical code. It's very important, but isn't a feature itself, just the technological choices backing them.

Something that falls under this category would be GatsbyJS. If you're unfamiliar, Gatsby is an OSS framework that allows you to build really performant and compliant websites and apps by abstracting away the messy complexity behind an API that focuses on having a small surface but being flexible enough to get really deep under the hood.

Even if you have very little experience with Gatsby you can get started by creating a file in a directory and it will magically show up on your website, with all the best practices already baked in. You have to actively work against Gatsby to build a poorly performing app.

Given it's flexible API, you can also really get into the nitty gritty. You can do all kinds of things, like adding dynamic sections to your site, building pages programmatically, and even pull from an eco-system of plugins that do way more. Gatsby is by no means the only one doing this. There are hundreds of tools and frameworks all working at making it easier to do something, but the tools I want to talk about today are the ones that are making it possible to build robust applications using a framework like Gatsby by providing the infrastructure needed to support your application.

However, tools that offer this type of experience often fail to scale once you move beyond the basics, making it impossible to grow without removing the tool that allowed you the speed you needed to get to growth in the first place. The reason is that the comfort they offer often limits the flexibility. It's not easy to do what Gatsby does and allow for that progressive disclosure of complexity (if you're unfamiliar with that terminology Jason Lengstorf wrote a great article about it here).

These are services like Firebase, Netlify, and Amplify. Any one of these provides some combination of authentication, serverless functions, APIs, data and file storage, and even some things like machine learning services. All targeted at making it possible to build on top of cloud environments, while using best practices, without having to manage all the complexity yourself. However, the question with these types of services always is, will I outgrow it?

Scaling with Amplify

When we talk about scale in regard to a toolset like Amplify, we're actually talking about two types of scale. Scaling to accommodate more users (a great user experience), and scaling to accommodate more developers working on the product (a great developer experience). So when we ask the question, does Amplify scale, it's important to think of both the UX and DX, because both need to scale in order to be usable beyond the first users.

Scaling the Developer Experience

"Great things in business are never done by one person; they're done by a team of people." – Steve Jobs

While Amplify provides a great DX for the individual developer, DX isn't just about what's going on inside your code. It's also about sharing code across teams or an organization, working with environments for QA and feature development, where and how data is sourced to clients, or how the project is built and deployed. While most of these things are generally thought of as separate ideas or workflows, you can't ship efficiently if any of these pieces are not working or set up properly.

Providing a great developer experience may cause issues with scalability of services, and making services scale well generally adds complexity at the infrastructure layer, making the developer experience worse.

Things like serverless multiple this by allowing you to only write code that solves business problems by shifting all the complexity to the infrastructure layer. Development is about tradeoffs and we have to really know what tradeoffs we're making. This is why there are entire teams dedicated to solving this problem.

DevOps is a response to a need for better developer experience in cloud environments and cloud environments are a response to a need for better user experience.

Scaling Tooling

This tweet by @swyx really summarizes what makes a good DX for tooling:

The Amplify CLI exemplifies a lot of these characteristics and I'd like to go over a few in particular:

Good defaults: When you use the Amplify CLI to configure AWS services, the CLI walks you through your options and always provides the opportunity to either use the current best practices defaults or allows you to configure yourself (there's that progressive disclosure of complexity). This allows you to do things like set up authentication that is enterprise ready in a matter of minutes without having to be a security expert.

Incrementally adoptable: Don't want to use Amplify for everything? Cool, just pick a category and start there! Amplify uses categories to break logic up into common workflows or scenarios that developers face when creating any product. You can incrementally add as many categories as you want. Some popular categories are:

• Authentication (Federated, LDAP, etc)
• API (REST and GraphQL supported)
• Analytics
• Storage (File storage or data)
• Functions (Serverless functions with Lambda)
• Predictions (Machine Learning and Artificial Intelligence)

Small API/config surface area: Amplify CLI only has a few base commands and requires no config from you. It may ask a few questions to help you select the proper options, but it generates the necessary config. At the end of the day the CLI is really a codegen tool, whether it's generating CloudFormation templates (responsible for what services get created and how they interact), types (Flow or Typescript), or GraphQL operations (queries, mutations, subscriptions), it eliminates a lot of boilerplate code that would otherwise be needed.

Learn once and reuse forever: The Amplify CLI's small surface area and repeated patterns make it intuitive to use for any category once you learn the basics of the CLI.

Remembers and adapts to you: One of the best features of the CLI is how it's aware of the current Amplify configuration and can generate the necessary config to tie together services. This is one of the hardest parts for most developers to understand about AWS, and the Amplify CLI handles this for you. If you add authentication and then want to add an API, Amplify will set up the proper permissions between the services. Also, when it comes to adaption, the CLI's plugin based architecture makes it possible for DevOps teams to extend the functionality of the CLI by creating custom in-house plugins. Allowing them to add categories or workflows for other AWS services, or even other cloud providers. You can find out more about the plugin system here.

The CLI does a lot more than this, but these are a few of the key features that allow the tooling to scale to handle any needs and any number of developers. Learn more about the CLI here.

Scaling Collaboration

When it comes to a good DX, collaboration is a must. Unless you're a sole developer working on a project (and even then you most likely don't work completely in isolation) than you are going to want to use tools and services that make it easy to share/show/ship products. The Amplify framework provides a number of features that make collaboration not only possible, but pretty darn enjoyable. Here's a few of my faves:

Git like workflow for CLI: It's not uncommon to want to set up specific environments of your system. Whether it's for testing or feature development, generally not everything is done in same environment that your users use. There may be a QA env, a staging env, or maybe a new environment is created for each bit of work done. Amplify supports all of these workflows with multiple environments. By using commands similar to git, you can create, push, and pull environments like you can with git branches.

Ephemeral environments: While we're talking about environments, having the ability to recreate them on the fly is invaluable. As your team and projects grow, generally so will the environment around it. Being able to easily recreate them opens the door to some really efficient workflows. A very common workflow I see is branched based deployments. Branch based deployments is when you deploy a new environment for each branch pushed to whatever git provider you are using. You do this via Amplify Console which is a hosting service for fullstack serverless web apps. This makes it easy to create isolated environments for testing, previewing, or sharing with other developers. When your infrastructure is easily repeatable it can scale to accommodate any number of new collaborators.

Deploy previews: It's hard to talk about collaboration without talking about pull requests. Pull requests are the epicenter of a project and it's generally where the most collaboration takes place. When I was at Major League Soccer everyone from product, QA, and design would all collaborate around pull requests and whether it was a web app or mobile app, we all wanted to be able to be able to preview the environment. The most efficient way to do this was to have a deployment preview linked in the PR itself. Now we had to set this up ourselves but it was definitely worth the work because of how efficient it was.

If you're curious about deploy previews with Amplify Console give me a follow as I'll be writing about it very soon!

Starters and sharing config: Because we can create these environments based off of CloudFormation templates generated by the Amplify CLI, we're able to share configuration across an organization if needed. Complete starters can be created and deployed to Amplify Console in a matter of minutes with the Deploy to Amplify Console button. If you're an agency that regularly builds e-commerce sites for clients, you can set up a default template that teams can use as a starting point for each project. They can then add any other functionality they need on top of the starter in their own environment. It's also possible to pull down existing config for an Amplify project from Amplify Console, much like pulling a remote branch with git.

Scaling Infrastructure

Until recently it was really difficult to take advantage of the recent developments in cloud and serverless computing. Without have a team of seasoned devops engineers or outsourcing that work it was nearly impossible to create the right configuration to scale well. Entire companies exist just to help you optimize your infrastructure configuration, making sure you don't waste money on unneeded resources, or worse that you aren't scaling properly to meet the demands of your customers.

However, as serverless and cloud move more into the commodity phase there has been a lot of progress on the user experience around serverless and cloud computing. Nader Dabit sums it up well in the following tweet:

It's now possible to take advantage of serverless without being a master of cloud engineering and developers further up the stack are starting to build cloud based products on top of serverless. By using Amplify you get to take advantage of best practices when it comes to managing your infrastructure on AWS without having to know how to set it up yourself. Let's take a look at a few of the ways Amplify can help you scale.

Backend Infrastructure

Infrastructure based on data needs: When you use Amplify to create your GraphQL APIs you can specify what data should be stored via the @model directive in your GraphQL schema. This allows you to only create the storage needed at that exact moment. You can easily update this at any time by changing the schema. More info here.

Enterprise ready services: Amplify itself isn't a service, it creates the underlying CloudFormation templates needed to spin up the AWS services needed to complete that workflow. If setting up an API this could involve Amazon Cognito for authentication, AppSync for querying data, and DynamoDB for data storage. All of these services are relied on by enterprise level companies that need enterprise level services to ensure that their customers needs are always met. By using Amplify you get to use those same services without having to handle them yourself.

API performance: AppSync provides built-in server-side caching capabilities for any supported data source, improving the performance of latency-sensitive and high-throughput applications and allowing developers to fetch data from a fast, in-memory, managed cache, delivering data at low latency. As well as enhancements to GraphQL real-time subscriptions with pure WebSockets supporting a higher maximum payload size (240kb vs. 128kb with MQTT over WebSockets), enhanced connection and broadcast rates to effortlessly scale to millions of connections, CloudWatch metrics, and selection set filtering for GraphQL subscriptions. Check out this link for more info.

Serverless functions: Need to do some computing outside of CRUD operations? No worries, by connecting serverless functions to your API (or using via events or a REST endpoint) you can scale workloads infinitely. Need to create thousands of thumbnails as images are uploaded to your new e-commerce platform? Lambda can handle that and pretty much anything else.

Frontend Infrastructure

DataStore for Client Performance: When we think of infrastructure we generally think of backend services. But what about the network layer on device? Amplify DataStore provides a persistent on-device storage repository for you to write, read, and observe changes to data if you are online or offline, and seamlessly sync to the cloud as well as across devices.

Custom components and multi-platform support: Amplify also has pre-built components for most front-end frameworks and platforms that allow you to quickly connect to your backing infrastructure without having to manually set up anything. Display S3 files or handle uploads with the Picker component, set up entire auth flows without having to write any backing logic, or present a UI for interacting with chatbots. All of these components are plug-and-play, saving you countless hours or even weeks of development time.

Conclusion

While I've covered some of the more key features here it's important to note that I've really only scratched the surface in ways that Amplify can help you scale, both from a DX and infrastructure point of view. If you're curious about how else Amplify can help you get your product in production then I highly recommend taking a look at the docs!