Web Application Scaling: For Dummies (like me)

Let's say you have created a blog website where you share your ideas and knowledge on coding. At the start, your website was small, and the traffic was low. A few weeks went by, and you see there is a lot of users now. They are requesting that you add forum system to the website (much like StackOverflow). You add that, and now everyone's happy. Seeing new features, there are more users now. Next week, a user contacts you via your email saying that they can't access your site.

Because the web server crashed....

When you log onto your VPS/server where your application is hosted, you see that the resource usages are peaking at 100%. Your single server/application instance cannot handle this insane amount of traffic coming to your website. So how do you ensure that your website is always up, and available?

By scaling!!

Tell me, what in the bananas is scaling?!?!?!?

Tldr, scaling is the process of making your application cope up with a huge amount of users. If one instance of your application cannot handle this much traffic, you spin up another server (cloud service providers like GCP, AWS, DigitalOcean, Azure, etc, make it easy and cheap) and host your application there.

That way, if one of your application server crashes, the other one can handle the traffic. Or, the huge traffic can be distributed to both of the servers, depending on which one is the least busy.

But here's a catch!

Since your application is hosted on two different servers (for the sake of scaling), each of the servers have a different IP address. But you have a single domain, so how will the user know which server to connect to?

We need to call upon the mighty Load Balancer!

What is a Load Balancer?!?!?!?

Well, since you have multiple servers for the web application, we need a server in the middle which will receive all the client requests, and distribute them to the application servers. So instead of assigning the domain to the web application servers, you assign the domain to the Load Balancer server, and all the users/clients will send the request to the Load Balancer. The Load Balancer will simply receive all the traffic, and send them to one of your application servers depending on which one is the least busy.

Most cloud service providers have their own Load Balancers which work very well. But if you are like me, and want to set up a Load Balancer form scratch, then you can use Nginx as your Load Balancer. Now don't get me started on Nginx, I can write a whole post about how great and useful Nginx is. Simply, it's not just a web server, you can use it as a reverse proxy, Load Balancer, TCP proxy, etc.

So now we have a simple but efficient Load Balancer which will distribute all of our traffic to the application servers. End of problem, right? Well, I wish I could tell you that. But there are still some things that need to be fixed.

First, the database.

How do you scale a freaking database?!?!?!?!?

Right now, a database instance is being hosted on each of your application servers. That would have been fine if you only had a single instance of the application, and the database is hosted on the same server as the application. But since we have multiple instances of the application now, it will be a problem. As the Load Balancer is sending the traffic to different servers at a time, each application server that receives the request will write to the database that is hosted on it's own server. So the data will be different for each server.

So how do we solve this?

Well, the best way to do it would be to have a centralized database. Now if your application is VERY big and works at a high scale, you might need to partition your database, or use some other database scaling pattern. But for now, just having a good old centralized database would work just fine. You need to spin up another VPS/server, and host your database software there. Or, you can use a managed database service like DynamoDB (AWS), RDS (AWS), SQL Database (Azure), etc, which you don't need to worry about scaling as the cloud provider will scale it for you.

Now, just make the application servers connect to that database. This way, the application state/data would be the same across every single instance since they are all using the same database server.

So we have a centralized database now, and all users are happily using your application. But suddenly, your blog started to pop off. People liked the content you were making, and they are so many users visiting your page that your database started to crash, and your reads per second rates are VERY high.

How do we solve this??

Let us implement a caching mechanism!

Bro, what is caching now?!?!?

Caching is a very simple concept, yet it is very useful. Whenever someone visits your website/blog, it is making a request to the database for the post. So if many users visit your blog, the number of database read requests is high. So instead of making a database request for every time someone visits a page, we can use a caching mechanism (or caching layer) that will simply cache your posts in memory, for some time. That way, instead of making a request to the database every time someone visits your blog, the request can be made to the cache, and that cache will return the data that is stored in the memory.

The caching layer (or simply, the "cache") can update itself every few minutes so that the data is not stale. Since it is a blog website, you won't be writing posts every 5 minutes, jeez. So the cache can just fetch the updated data from the database every 15 or so minutes, and store it in memory. That way, whenever a user visits your blog, it will fetch the data from the cache instead of the database; reducing the number of database read requests.

Conclusion

So now we know how we can scale a basic web application. This are just very simple scaling patterns, and bigger applications might need more advanced procedures to ensure maximum performance and uptime. Also, this is my first ever tech blog post, so please ignore any grammatical/spelling mistakes. That's all for this post, hope you found it helpful :)

The best type of application is the application that scales well - Dev Tzu, Art of Code