AJAX Demystified - Why Do We Need AJAX?

This two-part series came into existence after I started the JavaScript Client-side Frameworks series. While I was researching materials for my posts, I can't tell you how many times I came across the AJAX acronym. A quick Google search gave me some confirmations that I somewhat understand what AJAX does, but the more I dug, the more I realized that it's way more complicated than I thought, yet it's one of the building blocks in all JS frameworks. I was quickly convinced that understanding AJAX is fundamental to our understanding of modern JS frameworks. In this first part of this series, we will explore why we need AJAX in the first place.

First off, don't let the cover image fool you. Yes, we're talking about AJAX - the technology, not Ajax - the Greek hero.😜

Asynchronous JavaScript and XML (AJAX) is a collection of interrelated web development techniques - we can think of AJAX as design patterns - that are used for asynchronous web applications. AJAX allows web pages to send and receive data asynchronously in the background, without blocking the user from interacting with the web page while the request is being processed by the backend server.

The classic web app model was built around the notion of pages. A document is displayed to the user, containing lists of links and form elements that allow them to drill down to further documents. Most user actions in the interface trigger an HTTP request back to a web server. The server does some processing before returning a new HTML page to the client. While the server is doing its thing, the user has to wait. This server-side web architecture models the transition between pages as state transition diagrams and the site itself as a collection of pages.

It may be hard to comprehend the disadvantages of such an approach in traditional web app development since we are so used to the fast and responsive web apps as we know them today. So you will have to think of this in the context of how the Internet was being used in the years 2000's.

Before we get into the WHY behind AJAX, there are a few concepts that we need to understand first. So, let's cover these prerequisites.

Sovereign vs. Transient Applications

Software usability expert Alan Cooper has written about usage patterns and defines four distinct categories of software we use in our daily life: sovereign, transient, daemonic, and parasitic. For the purpose of this article, we will only focus on the first two usage modes: sovereign and transient.

1. Transient Applications A transient application might be used every day, but only in short bursts and usually as a secondary activity. When users are using these applications, they have already stepped out of their regular flow of work, and so a certain amount of clunkiness is acceptable. A calculator is a transient application, which is a type of helper program that is used for a specific task and then quickly closed again.
2. Sovereign Applications A sovereign application, in contrast, must cope with the user's full attention for several hours at a time. A good interface for a sovereign application must support the users _ invisibly _, without breaking their concentration on the task at hand. Microsoft Word, Excel, and Photoshop are all examples of sovereign applications.

Why is this relevant to AJAX anyway? 🤔 Valid question!😜 It's quite relevant, actually.

In the late 90s to early 2000s, the classic and dominant page-based web pages were only good enough for _ transient _ use. The browser itself is a _ sovereign _ application that is typically used for hours on end, but the web pages were mostly _ transient _ in their nature. In the early days of the Internet, users typically visited a specific website to check the weather, news, other information and then left. Desktop applications were dominant at the time when it comes to more sophisticated software that was created to solve business problems.

_ "Why couldn't we convert existing desktop applications into web applications instead?" _ is the next logical question one may ask.🤔 As a matter of fact, that was on web developers' wish list. One of the advantages of web applications is that it allows collaboration , i.e. resource sharing between team members, and flexibility , i.e. users can access applications from any computer with a browser and not have to worry about data backup. But to get there, there were a few challenges that needed to be solved first.

The challenges lie in the way users interact with existing desktop applications, which were mostly _ sovereign applications. _ With the traditional server-rendered web pages, When a user visits a web page, there is normally a small lag between when a button or a link is clicked (I'll explain where this lag in time is coming from in the next section) because a round trip to the server is needed, and the whole page needs to be redrawn.

If it's a typical weather site, the small lag doesn't bother the user much since the task itself - checking the weather forecast - is a transient task that the user doesn't have to endure for a long period of time. However, if it's an application that is typically used for a few hours at a time, i.e. a sovereign application. , it would drive the user nuts. How would you feel if there was a half of a second lag in Microsoft Word every time you press a button?😫 It would get extremely frustrating, wouldn't it? So until this problem was solved, web developers back then accepted their fate and designed their websites to be used as transient applications instead.

Now that you understand the challenges that web developers were trying to solve. Before we move on to how AJAX delivers the solution, let's first try to explore where the time lag is coming from.

Local vs. Remote Procedure Calls

Why did desktop applications have the _ richness _ and _ responsiveness _ that seemed to lack in the classic web applications? In order to understand where the time lag presented in a typical web application is coming from, we need to understand the differences between local and remote procedure calls.

In _ local procedure calls _, when a _ non-networked _ piece of code is compiled or interpreted, the various methods and functions are coded as instructions stored in the same _ local memory _ as the _ data _ on which the _ methods _ operate. Thus, passing data to a method and returning a result is pretty straightforward. Local procedures are widely used in desktop applications since the logic and data model are both executed in a _ closed environment _. Let's use an Excel desktop application as an example, a spreadsheet sits on its own little pile of data, stored locally in memory and on the local file system.

In contrast with a typical web page, in which a web browser has to contact the webserver sitting somewhere else (could be on the other side of the world) to request the pages that the user is trying to access. In _ remote procedure calls _, a lot of computation is going on "under the hood" at both ends of a network connection in order to send and receive data:

1. Encode and serialize (convert into a linear set of bytes) the request from the caller.
2. Pass the serialized data to the HTTP application protocol
3. The data packet is then sent across the physical transport (cable, wireless connection, etc.)
4. The remote machine receives the data and starts decoding and deserializing it before executing the requested procedure on the received data.
5. The response is then encoded, serialized before sending over the wire back to the caller.

Network Latency

The grand vision of the Internet age is that all computers in the world interconnect as one very large computing resource. The idea was that _ remote _ and _ local procedure calls _ become indistinguishable, and issuers of the calls are no longer even aware of which physical machines they are working on. Unfortunately, communications over a network are expensive as remote and local procedures are not the same thing at all. As mentioned in the section above, communications over a network are slow and unreliable.

So, making a call over a network will never be as efficient as calling a local method in memory. I used to think that the physical journey along the wire is the most time-consuming process, but it's actually the computations under the hood that slow things down. Furthermore, the unreliability of the network, which is why lost packets of information need to be resent, only makes the matter worse. This delay, which is referred to as _ network latency _, is a common cause of delayed responsiveness between users' initiated actions and the page's responses.

HTTP

As mentioned above, the HTTP protocol is used to transport serialized data between the requester (browser) and the server.

Modern JavaScript provides a number of ways to send HTTP requests to remote servers. An HTTP request consists of a request by the browser, followed by a response from the server. An HTTP request is mostly composed of headers , with the body containing some data or parameters. Think of the headers as lines of an address written on an envelope and the body as the letter inside. The headers simply instruct the receiving party what to do with the letter contents. The response typically contains the HTML markup for the returning page.

Asynchronous Requests

As mentioned above, _ network latency _ is inherent in _ remote procedure calls _ and is the common cause of delayed responsiveness in web applications. In other words, we can't get rid of network latency. So how are we going to solve this problem then? 🤔

There was only one sane solution to the network latency problem: _ we must try to make the UI responses independent of network activity. _ But HOW?🤔

Asynchronous remote event handling comes to the rescue.🦸‍♂️ Fortunately, although a half a second delay, so to speak, each time the user clicks a button is not acceptable, a slight holding response to acknowledge the receipt of a request is often sufficient, as long as it is timely. Using an example from our physical world, when you make your morning coffee, you don't stand next to the coffee machine staring at it while waiting for it to brew, do you? So why are we doing that in our computer world?😉

Seriously, there should be no reason for the user to stare at the computer screen waiting for the newly requested web page to arrive. Let's use the Amazon website as an example. After adding a new item to my shopping basket, I should be able to browse other items on the site while the total cost for all items in my basket is being calculated, right?

With any UI, it's a well-established practice to spawn an asynchronous thread to handle any lengthy piece of computation and let it run in the background while the user gets on with other tasks. The user is necessarily blocked while that thread is _ launched, _ but this can be done in an acceptably short span of time. So, _ asynchronous remote event handling _ is not a new concept, and so far, it seems like we've already figured out the solution to our problem, which is _ asynchronously processing remote calls _. So what is so revolutionary about AJAX anyway?

Here is the catch!😜 Unfortunately for us web developers, HTTP is a request-response protocol. That is, the client issues a request for a document, and the server responds, either by delivering the document, saying that it can't find it, offering an alternative location, or telling the client to use its cached copy, etc. In other words, a request-response protocol is one-way. The client can make contact with the server, but the server cannot initiate communication with the client. As a matter of fact, the server doesn't remember the client from one request to the next, and it shouldn't have to. Okay, so where is the catch again?😁

So we said earlier that the key feature of our asynchronous callback solution is that the client gets notified twice: once when the thread is spawned and again when the thread is completed. How can the server notify the client that it has completed the request if HTTP protocol is a _ one-way communication? _ Aha!😜 But don't worry! That's what AJAX is for.😎

Now that you understand AJAX's mission, we are now ready to learn what AJAX really is. See you in the next part of this series! 👋