Cache-Control explained

#http #webdev #cache

HTTP Caching is a mechanism that helps developers speed up web content loading. It uses a set of headers to instruct local and intermediate caches to store responses and serve them on subsequent requests without going to the origin server repeatedly. This post will focus on the Cache-Control header, one of the most essential headers associated with HTTP caching.

But first, why do we even need caching on the web?

Well, modern websites and web apps are heavy. They contain various assets: images, documents, audio files, CSS stylesheets, and client-side libraries. Frequent download of these files slows page loading, hurts SEO scoring, and increases the load on the origin server.

HTTP caching helps to mitigate these issues in two ways:

Caching resources saves the user a trip to the origin server, as the browser can serve those cached assets without needing to access the remote server.
Modern CDNs cache responses from remote servers on "edge servers" located close to the user's machine. This feature helps to reduce latency and improve page load times.

The Cache-Control Header

Cache-Control is a standard HTTP header that can appear on both the request and response objects. It contains one or more directives, separated by a comma. This header instructs the cache on how and when to store the response from the origin server and how to serve it for future requests.

Here's the structure of the Cache-Control header:

Cache-Control: directive, [directive, ...]

Before we dive into the technicality of cache control, let's review the terminology we'll use in this article:

Private vs Shared Cache: Private cache is the local cache on the user's machine. Shared cache, on the other hand, refers to intermediate caches (e.g., CDN, ISP cache, proxies, etc.).
Age: The time passed since a response was generated. It is measured in seconds.
Max Age: A measure that determines the "freshness" of a cache item. If the cache item's age is less than its max age, it is said to be "fresh". Otherwise, it is considered "stale".
Revalidation: When a cache item becomes stale, the cache will ask the origin server to confirm its validity before serving it to future users. This process is called "revalidation". It is done by sending a conditional request to the origin server requesting the most up-to-date version of the item. The origin server will either confirm the validity of the cached item or reply with the newer version.

Note: If the cache is disconnected from the origin server, it can return stale items until it reconnects.
Conditional Request: This type of request is sent during revalidation. It's a GET request that contains one or more validator headers. (usually, the If-Modified-Since or the If-None-Match header, in the case of cache revalidation).

The server will reply with a 304 Not Modified status if the version of the resource on the server is the same as the version provided in the request or with the new version otherwise.
Cache Busting: A technique where a static resource contains a unique hash/version in its filename. Whenever the content of the file changes, the unique hash/version in its name also changes. Using this technique "locks" the URL of a resource to a specific version of that resource (e.g., /static/react-16.0.3.js).

A common trap in HTTP caching is when the developer sets a long max-age for a file (we'll learn more about max-age in the next section), and during that time, he makes changes to the file, but since the old version in the cache is still fresh, users who request the file, get the old version from the cache.

The "Cache Busting" technique avoids this trap because when the file content changes - so does its URL. Users who want to get the newer version of the file - only need to request the new URL of the file. Since the cache associates each item to a URL and each file version has a "locked" URL - this guarantees that no URL will have two versions associated with it. Later, in the use cases section, we'll see other ways to handle this trap.

HTTP Cache-Control Flow

When a user requests a resource for the first time, the server responds with the resource and includes a Cache-Control header to indicate how and when to store the response and use it again. Intermediate caches, as well as the browser's local cache, store the response accordingly. The next time a user requests that resource, the browser will try to serve it from its local cache. If the resource has gone stale, the browser will dispatch a conditional request to intermediate caches. If their version has gone stale, they'll forward it to the remote server. The server will check the version of the resource provided in the request and compare it to its copy of the resource. If the version of the requested resource is the same as the server's version of that resource, it'll respond with a 304 Not Modified status, indicating to the caches that their version is still valid and they can turn it fresh again and use it. However, if the server has a newer version of the resource, it'll respond with a 200 OK status and the caches will update their copy.

Okay, now that the process of HTTP caching is clear, let's review the most common Cache-Control directives.

Directive	Description
`max-age`	The `max-age` directive instructs the cache for how long to keep the response fresh. The value for this directive is the time measured in seconds. After that period, the response will become stale, and the cache will try to revalidate the response with the origin server before using it again.
`s-maxage`	Similar to `max-age`, but shared caches (e.g., CDNs, proxy servers) will ignore `max-age` if this directive is also present. This directive can be used along with the `max-age` directive to give the response a different max age on shared vs. private caches.
`private`	This directive indicates that the response is intended for a single user (e.g., personal page) and can be stored only in private caches (i.e., browser cache).
`no-cache`	Allows caches to store the response, but they must revalidate it every time before using it. This has the benefit of always getting the most up-to-date response while not having to generate it every time (if the response has not been changed on the origin server, the cache will reply with its copy).
`no-store`	This directive prohibits caches from storing the response. They should forward the request to the origin server every time.
`must-revalidate`	This directive prohibits the caches from returning stale responses under any circumstances (see "revalidation"). If the origin server is unreachable, the cache will reply with a 504 Gateway Timeout status.
`immutable`	When a user clicks the reload button, the browser revalidates its local cache, even the fresh items. For a page containing many static files that do not change - this results in useless revalidation requests. To prevent that, add the `immutable` directive to instruct the cache never to revalidate these items while they're fresh. This directive is used when implementing the "Cache Busting" technique.
`stale-while-revalidate`	When a response becomes stale, the cache waits until the revalidation response returns from the origin server to resolve the request. This delay can negatively affect the user experience (especially if the `max-age` of the response is relatively short, leading to more frequent revalidations). This directive specifies a period of time after a response went stale, during which the cache is allowed to serve the stale response while revalidating it in the background for the next request.

Use Cases

Here, we'll look at common scenarios and what cache directives we should apply for each case.

Static Web Page Think of a web page showing similar content to all users, like a blog post. Such a page will not update frequently, and usually, when there is an update, it's okay to display an outdated version of the page for some time until the update propagates to all caches. We can take here the most naive approach:

  Cache-Control: max-age=604800;

We instruct the cache to store the page for a week (604800 seconds). If the page is updated during that time, the user will still see the old version until the cached page becomes stale, and then the cache will fetch the updated version from the origin server.

In this example, changes to the page will take up to a week to be shown to all users.

You can modify the number of seconds in the example above to cache the page for a shorter or longer time.

Sometimes, we want changes to propagate as soon as possible to the users. In this case, we can take a slightly different approach: We'll cache the page on the user's browser for a short time (e.g., 10 minutes) and cache it on the shared cache for a longer time (e.g., a week). When we want to update the page, we can deploy our change, clear the shared cache manually, and wait up to 10 minutes for the cached page on the user's browser cache to go stale. After 10 minutes, the users will get the most up-to-date version of our page from the origin server/shared cache.

  Cache-Control: max-age=600, s-maxage=604800, stale-while-revalidate=86400;

I added stale-while-revalidate to prevent users from "hanging" while the browser revalidates the page. See above for an explanation of this directive.

Real-Time Information Let's shift our focus to a news website or a weather report website. These websites update frequently and our users expect to see the most up-to-date information when they visit them. In this scenario, we can still cache the page, but we have to revalidate it every time with the server:

  Cache-Control: no-cache, max-age=0, must-revalidate;

The first directive allows the cache to store the page, but it has to revalidate the page every time with the origin server before serving it to users. The subsequent two directives are "fallback directives" for caches that do not support the first directive: We set the max age of the response to zero, and we tell the cache that it has to revalidate the response after it has gone stale (i.e., immediately after it was received).

Immutable Assets These assets include CSS stylesheets, JS libraries, images, audio and video files, documents, etc. Since these files are included in the page (e.g., via links) and not accessed directly by users, they should be versioned and immutable.

  Cache-Control: max-age=31536000, immutable;

We can cache these files for a year since their URL is fixed to their current version, and we add the immutable directive to prevent the browser from revalidating them.

FAQ

Here, I'll answer the most common questions about cache control:

What happens if no Cache-Control header is present on the response?
The behavior is unspecified, and the cache can decide whether to cache the response. To prevent caching of a response, specify it explicitly with:

Cache-Control: no-store.
What is the difference between no-store, no-cache, and must-revalidate?
no-store - don't save the response in the cache. The origin server needs to generate the response for every request.
no-cache - save the response in the cache, but revalidate it with the origin server for every incoming request.
must-revalidate - don't serve stale responses without revalidating them. If the origin server is not accessible, return a "504 Gateway Timeout" response instead.
What is the difference between the public and private, and when should I use each?
Both directives allow a cache to store a response, even if it's not cacheable by default. The private directive enables this behavior only to private caches (i.e., browser cache), and the public directive allows the response to be stored on private and shared caches.
I see a lot of examples where the public directive is used. Should I use it as well?
Probably no. The public directive allows shared caches to store responses that otherwise are not cacheable by shared caches. An example is a response to a request that contains the Authorization header. Such response is access controlled and thus not cacheable by shared caches by default.

Shared caches automatically cache a response if it contains the s-maxage or the must-revalidate directives.

From RFC9111 section 5.2.2.9:

The public response directive indicates that a cache may store the response even if it would otherwise be prohibited ... Note that it is unnecessary to add the public directive to a response that is already cacheable.