Limit the Number of Pre-rendered Pages in Next.js

They say that history repeats itself, and it really does ring true for so many things in our lives.

A little bit of Back in the Day ...

Back when the "World Wide Web" first started getting popular, websites were static through and through, because reactive and dynamic web tech hadn't been invented yet.

Enter the dynamic web ...

Then came the dynamic web, and with it, the ability to build sites that were reactive and interactive.

Initially this was done with server-side environments/languages like Java, PHP, ASP, JSP, etc. combined with JavaScript on the client-side, but eventually client-side JavaScript libraries like jQuery started to appear. There was even Flash ... remember that? 😂

But it wasn't really until frameworks like Angular, and libraries like React and Vue came along, that the pendulum swung the other way, and static sites were now seen as old and outdated.

This was a huge leap forward, and it was amazing to see what could be done with this new technology.

The new hotness was to build your site as a single-page application (SPA), and let the client-side JavaScript handle all the rendering. This was an amazing way of doing things, but the issue with this approach is that it's not very SEO friendly, and for very large sites, the initial load time can be quite long.

Hello meta-frameworks ...

More recently, with the advent of meta-frameworks like Next.js, Remix.js, Astro, et al., and the ability to pre-render your site at build time, static sites are back on the menu once again.

But there can be issues with this approach as well, especially for very large sites: pre-rendering your entire site at build time can be very costly, both in terms of build time, as well as the actual dollar cost of running the build.

The Problem

I stumbled on a reddit thread (or maybe it was a Stackoverflow question?) the other day, where someone was asking about migrating a site with over 250,000 pages to Next.js. I think it was an e-comm site or something, but I can't seem to find the thread again. Anyway, the gist of the question was that they were concerned about the build time and cost of pre-rendering all those pages.

Now I have no idea how long it would take to pre-render 250,000+ pages, but I can promise you it wouldn't be quick. If I could make a rough estimate, I'd wager you'd be looking at about 2 hours or so, give or take.

Now that's bad enough, but the real issue is that you'd have to do that every time you update the site. And if you're running a business, you're going to want to update your site fairly regularly, so that's a lot of time and money spent (wasted) on builds.

In an ideal world...

Ideally, you'd be able to do the initial pre-rendering of all your pages in Next.js on your very first build, and then only re-build the pages that have changed since the last build on subsequent builds. This would save a ton of time and money, and would be a much more efficient way of doing things.

I searched online to see if Next.js had this ability, but couldn't find anything. Part of the issue is that with cloud hosts like Vercel and Netlify, your code is pulled fresh from your repo every time you build, so there's no way to know which pages have changed since the last build.

I did see ISR (Incremental Static Regeneration) mentioned a few times, but that's not really what I was looking for. ISR is great for pages that are updated frequently, but it doesn't help with the initial build time.

So what's a Next.js dev to do?

The Solution (well ... sort of)

Unfortunately I don't have a solution for only re-rendering pages that have changed since the last build, but there is a way to limit the number of pages that are pre-rendered at build time.

So while limiting the number of pre-built pages isn't perfect, it's still a good compromise, and might save you time and money in the long run.

The Gist

The gist of this solution is to only spit out the pages that you want pre-rendered in the getStaticPaths function of your page. This is the function that tells Next.js which pages to pre-render at build time.

You could base it on which of your pages are the most popular, or which ones are the most likely to be visited, or you could just pick a random selection of pages. Alternatively, if you're dealing with a blog for example, you could just pre-render the 50-100 most recent posts, the rest of which would get rendered automatically at runtime.

The Code

The following is an (incomplete) example of how you could achieve this:

// pages/[slug].js

import { getAllPosts, getPostBySlug } from '../../lib/api';

export default function Post({ post }) {
  return (
    <div>
      <h1>{post.title}</h1>
      <div dangerouslySetInnerHTML={{ __html: post.content }} />
    </div>
  );
}

export async function getStaticPaths() {
  const posts = getAllPosts();

  // Sort posts by published date, descending
  let sortedPosts = posts.sort((post1, post2) =>
    post1.published > post2.published ? -1 : 1
  );

  // Limit the number of pre-rendered pages to 50
  const paths = sortedPosts.slice(0, 50).map((post) => ({
    params: { slug: post.slug },
  }));

  return {
    paths,
    fallback: 'blocking',
  };
}

export async function getStaticProps({ params }) {
  const post = getPostBySlug(params.slug);

  // Check to make sure the page exists, and return a 404 if it doesn't
  if (post.slug === null) {
    return {
      notFound: true,
    };
  }

  return {
    props: {
      post: {
        ...post,
        content: post.content || '',
      },
    },
  };
}

Now, I've made some assumptions here, namely that you have a lib/api TS or JS file that contains functions for getting all your posts, and getting a single post by slug. I've also assumed that your posts have a published date, and a slug property.

In my own code, I do the sorting by date in the getAllPosts() function, but I included it here in getStaticPaths() for clarity.

Key Points

• In this example, we're using the slice() method to limit the number of pages that are pre-rendered to 50. You could also use filter() to only pre-render pages that meet certain criteria, or any other array-manipulation method you like.

  In my own code, like with the sorting, I actually slice the array in the getAllPosts() function, but included it here in getStaticPaths() once again for clarity.

• The fallback: 'blocking' key in the return object in getStaticPaths(). This tells Next.js to render any pages that weren't pre-rendered at build time at runtime instead, since they haven't yet been rendered.

  Alternatively, you could set fallback to true, which will also render the extra pages at runtime, but will show a fallback page while the page is being rendered, whereas setting it to 'blocking' will block loading until the page is rendered, similar to SSR.

  I'm using 'blocking' here, because I don't want to show a fallback page, and I don't want to show a loading indicator either. I'd rather just have the page load instantly, even if it takes a few seconds to render, but that's just my preference.

• The notFound: true key in the return object in getStaticProps(). This tells Next.js to return a 404 page if the page doesn't exist. When fallback is set to false, Next.js knows to issue a 404 for any pages that aren't in the paths array, but when fallback is set to true or 'blocking', it doesn't know which paths/files exist or not, so you have to explicitly tell it.

The Result

So what does this look like in practice?

Well, if you visit a page that's in the paths array, because it was pre-rendered at build time, you'll see the page load instantly.

On the other hand, if you visit a page that isn't in the paths array, it will be rendered at runtime, and you might see a loading indicator (or a fallback page if you set fallback: true) while the page is being rendered, depending on the size of the page and the speed of your connection.

Note: The rendering of non-pre-rendered pages will only happen once, after which the page will be cached for future visits. So it's not like you'll be waiting for the page to render every time you visit it.

Conclusion

So there you have it. While this solution isn't perfect, it's a good compromise, and might save you time and money in the long run.

Know of a way in Next.js to only re-render pages that have changed since the last build? Please, let me know in the comments!! I'd love to hear about it. 😁