Generating a sitemap and RSS feed with Next.js

There are many options for generating content with Next.js. These different methods can be used for automatically generating sitemaps and RSS feeds based on content fetched from a CMS. I’ll explore the different options available, their pros and cons.

The difference between Statically and Dynamically rendered content

Next.js terminology can be a little befuddling. To most people, SSR, SSG and ISR are nothing but meaningless acronyms. Fundamentally, Next.js projects can serve content in three different ways:

• Static content : These are files put in the /public directory and are served by next.js as they are. For example, an image stored at /public/image.png will be served at www.example.com/image.png.
• Build-time content : This content is created at built time. On a page, this is characterized by exporting a getStaticProps function. In Next.js jargon, they call it Static Site Generation (SSG), a name I don’t find very appropriate as the content isn’t completely static (it’s built), and the entire site doesn’t have to use this. It would be more appropriate to call this Build Generation.
• Dynamic content : This content is created every time someone visits a page. For example, this is what a PHP-based site does for every page written in PHP. On a Next.js page, this is characterized by exporting a getServerSideProps function. It’s commonly called Server Side Rendering (SSR), which isn’t very appropriate as it implies that other forms of generating content don’t involve any action from the server. I’ll call this Dynamic Generation.

As for Incremental Static Regeneration (ISR), it’s a balance between build-time and dynamic content: The content is generated at build time, but is regenerated in the background on every request (meaning that the visitor doesn’t get disrupted), in a specified window.

This is done by specifying a revalidate time:

export async function getStaticProps() {
  const res = await fetch(‘https://…/posts’);
  const posts = await res.json();

  // If the request was successful, return the posts
  // and revalidate every 10 seconds.
  return {
    props: {
      posts,
    },
    revalidate: 10, //This is the revalidate time in seconds
  };
}

Phew, that was a lot of information! Now we can finally get to the bottom of sitemaps and rss feeds. For each content serving method, one can serve files such as sitemaps or RSS feeds in an equal way.

Methods of serving built XML files

What is a sitemap?

A sitemap is an XML-formatted file that is basically an index of your site: It provides a way for crawlers (search engine robots) to see the pages available on your site, and index them properly. This improves visibility to search engines.

Static sitemaps

The first, most obvious option is to hand-code a sitemap and have it live in your /public folder. Here’s an example:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://felixrunquist.com</loc>
</url>
<url>
<loc>https://felixrunquist.com/posts</loc>
</url>
<url>
<loc>https://felixrunquist.com/contact</loc>
</url>
<url>
</urlset>

However, the obvious issue here is that it every time you add a new page to your site, or if you fetch content via CMS (Content Management System), you need to remember to hand-code the changes yourself. This can get quite tricky and time-consuming.

Building dynamic sitemaps

The next example is to add a sitemap.xml.js file in pages and use Dynamic Generation (or SSR if you prefer) using getServerSideProps. Since that function is run on every request, a req and res object is passed to the function, like an API, the res object is used to specify that the file type is XML and it is used to end the request before the main function is run. This is a method mentioned in Next.JS documentation.

/pages/sitemap.xml.js:

const URL = 'https://felixrunquist.com'
function generateSiteMap(posts) {
  return \`<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
       <loc>https://felixrunquist.com</loc>
     </url>
     <url>
       <loc>https://felixrunquist.com/posts</loc>
     </url>
     \${posts
       .map(({ id }) => {
         return \`
       <url>
           <loc>\${\`\${URL}/\${id}\`}</loc>
       </url>
     \`;
       })
       .join('')}
   </urlset>
 \`;
}

export default function SiteMap() {//This function is there just to avoid compile errors
}

export async function getServerSideProps({ res }) {
  const posts = await fetchPosts();//External function to fetch posts

  const sitemap = generateSiteMap(posts);// generate the XML sitemap with the posts data

  res.setHeader('Content-Type', 'text/xml');// send the XML to the browser
  res.write(sitemap);
  res.end();

  return {
    props: {},
  };
}

However, if you’re fetching data from a CMS, such as custom posts, this means that for every request to your Next.js site, there will also be one to the CMS which greatly increases overhead. This also affects loading times, as the sitemap won’t be served until all the content is fetched and built.

The best of both worlds

For my sitemap, I didn’t like the idea of having to manually update it every time, nor did I like the added overhead with Dynamic Generating. I found a way (advocated by Next.js) to generate a sitemap.xml file in the /public folder at build time. However, their method modifies the next.config.js file to run the function as a script.

Instead, I decided to call the function in the getStaticProps function of my index.js page. This means that the sitemap will be generated at build time, but I can also add a revalidate time in order to use incremental regeneration if I wish.

This is the idea: At build-time (or on regeneration), fetch the Next.js page content as well as any CMS page content, then compile XML sitemap code. Using the fs module, this will be written to pages/sitemap.xml.

/public/robots.txt:

User-agent: *
Allow: /

Sitemap: https://felixrunquist.com/sitemap.xml

/pages/index.js:

import {generateSitemap} from 'lib/generate.js'
export function getStaticProps(){
  generateSitemap();
  return {
    props: {}
  }
}

/lib/generate.js:

import fs from 'fs'
const SITE_PATH = 'https://felixrunquist.com';

function getSitemapXML(posts) {
  return \`<?xml version="1.0" encoding="UTF-8"?>
   <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
       <loc>\${SITE_PATH}</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/posts</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/contact</loc>
     </url>
     <url>
       <loc>\${SITE_PATH}/showcases</loc>
     </url>
     \${posts
       .map(({ node }) => {
         return \`
       <url>
           <loc>\${SITE_PATH + '/posts/' + node.slug}</loc>
       </url>
     \`;
       })
       .join('')}
   </urlset>
 \`;
}

export default async function generateSitemap(){
  const posts = await getPosts()//Fetch posts via API


  const sitemap = getSitemapXML(allPosts.edges);// Generate the XML sitemap with post data
  fs.writeFileSync('public/sitemap.xml', sitemap);//Write file as a statically-served file
}

To indicate to search engines where your sitemap file is, you also need to add a robots.txt file. Since this will be changed rarely, I added this as a static asset.

What about RSS feeds?

An RSS (Really Simple Syndication) feed is a file that lists the posts and pages of your website in a computer-readable XML format, which makes a standardized format for notifying other sites and services that you’ve published new content. This is most useful if you publish blog posts or other dynamic content regularly.

To serve this file, I’m going to use the example using Build Generation, which is the best option, in my opinion.

I used the feed package which makes exporting RSS data as XML really easy.

import fs from 'fs'
import { Feed } from 'feed'; //Uses the RSS feed generator library
import {SITE_PATH, SITE_DESC} from './constants';

export default async function generateRSS() {
   const feedOptions = {
    title: 'Posts | RSS Feed',
    description: SITE_DESC,
    id: SITE_PATH,
    link: SITE_PATH,
    image: \`\${SITE_PATH}/og-image.png\`,
    favicon: \`\${SITE_PATH}/favicon/favicon-32x32.png\`,
    copyright: \`All rights reserved \${new Date().getFullYear()}\`,
    generator: 'Feed for Node.js'
   };

   const feed = new Feed(feedOptions);
   const posts = await getPosts();//Fetch posts from API
   posts.edges.forEach(({node}) => {
     feed.addItem({
       title: node.title,
       id: \`\${SITE_PATH}/posts/${node.slug}\`,
       link: \`\${SITE_PATH}/posts/${node.slug}\`,
       description: node.excerpt,
       content: node.content,
       date: new Date(node.date),
      });
   });
   fs.writeFileSync('public/feed.xml', feed.rss2());//Write as static asset

}

You can also add a link tag to your header to indicate the presence of an RSS feed to crawlers and RSS readers:

<link rel="alternate" href="feed.xml" type="application/rss+xml" title="RSS feed for posts" />

The RSS can also used to cross-publish to different platforms such as dev.to, which I will explain in a future post.