How to add a blog using Dev.to as a CMS to a Next.js website

For a shorter introduction (about half the length) check out "I completely rewrote my personal website using Dev.to as a CMS".

Preface

I've been posting on Dev.to for a few months now. I love the platform, the editor, the ability to draft, edit and publish an article making it available to the millions of Dev.to users.

Recently, I decided that I wanted to present them on my own website. After researching different ways to achieve this, I concluded using the Dev.to API to create the blog section of my website would be the perfect solution. I decided that articles would only show up on my website if I'd added a canonical URL to the article on Dev.to - meaning my website is seen as the source of the article (even though it was written on Dev.to).

Continuing to use Dev.to also means that I don't need to configure storage for saving the articles or any images used. Additionally, I can take advantage of the built-in RSS feed which other blogging sites can read to automatically import my articles.

I came up with the following list of requirements:

1. Use the Dev.to API to fetch all my articles and display them on my website.
2. Fetch and render each article at build time to ensure the website would be fast and to ensure good SEO for the individual blog pages. Using dynamic pages would make the website load slower as it would query the Dev.to API on the client-side and also mean that I would have the same SEO data, such as page title, for each blog page.
3. Set the canonical URL of an article on Dev.to and have that be the article's URL on my website. I wanted to continue to use the Dev.to editor to write and manage my articles, so they should only show on my website once I've added a canonical URL.
4. Have a nice URL for the blog posts on my website that I would be in complete control of. Neither the post ID nor the Dev.to path to the article.
5. Rebuild each time an article is created or updated. This was crucial as the blog would be static - I didn't want to press the rebuild each time I changed something.

I was able to achieve all of this using a combination of Next.js dynamic pages, Vercel deploy hooks and the public Dev.to API.

Setting up the project

Key technologies used

1. TypeScript - if you prefer plain JavaScript for code examples, this GitHub repository has the same functionality as described below but is purely JavaScript.
2. Next.js, React.js etc (required to create a Next.js app).
3. Tailwind CSS, Tailwind CSS Typography plugin (for styling).
4. Remark Markdown parser and plugins such as remark-html to convert the Markdown returned by the Dev.to API to HTML. Other plugins I use enable features such as code highlighting, GitHub flavour Markdown compatibility (for strikethrough etc) and stripping out Front Matter from the displayed HTML.
5. The Dev.to API and it's https://dev.to/api/articles/me endpoint.
6. Vercel deploy hooks. I use Vercel to host my Next.js site and their deploy hooks allow me to rebuild my website automatically when an article is added or edited on Dev.to.

To see all the packages I'm currently using on my website, check out the package.json on GitHub.

The two Next.js functions that run my website

My personal website is built using Next.js. To ensure that all content continued to be generated at build time, I used two built-in Next.js functions that can be used to fetch data for pre-rendering. These are:

• getStaticProps - fetch data from a source (think API or file) and pass it into the component via props.
• getStaticPaths- provides the ability to use dynamic routes with a static site.

I'll be using both functions to make the dynamic article page called [slug].ts - the square brackets denote that it is a Next.js dynamic page and the name slug is the name of the parameter that will be passed into getStaticProps from getStaticPaths.

How do I determine which articles appear on my website?

For articles to appear on my website they have to have a canonical URL pointing at https://wallis.dev/blog.

Whenever I refer to the page slug I'm referring to the last section of the canonical URL (after /blog). When reading the canonical URL from the Dev.to API I use the following function to convert the URL to the slug.

const websiteURL = 'https://wallis.dev/blog/';

// Takes a URL and returns the relative slug to your website
export const convertCanonicalURLToRelative = (canonicalURL) => {
    return canonicalURL.replace(websiteURL, '');
}

When I pass https://wallis.dev/blog/a-new-article to convertCanonicalURLToRelative it will return the slug a-new-article.

How to add a blog with using Dev.to as a backend

The individual article pages (/blog/${slug})

Overview

Each individual article page is generated at build time using the getStaticPaths Next.js function that fetches all my Dev.to published articles, and saves them to a cache file. getStaticProps then fetches an individual article from the cache and passes it into the page component via its props.

A cache file must be used because Next.js doesn't allow passing data from getStaticPaths to getStaticProps - aside from the page slug. For this reason, the page slug is used to fetch an article from the cache file.

Flow Diagram

The diagram below should explain the process that is followed when creating dynamic pages through Next.js using the getStaticPaths and getStaticProps functions. It outlines the most important function calls, briefly explains what they do, and what is returned.

Implementation

View on GitHub

Below you will find the code that dynamically creates each article page.

import fs from 'fs';
import path from 'path';

import Layout from '../../components/Layout';
import PageTitle from '../../components/PageTitle';
import IArticle from '../../interfaces/IArticle';
import { getAllBlogArticles, getArticleFromCache } from '../../lib/devto';

const cacheFile = '.dev-to-cache.json';

interface IProps {
    article: IArticle
}

const ArticlePage = ({ article }: IProps) => (
    <Layout title={article.title} description={article.description}>
        <img
            src={article.coverImage}
            alt={`Cover image for ${article.title}`}
            className="md:mt-6 lg:mt-10 xl:mt-14 h-40 sm:h-48 md:h-52 lg:h-64 xl:h-68 2xl:h-80 mx-auto"
        />
        <PageTitle title={article.title} center icons={false} />
        <section className="mt-10 font-light leading-relaxed w-full flex flex-col items-center">
            <article className="prose dark:prose-dark lg:prose-lg w-full md:w-5/6 xl:w-9/12" dangerouslySetInnerHTML={{ __html: article.html }} />
        </section>
    </Layout>

)

export async function getStaticProps({ params }: { params: { slug: string }}) {
    // Read cache and parse to object
    const cacheContents = fs.readFileSync(path.join(process.cwd(), cacheFile), 'utf-8');
    const cache = JSON.parse(cacheContents);

    // Fetch the article from the cache
    const article: IArticle = await getArticleFromCache(cache, params.slug);

    return { props: { article } }
}

export async function getStaticPaths() {
    // Get the published articles and cache them for use in getStaticProps
    const articles: IArticle[] = await getAllBlogArticles();

    // Save article data to cache file
    fs.writeFileSync(path.join(process.cwd(), cacheFile), JSON.stringify(articles));

    // Get the paths we want to pre-render based on posts
    const paths = articles.map(({ slug }) => {
        return {
            params: { slug },
        }
    })

    // We'll pre-render only these paths at build time.
    // { fallback: false } means other routes should 404.
    return { paths, fallback: false }
}

export default ArticlePage

The flow diagram above combined with the comments throughout the code should enable a full understanding of the code. If you have any questions, comment below.

You'll notice that two functions are called from the lib/dev.ts file. getArticleFromCache does what it suggests, it finds an article in the cache and returns it. getAllBlogArticles, on the other hand, is the function that fetches all my articles from Dev.to and converts the supplied markdown into HTML - using functions from lib/markdown.ts.

Devto.ts

import axios, { AxiosResponse } from 'axios';
import IArticle from '../interfaces/IArticle';
import ICachedArticle from '../interfaces/ICachedArticle';
import { convertMarkdownToHtml, sanitizeDevToMarkdown } from './markdown';

const username = 'jameswallis'; // My Dev.to username
const blogURL = 'https://wallis.dev/blog/'; // Prefix for article pages

// Takes a URL and returns the relative slug to your website
export const convertCanonicalURLToRelative = (canonical: string) => {
    return canonical.replace(blogURL, '');
}

// Takes the data for an article returned by the Dev.to API and:
// * Parses it into the IArticle interface
// * Converts the full canonical URL into a relative slug to be used in getStaticPaths
// * Converts the supplied markdown into HTML (it does a little sanitising as Dev.to allows markdown headers (##) with out a trailing space
const convertDevtoResponseToArticle = (data: any): IArticle => {
    const slug = convertCanonicalURLToRelative(data.canonical_url);
    const markdown = sanitizeDevToMarkdown(data.body_markdown);
    const html = convertMarkdownToHtml(markdown);

    const article: IArticle = {
        // parse into article object
    }
    return article;
}

// Filters out any articles that are not meant for the blog page
const blogFilter = (article: IArticle) => article.canonical.startsWith(blogURL);

// Get all users articles from Dev.to
// Use the authenticated Dev.to article route to get the article markdown included
export const getAllArticles = async () => {
    const params = { username, per_page: 1000 };
    const headers = { 'api-key': process.env.DEVTO_APIKEY };
    const { data }: AxiosResponse = await axios.get(`https://dev.to/api/articles/me`, { params, headers });
    const articles: IArticle[] = data.map(convertDevtoResponseToArticle);
    return articles;
}

// Get all articles from Dev.to meant for the blog page
export const getAllBlogArticles = async () => {
    const articles = await getAllArticles();
    return articles.filter(blogFilter);
}

// Get my latest published article meant for the blog (and portfolio) pages
export const getLatestBlogAndPortfolioArticle = async () => {
    const articles = await getAllArticles();
    const [latestBlog] = articles.filter(blogFilter);
    const [latestPortfolio] = articles.filter(portfolioFilter); // ignore this! It's meant for another page (see the wallis.dev GitHub repository for more information)
    return [latestBlog, latestPortfolio];
}

// Gets an article from Dev.to using the ID that was saved to the cache earlier
export const getArticleFromCache = async (cache: ICachedArticle[], slug: string) => {
    // Get minified post from cache
    const article = cache.find(cachedArticle => cachedArticle.slug === slug) as IArticle;
    return article;
}

The key points to note about the devto.ts file is:

1. I've used the authenticated https://dev.to/api/articles/me endpoint to fetch all my articles from Dev.to. This endpoint is the only one that returns all my articles (ok, 1000 max...) and includes the article markdown. Authenticating also gives a slightly higher API limit.

   • Previously I used the built-in HTML returned in the https://dev.to/api/articles/{id} but I kept hitting the API limit as each build made as many API calls as I had articles.
   • Get a Dev.to API Token following the instructions on the API docs.

2. The convertDevtoResponseToArticle function converts the markdown into HTML using a function from the lib/markdown.ts.

Markdown.ts

import unified from 'unified';
import parse from 'remark-parse';
import remarkHtml from 'remark-html';
import * as highlight from 'remark-highlight.js';
import gfm from 'remark-gfm';
import matter from 'gray-matter';
import stripHtmlComments from 'strip-html-comments';

// Corrects some Markdown specific to Dev.to
export const sanitizeDevToMarkdown = (markdown: string) => {
    let correctedMarkdown = '';

    // Dev.to sometimes turns "# header" into "# header"
    const replaceSpaceCharRegex = new RegExp(String.fromCharCode(160), "g");
    correctedMarkdown = markdown.replace(replaceSpaceCharRegex, " ");

    // Dev.to allows headers with no space after the hashtag (I don't use # on Dev.to due to the title)
    const addSpaceAfterHeaderHashtagRegex = /##(?=[a-z|A-Z])/g;
    return correctedMarkdown.replace(addSpaceAfterHeaderHashtagRegex, '$& ');
}

// Converts given markdown into HTML
// Splits the gray-matter from markdown and returns that as well
export const convertMarkdownToHtml = (markdown: string) => {
    const { content } = matter(markdown);

    const html = unified()
        .use(parse)
        .use(gfm) // Allow GitHub flavoured markdown
        .use(highlight) // Add code highlighting
        .use(remarkHtml) // Convert to HTML
        .processSync(stripHtmlComments(content)).contents;

    return String(html);
}

This file is pretty simple; the comments should explain everything, so I won't add anything more. If you'd like to learn more about using Remark converts with Next.js, you can read my blog titled "How to use the Remark Markdown converters with Next.js projects".

Summary

Phew, that was a lot. Hopefully, I didn't lose you in the code examples and explanations!

Everything above explains how I've built the dynamic article pages on my website. I've included all the code that you'll need to create the dynamic blog pages on your own website.

By the way, when the code above is compiled it produces an article page such as https://wallis.dev/blog/nextjs-serverside-data-fetching.

Let's move onto the blog overview page (wallis.dev/blog).

The article overview page (/blog)

Building a page for each of your Dev.to articles at build time is great but how will a user find them without an overview page?! They probably won't!

Overview

The overview page is much simpler than the dynamic article pages and only uses functions from the lib/devto.ts file introduced above. So this section will be shorter than the last.

Flow Diagram

As before, I've made a diagram to display the process followed when displaying all the article summaries on the overview page. You'll notice that this time I'm only using getStaticProps rather than getStaticProps and getStaticPaths. This is because I'm only loading data for one page rather than creating dynamic pages (which is what getStaticPaths allows you to do).

Implementation

View on GitHub

import Layout from '../components/Layout'
import PageTitle from '../components/PageTitle'
import Section from '../components/Section'
import ArticleCard from '../components/ArticleCard'
import IArticle from '../interfaces/IArticle'
import { getAllBlogArticles } from '../lib/devto'

interface IProps {
    articles: IArticle[]
}

const title = "Blog ✍️"
const subtitle = "I share anything that may help others, technologies I\'m using and cool things I\'ve made."

const BlogPage = ({ articles }: IProps) => (
    <Layout title={title} description={subtitle}>
        <PageTitle
            title={title}
            subtitle={subtitle}
        />

        <Section linebreak>
            {articles.map(({ title, description, publishedAt, tags, canonical }) => (
                <ArticleCard
                    key={title}
                    title={title}
                    description={description}
                    date={publishedAt}
                    tags={tags}
                    canonical={canonical}
                />
            ))}
        </Section>
    </Layout>
)

export async function getStaticProps() {
    // Get all the articles that have a canonical URL pointed to your blog
    const articles = await getAllBlogArticles();

    // Pass articles to the page via props
    return { props: { articles } };
}

export default BlogPage

Essentially the above code:

1. Loads the articles from the Dev.to API
2. Passes them into the component
3. Maps over each article and creates a summary card for each which links to the dynamic article page created in the previous step.

The overview page looks like this:

Summary

Amazing, that's the overview page complete! If you're following along you should now have:

1. Blog pages being created dynamically
2. An overview page that links to the dynamic blog pages

Rebuild each time an article is created or updated

The final step that I took to create my Dev.to powered website is to set up a Vercel deploy hook. My website is hosted on Vercel so I am able to use a deploy hook to programmatically trigger a rebuild, refreshing the article content in the process.

Deploy Hooks allow you to create URLs that accept HTTP POST requests in order to trigger deployments and re-run the Build Step.

• https://vercel.com/docs/more/deploy-hooks

To trigger the deploy hook, I have created a Dev.to API webhook that calls it each time an article is created or updated.

Configuring the automatic rebuild

A prereq for this section is that you're website needs to be deployed onto Vercel. I've created instructions on how to do this.

To create a deploy hook, follow the Vercel documentation - it's a lot more simple than you'd think.

Once you have the deploy URL we can use the Dev.to API to create a webhook to trigger it.

You can do this using curl (make sure you add your API_KEY and change the target_url to be your Vercel deploy hook URL):

curl -X POST -H "Content-Type: application/json" \
  -H "api-key: API_KEY" \
  -d '{"webhook_endpoint":{"target_url":"https://example.org/webhooks/webhook1","source":"DEV","events":["article_created", "article_updated"]}}' \
  https://dev.to/api/webhooks

For more information, see the Dev.to API docs.

Summary

Nice one, now your website will automatically redeploy each time you create or update an article on Dev.to!

Next steps

I love my website right now and using Dev.to to manage most of its content has made adding content much more efficient than previously. However, there are a couple of things I want to improve in the future:

• If a user is viewing a blog on Dev.to and it links to another of my articles, the user should stay on Dev.to. But if they're on wallis.dev, they should stay on it rather than being taken to Dev.to.
• Another Dev.to user made a comment in another of my articles and made the point that if Dev.to suddenly turned off, I'd lose my articles. However unlikely, I want to set up a system to take daily backups of my articles to mitigate the risk of losing them.

Round up

In this article, I've taken you through the code that allows Dev.to to power my website. If you venture onto my GitHub you'll see that in addition to having a blog section (https://wallis.dev/blog), I also use Dev.to to display my portfolio entries (https://wallis.dev/portfolio).

If you want more background on why and how I've used the Dev.to API to power my website, read my initial post discussing it.

If you found this article interesting or it has helped you to use Next.js and the Dev.to API to build your own website using Dev.to as a CMS, drop me a reaction or let me know in the comments!

Anything I can improve? Let me know in the comments.

Thanks for reading!

PS, I'm currently deciding whether I should create a tutorial series that will take you through building a Dev.to powered blog from scratch - is this something you would read/follow?