Launch Week Day 4 - The Future of Headless

The tech ecosystem, especially frontend, changes at light speed. It gets overwhelming to try and keep up. But I kinda love change and I think it's once again time for change in the CMS world.

Don't wanna read this whole monologue? Then skip right to the good stuff. Spoiler alert, you can now deploy Payload serverlessly on Vercel within an existing NextJS app.

Steven Tey and I dove into the topics covered in this blog post on a livestream yesterday.

Back in ~2015, I was an early advocate for headless CMS because I wanted our agency to be able to use React for the frontends of the sites we built. I had a big pitch I'd give to my clients about the value of "headless" CMS, and they bit. But now we're taking a second to re-evaluate that pitch.

In my pitches, I would fire off the benefits of headless CMS, focusing on ideas like separation of concerns, more maintainable architecture, and the ability to deliver omnichannel content from a single source of truth.

But in retrospect, I had to gloss over the unfortunate side-effects of headless. The simple fact is that I was pushing my technical agenda on our clients simply because I wanted to use React. Granted, we built some killer stuff for our clients and React enabled that.

Lots of the other upsides to headless CMS were true, for sure. A CMS should obviously be API-first nowadays. That's a given that doesn't even need to be stated anymore. But I can count on one hand the amount of times we used a headless CMS for OmNiChAnNeL content. Most all of our headless CMS builds were just used to power a single website. Actually reading about how some of the big headless CMS position themselves as thought leaders around decentralized / omnichannel / content hub / etc. just makes me laugh at this point.

Anyway, our move to headless certainly posed struggles for us along the way, especially early on. Here's a few:

Keeping CMS changes in sync with the frontend

When we'd update a content model on the CMS side, we'd need to very deliberately think about how to deploy all of our changes to the CMS at the exact same time as our changes to our frontend to minimize downtime and prevent breaking code.

Doesn't matter if we used Contentful, Sanity, Wordpress, whatever—the CMS being separate meant that we had to orchestrate deployments with caution.

Sharing TypeScript types

A good CMS (like Payload 😈) can generate TypeScript interfaces for you which could theoretically be re-used on your frontend. But, with the separation of the CMS and the frontend, you have a few equally bad options to share these types:

• Manually copy / paste types from the CMS to the frontend repo
• Publish a third-party types package that can be leveraged for both backend and frontend (still need to copy / paste)
• Use a monorepo and get wild with scripts to automate your copy/pasting

Rough edges to do with "preview"

Back in the monolithic CMS days you'd log into your CMS and then browse around on your frontend. You'd have a nice little "admin bar" that was purpose-built to give connectivity to your frontend and your backend, but we've lost a lot of that with the move to headless.

There are "headless" CMSs like Storyblok who give you a full frontend visual editor, but... is that still headless? Cool in any way you look at it for sure, but I question if a CMS can call itself "headless" if it literally gives you a preview environment for your website - the "head".

You need to have a strong engineering team

Lots of the pushback I would hear from my clients, and rightfully so, is something like this:

"What happens if you and your team go away? Will we be stuck with a beautiful engineering masterpiece with no one smart enough / cost-effective enough to work on it? If we use WordPress, we can grab a developer off the street that works for cheap."

This always reminded me of Ducatis. Beautiful machines, but RIP if you have to work on one. If you've got a crazy dev environment that has three separate services all combined, sharing code in separate NPM packages, the onboarding for a developer coming into the project is gonna be steep.

Let's take a second to remember that we're doing this to ourselves. All for the sake of a website in most all cases.

It's time for things to get simpler

Over the past quarter, the Payload team has been thinking about all of the above quite a bit. My thesis on the topic is that the whole web dev ecosystem perpetually swings back and forth from complexity to simplicity. Someone comes up with new tech, and it's complex, but worth it. And then people get tired of the complexity and want something simpler. And then someone makes it simpler. Repeat for the rest of our lives.

I think right now we are swinging back toward simplicity. Case in point: I'm personally tired of manually solving for headless CMS "rough edges" if we're just building a simple website.

Many of you already know that one of Payload's core pillars is simplicity. Is there a way that we can solve some of the above complexity issues with Payload?

Where we're going at Payload

We envision a way to make Payload embeddable in just about any environment - be it an Express server, serverless functions, or edge functions. We want it to be portable so that you can install it within your website itself, no matter if your website is running Svelte, Next, Remix, Vue, Gatsby, whatever. If your environment can run Node, it should be able to run Payload.

We want to promote simplicity once again. Building a website? Make it easy on yourselves, not harder. Install Payload in the same repo as your frontend. Share types, simplify dev ops, take some of the rocket science away.

To realize this vision, the Payload team and I have dedicated much of the last few months to simplifying, streamlining, and modularizing our codebase so that it can be run directly in as many different types of projects as possible.

The first steps

We've simplified our code so that the Payload Local API is as barebones as possible. No HTTP overhead, no GraphQL bloat, just Node methods that are strongly typed and hit your database directly. It's almost like an ORM but it gives you an admin panel as well. This allows you to bring Payload into any project, even if it's not Express-based.

Yesterday, we announced that you can now deploy Payload serverlessly, within an existing NextJS app, on Vercel.

This is a first step, but it's a monumental shift from the "headless CMS" paradigm. You can now use Payload directly in Next server components to get strongly typed data, hitting your database directly, without any HTTP overhead. It's kinda like we're giving an optional "head" back to the headless CMS.

// Here's a NextJS server component
const Page = async ({ params: { slug } }) => {
  const payload = await getPayload();

  // Zero HTTP overhead - this doesn't hit a REST API,
  // it goes straight to your database
  const pages = await payload.find({
    collection: "pages",
    where: {
      slug: {
        equals: slug,
      },
    },
  });

  const page = pages.docs[0];

  if (!page) return notFound();

  return (
    <React.Fragment>
      <AdminBar adminBarProps={{ collection: "pages", id: page.id }} />
      <Hero {...page.hero} />
      <Blocks blocks={page.layout} />
    </React.Fragment>
  );
};

Being able to deploy Payload within your app, wherever your app is running, will solve for basically all of the above complexities that I mentioned. It'll be faster, simpler, and require less black magic.

Future goals

We're going to keep chasing this idea of simplicity. In the future, you'll be able to do this for any type of app. It doesn't need to be React or even Next. Our goal is to let you install Payload wherever you need it, and simplify your stack.

We're also already working on being able to deploy Payload within edge functions. We've got a bit of work to go on that, but it'll happen.

Serverless not your thing? Can't deal with cold starts?

Then you can combine Payload and NextJS into a single server-based repo as well. You can even deploy it on Payload Cloud.

There are edge cases, of course

I've been going on about how most people always deploy CMS in a website, blah blah. But that doesn't mean that Payload is going to lose track of our API-based DNA. There are lots of cases where deploying a CMS separately will still be the move, and any CMS should have omnichannel capabilities nowadays via REST / GraphQL / etc. no matter where or how it's deployed. I just think that is a given at this point.

Conclusion

It's time the headless CMS paradigm moved back toward simplicity. With our launch of the next-payload package, we're taking an important first step towards restoring simplicity.

I'm not sure of what the word should be that describes what I'm trying to get to but I hate coining new words for obvious concepts.

Some other CMS use the word "composable", but I feel like that's obvious if you're working with an API-based CMS. But what is true composability? Microservice / SaaS hell? Yes, let me go sign up for a SaaS CMS, then go sign up for a forms provider, then go sign up for Hubspot, then go sign up for a mail service, and then deploy my site on Next / Vercel. All just for a website? I don't care how big the site is, that doesn't sit right with me. Actually that makes me kinda want to puke.

Let's fix the complexities inherent with headless CMS and see how far we can take this idea of simplicity. Let's make our jobs easier. That'll allow us to deliver better products.