Build a full API with Next.js

After years and years fighting with and against JavaScript build stacks, I eventually gave a try to Next.js and fell in love with it for two simple reasons: it's barely opinionated, and it packages a simple and unique build configuration shared across back-end and front-end. But as it's not Express underneath the API routes, we have to find some workarounds to be able to build a real all-in-one application.

To call it a decent API, we need quite more than just routes handling. Standalone entry points are required for executing scripts and workers; chaining middlewares really helps keeping route security layers declaration succinct; and as most middlewares and router-dependent packages have been written for Express, we also need a way to integrate them seamlessly.

One solution would be using a custom Express server, but we'd go against the framework and lose its main advantage: Automatic Static Optimization. So let's try to use the built-in server, and address the issues one by one to make it all run smooth.

Issue 1: chaining middlewares

This one is a no-brainer. Just use next-connect! It emulates the next() behavior of Express and gives us back our well-appreciated .use(), .get(), .post(), .all() etc. methods that takes away the need for the verbose in-route method checking (if (req.method === 'POST') { ... }) that Next.js suggests on their documentation.

import nc from 'next-connect';

const handler = nc()
  .use(someMiddleware())
  .get((req, res) => {
    res.send('Hello world');
  })
  .post((req, res) => {
    res.json({ hello: 'world' });
  });

export default handler;

Also, a very convenient feature is passing other next-connect instances to the .use() method, and therefore predefine reusable handler middlewares:

// /path/to/handlers.js
import nc from 'next-connect';
import { acl, apiLimiter, bearerAuth } from '/path/to/middlewares';

export const baseHandler = () => nc({
  // 404 error handler
  onNoMatch: (req, res) => res.status(404).send({
    message: `API route not found: ${req.url}`,
  }),

  // 500 error handler
  onError: (err, req, res) => res.status(500).send({
    message: `Unexpected error.`,
    error: err.toString(),
  }),
});

export const secureHandler = baseHandler()
  .use(apiLimiter)
  .use(bearerAuth)
  .use(acl);


// /pages/api/index.js
import nc from 'next-connect';
import { secureHandler } from '/path/to/handlers';
const handler = nc()
  .use(secureHandler) // benefits from all above middlewares
  .get((req, res) => {
    res.send('Hello world');
  });
export default handler;

Issue 2: testing routes

Within the test environment, Next.js server is not running, forcing us to find a way to emulate both the request and its resolution. Supertest pairs really well with Express, but needs to run the server in order to pass the request to the handler through all its layers. That being said, it doesn't need to be Express.
So without adding any new dependency, we create a bare HTTP server with the native node http lib, and manually apply the built-in resolver of Next.js, nicely packaged as a utility function, just like this:

import { createServer } from 'http';
import { apiResolver } from 'next/dist/next-server/server/api-utils';
import request from 'supertest';

export const testClient = (handler) => request(httpCreateServer(
  async (req, res) => {
    return apiResolver(req, res, undefined, handler);
  },
));

In our test files, the only thing we need then is passing the handler to our client, with Supertest running as usual:

import { testClient } from '/path/to/testClient';
import handler from '/pages/api/index.js';

describe('/api', () => {
  it('should deny access when not authenticated', async (done) => {
    const request = testClient(handler);
    const res = await request.get('/api');
    expect(res.status).toBe(401);
    expect(res.body.ok).toBeFalsy();
    done();
  });
});

That way we don't have anything to setup repeatedly for each route test. Pretty elegant.

Issue 3: custom entry points

Entry points are scripts that are meant to be run manually - usually background processes like a queue worker, or migration scripts. If set as standalone node processes, they won't inherit from the 'import' syntax built-in inside Next.js, neither the path aliases you may have setup. So basically, you'd have to manually rebuild the build stack of Next.js, polluting your package.json with babel dependencies, and keep it up to date with Next.js releases. We don't want that.

To make it clean, we have to make these pipe through Next.js build. Adding custom entry points is not documented, though it seems to work with that solution, configuring next.config.js:

const path = require('path');

module.exports = {
  webpack(config, { isServer }) {
    if (isServer) {
      return {
        ...config,
        entry() {
          return config.entry().then((entry) => ({
            ...entry,
            // your custom entry points
            worker: path.resolve(process.cwd(), 'src/worker.js'),
            run: path.resolve(process.cwd(), 'src/run.js'),
          }));
        }
      };
    }
    return config;
  },
};

Sadly the only thing it does is compiling these new JavaScript files through the internal webpack process and outputs them inside the build directory, as is. Since they're not tied to the server, all the features of Next.js are missing, including the only important one for this case: environment variables.

Next.js relies on dotenv, so it's already set as a dependency that we could reuse. Yet calling dotenv at the top of these entry points, for some reasons, won't propagate the environment variables to the imported modules:

// /.env
FOO='bar';


// /src/worker.js
import dotenv from 'dotenv';
dotenv.config();

import '/path/to/module';

console.log(process.env.FOO); // outputs 'bar';


// /src/path/to/module.js
console.log(process.env.FOO); // outputs 'undefined';

That is very annoying. Thankfully, it can be quickly solved by dotenv-cli, which actually resolves .env files the same way than Next.js. We only need to prefix our script commands in package.json:

"worker": "dotenv -c -- node .next/server/worker.js",

Note that it calls the script from the build folder. You need either to have next dev running, or previously have run next build. It's a small price to pay in regard of the benefits of keeping them within the Next.js build stack.

Issue 4: Express-based packages

Next-connect already makes some Express packages compatible out of the box, like express-validator that I'm used to when it comes to checking request parameters. That's because they simply are middleware functions.

Some of these functions rely on Express-specific properties, like express-acl. Usually they throw an exception when hitting that missing property, and digging a little bit the error and the package source will help you find it and fix it with a handler wrapper:

import acl from 'express-acl';

acl.config({
  baseUrl: '/api',
  filename: 'acl.json',
  path: '/path/to/config/folder',
  denyCallback: (res) => res.status(403).json({
    ok: false,
    message: 'You are not authorized to access this resource',
  }),
});

export const aclMiddleware = (req, res, next) => {
  req.originalUrl = req.url; // Express-specific property required by express-acl
  return acl.authorize(req, res, next);
};

So the biggest challenge happens when the package deeply depends on Express because it creates router or app definitions. That's the case of monitoring interfaces like bull-board. When we can't find a standalone alternative, then our only chance is to find a way to emulate the whole Express application. Here's the hack:

import Queue from 'bull';
import { setQueues, BullAdapter, router } from 'bull-board';
import nc from 'next-connect';

setQueues([
  new BullAdapter(new Queue('main')),
]);

// tell Express app to prefix all paths
router.use('/api/monitoring', router._router);

// Forward Next.js request to Express app
const handler = nc();
handler.use((req, res, next) => {
  // manually execute Express route
  return router._router.handle(req, res, next);
});

export default handler;

A few things to note here:

• This file should be located inside /pages/api because Next.js only recognize server-side routes under that folder.
• For Express to handle all sub-routes declared by the package, we have to create a catch-all on Next.js route. That can be done naming our route file /pages/api/monitoring/[[...path]].js as specified in their docs (replace "monitoring" with whichever name you'd prefer).
• In this specific case, bull-board exposes an entire Express instance under the confusing name router. That's why we're calling router._router.handle() to manually execute the route handler. If by reading the source you find out it's a express.Router instance, call instead router.handle() directly.
• We also need to tell Express that the base URL of its entire app is the route we're calling it from. Let's just define it with app.use('/base/url', router) as we would normally do. Just keep in mind the confusion between express and express.Router instances.
• Finally, Express handles the response part as we're passing it the full Response object. No need for us to send headers on its behalf.

The reasons why I don't use this trick to forward the whole API to an emulated Express app is that I don't know how it'll affect performances, and most importantly, I'd rather respect Next.js natural patterns not to disorient other developers.

Not so bad, isn't it? We end up having a full-featured server with footprint-limited patches over the blind spots. I still wish Next.js could provide all these features in its core, but I'm happy we didn't denature it much either with these workarounds. Given the current state of JavaScript, Next.js may very well be the ultimate full-stack framework.

PS: I didn't go over setting up sessions and user authentication because with these issues now solved, you can virtually make everything work as usual. Though, I'd recommend looking into next-session or NextAuth.js.