Rest API : After CRUD

Hello and welcome to my first "technical" article since a very long time. Last one was on a time when Adobe Flash and Facebook Applications were still a thing. (and maybe you’re too young to know what I’m talking about 😀)

Context

I’ve been developing a REST API recently and I knew quite exactly what to do but I still keep on reading articles about API development to see if there’s new techniques that I don’t know about.

A lot of theses articles are really good to start building a REST API, like these ones for example :

https://dev.to/beznet/build-a-rest-api-with-node-express-mongodb-4ho4

https://dev.to/nditah/how-to-build-a-rest-api-with-node-prisma-and-postgresql-429a

But is it enough to code a CRUD - well more a GPPD (GET-POST-PATCH-DELETE) but let’s stick with readable acronyms 😀 - to put your api in production ?

Well no, not really and I found myself implementing a lot of annex features that are however pretty essential.

As I work with nodejs/express, I will focus on tools/code for javascript but the underlying concepts are pretty common.

Who are you? Who, who, who, who?

Articles about Authentication are not really rare.
It's main purpose is to prepare the request for authorization (we'll get there) and/or filter the endpoints return values.

An example ?
Imagine you work with several companies that uses the endpoint POST /users to add new users to your services and GET /users to retrieve the users list. You'll want to automatically set a reference to the company on the first endpoint and exclude other companies users on the second.

You could send an api_key to your partners that they would add to the endpoints requests (in the header, the query, etc.) so a simple db request would be enough to know who is interacting with your API.

Here’s a few solutions to get who your API caller is if you need :

External services like Auth0 (https://auth0.com/fr) that also provide authorization services - see below - security protection, etc.

Libs like passportjs (https://www.passportjs.org) where you can customize as you wish the authentication process (JWT, sessions, apiKeys, oAuth...).
You’ll find a lot of examples here : https://dev.to/search?q=passport

DIY with the solution that you prefer.
For example with JWT : https://dev.to/perrydbucs/using-jwts-for-authentication-in-restful-applications-55hc

For my API I wanted to use APIKey in http bearer AND a JWT in HTTPOnly cookie so I choosed passportjs that is common, reliable and has a lot of features.

You don't own me (I'm not just one of your many toys)

Ok, now you know who is calling (the "user") you may have the need to restrict some endpoints : does the user can see the resource, can interact with the resource or not ? That's Authorization.

An example ?

"Umbrella Corporation" calls the endpoint POST /virus with it's API_Key. Alas, this endpoint is only accessible by Admin Users and Umbrella is not one of them. The endpoint should return response with the HTTP error code 403 (https://en.wikipedia.org/wiki/HTTP_403).

Some of the ways to discriminate users are :

RBAC just add a role in the User entity of your base and checks on each request if the current user has the correct role(s) to access the resource. (I know my explanation is simplified)

ACL More granular than RBAC, this time each object has a list of users/roles/groups with the action authorized for each one of them. Thats very powerful but often really painful to manage on small to medium projects.

ABAC Permissions are based on attributes (you choose) of the object to check. creator, status, whatever you want can be used to make rules to verify if the user has permissions.

Here are some tools that can help you handle authorization with nodejs :

• https://github.com/stalniy/casl
• https://github.com/tschaub/authorized
• https://github.com/onury/accesscontrol

My advice on that point : start with the simplest way and change if it's really getting difficult to implements auth rules.

For my project I coded a simple solution inspired by Symfony's Voters :
I created a class for each typeof object (usually models) that needs permissions with the same static method check(). (yeah, use an Interface in any other language 😁)

class UserPermissions {
  /**
   * Checks if {user} can {verb} the {object} (or just {verb} is object is null)
   *
   * @params {User} instance of the user to check permissions
   * @params {string} permission name
   * @params {Object|null} object
   *
   * @returns {Boolean} has permission ?
   */
  static check(user, verb, object = null) {
    if (!user || !(object instanceof User)) return false;
    switch (verb) {
      case "read":
        return UserPermissions.canRead(user, object);
      case "write":
        return UserPermissions.canWrite(user, object);
      default:
        return false;
    }
  }

  // only admins and user him.her.self  can write (edit) a user
  static canWrite(user, object) {
    return user.role === "admin" || String(user.id) === String(object.id);
  }

  // only admins and user him.her.self can read (view) a user
  static canRead(user, object) {
    return user.role === "admin" || String(user.id) === String(object.id);
  }
}

export default UserPermissions;

And there's a kind of factory to get the proper auth class depending on the object.

/**
 * Returns an auth checker class relater to an object type
 */
const getAuthClass = (object) => {
  if (object instanceof User) {
    return UserPermissions;
  }
  /// ... other ones
  return false;
};

export const checkAuthorisation = async (user, verb, object) => {
  const authClass = getAuthClass(object);
  return authClass ? await authClass.check(user, verb, object) : false;
};

export default { checkAuthorisation };

and then a simple usage :

await checkAuthorisation(UmbrellaCorpUser, "read", JillValentineUser);
// returns false obviously

I also created a middleware (a previous middleware gets the object to test and stores it in req.element).

const PermissionMiddleware = {
  //
  can(verb, permissionClass) {
    return (req, res, next) => {
      if (permissionClass.check(req.user, verb, req.element)) {
        return next();
      }

      return next(new Error("Unauthorized")); // use httperrors
    };
  },
};

export default PermissionMiddleware;

//

import Machin from "models/Machin";

router.get(
  "/machin/:id",
  GetElementById(Machin), // the fetcher middleware
  PermissionMiddleware.can("action", MachinPermissions),
  function (req, res, next) {
    res.json({ title: "debug test" });
  }
);

It fits my usage for the moment but maybe using a more common lib is more easy, more tested, more documented.

Can't take my eyes off of you

Here I wanted to write a very long explanation about audit logs but I found this article during the making of that is really explanatory :
https://dev.to/chipd/making-audit-logs-sexy-best-practices-for-audit-logging-with-examples-1pab

I'll try to make it shorter : you, or your clients but principally you, want to know what happens with the modifying enpoints (POST, DELETE, PATCH) of your API : what whas send, when and what was the result.

If there's an error you wand to know why and reproduce. If there was a unexpected change in a DB entry you whan to know who did it and when (and ask this person why... with or without knives involved, that's your problem).

Here's my technique to do that, first I create an Express Middleware that will catch the res.end() and log the request before calling it manually

function apiLoggerMiddleware(req, res, next) {
  const oldWrite = res.write;
  const oldEnd = res.end;

  const chunks = []; // this will store everithing written in the res

  res.write = (...restArgs) => {
    chunks.push(Buffer.from(restArgs[0]));
    oldWrite.apply(res, restArgs);
  };

  res.end = async (...restArgs) => {
    if (restArgs[0]) {
      chunks.push(Buffer.from(restArgs[0]));
    }
    const body = Buffer.concat(chunks).toString("utf8");

    // APILog is an Mongoose Model but you can use whathever you want, Postgres, event Excel, I don't care. :)
    const apilog = new APILog({
      time: new Date(),
      headers: {
        ...req.headers, // beware of headers, sometimes you want to remove some elements
        remoteAddress: req.connection.remoteAddress,
        cookie: undefined, // don't store cookies, you shouldn't care user's JWT
      },
      method: req.method,
      originalUri: req.originalUrl,
      uri: req.url,
      path: req.url.split("?").shift(),
      requestUser: req.user, // specific to passportjs
      requestData: req.body, // here's the user input
      responseData: body, // here's your route response
      responseStatus: res.statusCode, // here's your route response status
    });
    await apilog.save();

    oldEnd.apply(res, restArgs);
  };

  next();
}

export default apiLoggerMiddleware;

Beware of something with this way of logging data : if you're in Europe or work with Europe you've got to be GDPR compliant. I really suggest to get in touch with a Data Protection Officer (DPO) to implement that.

Let's use this middleware with a (very simplified) route now :

const router = Router();

/**
 * GET /bundles
 */
router.post("/milk-shake", apiLoggerMiddleware, async (req, res, next) => {
  try {
    const isBoysToTheYard = await isBetterThanYours(req.user); // don't mind about theses fake functions
    const milkShake = await createMilkShake(req.body, isBoysToTheYard);
    res.json({ milkShake: formatMilkShake(milkShake) });
  } catch (error) {
    next(ErrorsHandler(error)); // I haven't talked about error handling... maybe another day
  }
});

You got to keep me focused, you want it, say so

Ok, your API is ready to rumble, and users will start to use it... but how ? I'm sure you don't want to share your source code and even less spen an hour of visio conference with each user to explain every endpoint.

It seems pretty obvious but yes you'll have to write a document that explains how to use you API in details : that's Documentation.

I won't lie : it takes a looot of time to write down useful documentation. You've got to be as exhaustive as possible, but simple enough to be understable quickly and give examples of course.

I will focus here on two points, documenting the API and documenting the whole usage of your app.

For the API documentation here are some tools :

APIDoc : Just add annotations to your routes and voilà, you can generate a documentation with a simple command-line.

I tried it and what I liked is the use of annotations to describe your API, but I left it because I didn't like the documentation template and didn't found any other one that fited my needs (there's not a lot of them).

You can find it here : https://apidocjs.com/

OpenAPI (formerly swagger) is a specification to describe your API so it can be understood by humans or automated scripts.
You can find it here : https://swagger.io/specification/

I use it on my Express Application with the plugin @wesleytodd/openapi that offers a middleware usable for each route to describe them - it can even validate input data.

When your specification is ready you can use a compatible UI to have a nice and useful presentation of your work. The Swagger UI is well known and very clear but I like the Redoc one. I guess it's a question of taste too. 😁

APIBlueprint I stumbled upon this specification recently because of the Chappe documentation engine (see below) and I didn't try it but it seems interesting.

You can find it here : https://apiblueprint.org/

What's up doc ?

That's nice you've got a gread documentation on your API endpoints but maybe there is more about your application to explain, give a glimpse of the workflows, schemas, and other stuff. There comes the usage for documentations engines. Here's a few of them.

Chappe Created by the company behind the Crisp messaging platform for their own documentation it has the advantage of havint the possibility to integrate a APIBlueprint specification document to generate a nice interface to browse your endpoints. Alas, no OpenAPI plugin seems available at the moment... since the project is open-source maybe in the future ?...

You can find it here : https://github.com/crisp-oss/chappe

Docusaurus from the open source Meta projects, this one helps you generate a static React documentation with plenty of features like i18n, versionning, etc.

You can find it here : https://docusaurus.io/fr/

If you know more tools like theses, or if you have tried some and have a constructive point of view, feel free to share in the comments. :)

This is the end, my friends

Hmmm, not really I'm sur there is plenty of things to do, but this is the end of my article. I hope you enjoyed it.

I just realized that I could have separated it to make a series but I guess I'm lazy and I've got some Kreple'h to finish cooking.

Cheers ! 🤗️