How to build filtering into your REST API

You’ve got the basics of your REST API working and you’ve created routes that respond with records from your database. Now you want your API to be able to handle more complex requests that support sorting, filtering and pagination. What’s the best way to go about this?

REST is not a standard, so there’s no specification or single source of documentation that you can refer to when trying to solve this type of problem. This can make it difficult to know what you should do. Wouldn’t it be great if there was something that could provide you with some inspiration and help guide you through the decisions you need to make to implement sorting, filtering and pagination functionality in your API?

Fortunately there’s help at hand, as Tania Rascia has written a fantastic article on this very subject: REST API: Sorting, Filtering, and Pagination.

How this article can help you

Recently, I made a backend API for some list endpoints, and implemented filtering, sorting, and pagination. There’s not really a set standard for creating these types of endpoints, and almost every one I’ve come across is different in some way. I made a few notes on what made sense to me, so this resource could be helpful for someone who is working on designing an API.

Tania’s comprehensive article is a great starting point for helping you understand how you could build filtering into your REST API. It covers the basics of sorting and pagination, as well as the different types of filtering you might want to do with strings, numbers and dates.

The article doesn’t just provide an example URL for each use case, but it also gives an example of how each URL could map through to an SQL database query in your application. I think this is a really practical way of thinking about filtering functionality.

Am I doing this right?!

There is no one "correct" way of building sorting, filtering and pagination into your REST API, but knowing the problems that you might run into and how you could solve them before you encounter them will give you a great head start.

From my own experience I can say that it’s easy to end up in a situation where you haven’t planned out these features for your API, and you just start tacking things on as new requirements emerge. Trying to combine everything later on into a solution that meets all of your needs is a recipe for disaster: it’s time to plan ahead.

Leave your keyboard and pick up your notepad

Before you even think about writing some code in your editor, grab a notepad and write down the requirements that you have for sorting, filtering and pagination in your API. Here are some questions to ask yourself:

• What fields do you want to be able to filter by and how e.g. exact match, partial match, greater than or less than (for numbers)?
• What fields, if any, do you want to be able to sort by?
• Is pagination necessary? If you’re sending always sending back a small number of records you might not need to build in support for pagination initially, if ever at all.

I find this a great way of getting my head away from the code and visualising the problems that I’m trying to solve.

Things to watch out for

Validating and parsing filters from query strings

When you’re building filtering functionality into an API it’s important to consider how you will handle the data you receive in the request query string on the server side in your application.

Often, searches are expected to be partial, so that when I look for "Tan" it will show me "Tania" and "Tanner". The solution I liked was using like:Tan as value as opposed to modifying the parameter (such as first_name[like]=Tan).

(Source: https://www.taniarascia.com/rest-api-sorting-filtering-pagination/#string-partial)

Mixing a filter operator (like) with a filter value (Tan) can make your filters more difficult to validate and parse in your application as you will need to write code to separate them. This might not be an issue if you only have one or two filters that will require an operator, but it’s worth considering if you’re expecting to create a lot of them. The first_name[like]=Tan approach might not look as nice, however it might be easier for you to validate and parse.

Complex filtering requirements

When looking at how you could build in filtering for a range between two number values, the article gives this query string example:

GET /users?age=gt:12[and]lt:20

If you need a range between two number values, using [and] in between them could be one option. This one could get complicated, depending on if you want to allow both greater than and greater than or equal, or other options.

(Source: https://www.taniarascia.com/rest-api-sorting-filtering-pagination/#number-range)

As Tania says, this could get complicated, and I completely agree. At this point you’re effectively beginning to write your own custom query language in a URL query string – the code you will need to write in your application to validate and parse these types of filters could become quite complex and be time consuming to test.

A REST API is not the right approach for everything: if you do need to build complex filtering into your API, I would highly recommend that you consider using GraphQL instead. GraphQL is a query language for APIs and it is very well suited to building complex filtering functionality. I recommend taking a look at the How to GraphQL website, which will introduce you to the basics of GraphQL so you can decide whether it’s a good fit for your project.

Next steps

Access the query string parameters in your application

In both Express and Fastify the request query string is automatically parsed into an object which is stored in the query property of the request object i.e. request.query (or req.query by the conventions in the Express documentation). This means that you can access the parsed query string object in any middleware or handler functions, as they are passed the request object as their first argument.

Here’s an example of accessing the query object in an Express route handler function:

/**
 * Example request URL: /users?first_name=Bobinsky&last_name=King
 */
app.get("/users", (request, response) => {
    const filters = request.query;

    /**
     * filters = {
     * first_name: "Bobinsky",
     * last_name: "King"
     * }
     *
     * Sweet, now we can use these filters in a database query!
     */
});

Consider using a tool to help you design your API

Using a pen and paper is great way to start planning the design of your API, but if it’s a large and complex API or if you’re working with other developers, it might be worth trying out a tool like Insomnia Designer or Swagger Editor.

These tools can help you produce an OpenAPI description in JSON or YAML format. The OpenAPI Specification is becoming popular as a way to document APIs and has been adopted by many companies, such as GitHub. OpenAPI description files can be read by a growing number of tools. These tools can help you do things like automate the testing of your API, or generate developer documentation in an HTML format.