Improve Your Web API Performance With Pagination

Hello guys, today continuing with the ASP NET Core web API series we will review an exciting topic and that is pagination I think that non of us are oblivious to pagination as in almost any website there is an implementation of this functionality say for example on Google Search when you search for anything you get some results back and if you hit the bottom then you have a control where you select the next page to get more results in the case that you did not found what you were looking for and that is exactly what we want to implement in our web API today.

What is pagination ❓

Pagination is a technique used in web APIs to break up a large set of data into smaller, more manageable pieces. This is achieved by limiting the number of results that we send back on each API response and providing a way for the client to request the next set of results.

In a typical implementation of pagination, the client sends a request to the server providing a certain number of results to be returned to per page and also the current page number.

Why should we use pagination in our web API ⁉️

Pagination is commonly used in web APIs to improve performance and reduce the amount of data that must be transferred between the server and the client, especially when dealing with large datasets. Also improves user experience as small data chunks are more digestible than large datasets and help to prevent bad user experience on slow internet connections as it will not load the entire dataset only what is requested.

Clone the GitHub repo 🐙

Before starting to code remember that you can follow along with the tutorial, you just need to go to the GitHub repo and clone the AsynchronousEndpoints branch to get the latest changes up to this point.

GitHub - Osempu/BlogAPI at AsynchronousEndpoints

Adding query parameters support to the Get endpoint 🪂

Currently, we call the Get endpoint and we get all the existing posts from the database so this is a great example of where we should implement pagination as the more our database grows the least likely we would like to return a large dataset.

The way the client will request the number of results and the page number is through query strings and ASP NET Core have a pretty simple way to deal with them using the [FromQuery] attribute int endpoint. This way we tell ASP NET Core to check the query string and then it will attempt to read the values from the query string and parse it into the parameter we specified.

[HttpGet]
public async Task<IActionResult> GetPost([FromQuery] int pageSize = 50, [FromQuery] int currentPage = 1)
{
        var posts = await repository.GetPostAsync(pageSize, currentPage);

        //code ommited for brevity
}

We passed the pageSize and the currentPage parameters both decorated with the

[FromQuery] attribute so ASP NET Core attempts to read its values from the query string as we said before and both parameters have default values in case the user does not provide any and then we pass these parameters to the GetPostAsync method. Now you should be getting and error as this method does not accept any parameter and for that, we need to update the IPostRepository interface and PostRepository class.

Updating the IPostRepository interface and PostRepositoryClass 📬

Now we need to update our interface and repo class respectively to support the pagination parameters

For the IPostRepository interface, we only need to define two parameters pageSize and pageNumber.

public interface IPostRepository 
{
    Task<IEnumerable<Post>> GetPostAsync(int pageSize, int pageNumbe);
        //Code omitted for brevity
}

And for the post repository, we will implement the interface signature supporting the same parameters. Now all that is left is to actually implement the pagination functionality.

public async Task<IEnumerable<Post>> GetPostAsync(int pageSize, int pageNumber)
{
        //Code omitted for brevity
}

Implementing pagination in PostRepository class 🏤

public async Task<IEnumerable<Post>> GetPostAsync(int pageSize, int pageNumber)
{
    var allPosts = context.Posts.AsQueryable();

    var pagedPosts = await allPosts

                     .Skip((pageNumber - 1) * pageSize)                             
                     .Take(pageSize)                                
                     .ToListAsync();

    return pagedPosts;
}

You may get tempted to do this in the Post controller after getting all the posts but the idea behind pagination is to limit the number of resources the database return and this needs to be done in the repository class because that’s where we are calling the database.

First, we make a call to all the posts but as a queryable type, this means that we are asking for all the posts existing in the database but the LINQ query has not been executed yet, this helps us to save resources as we are still building the query, then we make another query to all the posts. We call the skip method to skip the previous pages in case the client is asking to see for example page 3 or 8, then we take a specific amount of posts to show to the user, this is specified by the pageSize parameter and finally, we call the ToListAsync() method to execute the query and return a list of posts.

Now you can run the app and start testing the pagination, if you are using Swagger then you will see two new parameters which are the query params that you can enter for pageSize and pageNumber, and when you execute the call the request URL should look similar to this

http://localhost:5049/api/post?pageSize=10&currentPage=1

so you should be getting 10 posts from the first page.

The problem with this approach 💥

You must be thinking that this is all and we can move on right? well, spoiler alert, we can’t. The thing is that this solution does not scale well as we need to provide some default values for the query parameters in the case that the user does not provide them, although we can do that on the parameters imagine later we want to filter in our search that’s another parameter to pass, no think about sorting, we would end up with our endpoint having a lot of parameters to set and we know that’s not something good neither scalable that’s why we need to refactor our solution to make it more scalable for future changes.

Let’s create a new model class called QueryParameters, this class will hold all the parameters we need for pagination and for future changes in our API.

public class QueryParameters
{
  const int maxPageSize = 50;
  public int PageNumber { get; set; } = 1;
  private int pageSize = 10;
  public int PageSize
  {
      get { return pageSize; }
      set { pageSize = (value > maxPageSize) ? maxPageSize : value; }
  }
}

Now with the aid of this class, we will hold all the parameters we need for the pagination and future functionality to work. We are defining the max size the page can have, we set the page to a default value of 1 in the case the user does not provide the page to display and lastly we set the page size to a default value of 10 or whatever the user defines.

Implementing the new approach 🆕

[HttpGet]
public async Task<IActionResult> GetPost([FromQuery] QueryParameters parameters)
{
    var posts = await repository.GetPostAsync(parameters.PageSize, parameters.PageNumber);

    var postsDto = mapper.Map<IEnumerable<PostResponseDTO>>(posts);

    logger.LogDebug($"Get method called, got {postsDto.Count()} results");
    return Ok(postsDto);
}

If you run the app it will be running as before but now we have an improved version for when we add filtering and other parameters to enhance our endpoint.

Conclusion 🌇

Pagination is a powerful technique used to improve the user experience while saving server resources as we do not return all the data available, avoiding bottlenecks and performance problems over slow connections. As with all programming techniques, paging must be used wisely as in not every project it will make sense to apply it, for example, in an analytics service, in that context it makes sense to return hundreds if not thousands of records as the more data you have at hand the more reliable and precise this service becomes.