How to Cache Database Results with Redis in Node.js

The speed with which an application responds to user queries is an important performance metric that the application can be rated by. A major factor that has been proven to improve the response speed of an application is caching. Caching helps to enhance the performance of distributed systems.

When dealing with data that does not change too often, it is recommended that the results of the request be cached; this helps to reduce the number of API calls made to the database.

There are several tools for caching; however, in this article, we will be using Redis, an in-memory database that stores data in the server memory.

This article will be focused on caching and how to perform caching on database results with Redis in Node.

Why should you perform caching?
There are several reasons why developers should employ caching in their applications, amongst which are

• To reduce latency, thus reducing response time.
• To save or reduce costs. This is possible because caching reduces the load on the backend database(s), especially if the database charges per throughput.
• Caching can greatly improve the performance of your application and, in the long run, lead to user satisfaction.
• By implementing caching, developers can anticipate the performance of their applications, particularly during periods of high network requests.
• When data is stored in the cache, users can still access this data even when there is a network outage.

Prerequisite

To follow through with this tutorial, a basic understanding of Node.js and how to build REST APIs' is required. You should have the following installed on your computer;

• Node.js.

• Redis: You can either have a local instance of Redis running on your local machine or use the Redis cloud service. For the purpose of this tutorial, we will be using a local instance of Redis.
  For Windows users, you can follow through on how to download and install Redis on your local computer. For other operating systems, check out install Redis.

• RedisInsight: is a Redis Graphical User Interface (GUI) tool that helps with visualizing data stored in the Redis database. It also helps in monitoring real-time changes in the database. Download and install RedisInsight.

Getting Started

Step 1: Project Setup

We will start with installing all the dependencies needed for this project and start an express server
i. create the directory for the project using the mkdir command

$ mkdir node_cache

ii. Navigate into the project directory using the cd command

$ cd node_cache

iii. Initialize your project and create a package.json file in the root folder of the project directory using the npm init command

$ npm init -y

The -y flag accepts all default suggestions automatically.
iv. Install the following dependencies from:

• express,
• ioredis: a Redis client for Node.js which can be used to connect your application to the Redis server
• fetch If you are using the latest version of Node (> version 17.5), then you don't need to install the fetch API, as it comes inbuilt with the latest Node version. You can also use the axios API instead of the node fetch API, but to use axios you will need to install the package from the npm store. For the purpose of this project, we will be using the inbuilt node fetch API. To install the required packages all at once

$ npm i express ioredis

v. Now that the dependencies have all been installed, create an app.js file in the root directory of your project.

touch app.js

For Windows OS users, you can achieve the same method of file creation through Windows Powershell, using the command new-item. Thus to create an app.js file in windows

new-item app.js

You can also create the app.js file in your preferred text editor.
vi. Open the file in your preferred text editor. I will be using VS Code as my preferred text editor. In the app.js file, we are going to create a simple Express server

const express = require('express');

const app = express();
const PORT = 3300;

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});

Step 2: Retrieve data from an API without caching

We will make an API call to a third-party service, which, in turn, fetches data from a database, in order to retrieve the data we need. We will be making the API call from the Express server we built in step 1 above. Just so you know, the API call will be made without caching.
To begin, still, inside your app.js file, we will be making an API call to the REST COUNTRIES API to retrieve some data.

In your app.js, create a GET route. The GET method accepts two parameters, which are:

• route parameter: It defines what should happen when a request is made to that specific route, in this case, '/country/:countryName'
• callback function app.get('/country/:countryName', async (req, res) => {})

Note: The usage of the async keyword in the callback function of the GET route above is necessary due to the asynchronous nature of the fetch API. As the fetch function is asynchronous, we use async to declare the callback function as asynchronous. This allows us to use the await keyword within the function to wait for the result of the fetch API, ensuring that the data is fully retrieved before proceeding with further logic.

In the body of the callback function for our GET route, we will include a try{} catch{} block to effectively manage and handle any runtime errors that may occur

app.get('/country/:countryName', async (req, res) => {  
  try {

  } catch (error) {

  }
});

In the try{} catch{} block, we will call the node fetch API, which we will use to make an API call to REST COUNTRIES.

try {
   const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`)
  } catch (error) {
    console.log(error)
  }

When making a fetch request, it's necessary to invoke the .json() method on the response object to obtain the returned JSON data. Remember to include the await keyword, as you might get unparsed JSON data without it.

let result = await data.json();
    return res.status(200).send(result);

Our final code output will look like this:

const express = require('express');

const app = express();
const PORT = 3300;

app.get('/country/:countryName', async (req, res) => {
  const {countryName} = req.params;

  try {
    const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`);

let result = await data.json();
    return res.status(200).send(result);
  } catch (error) {
    console.log(error);
  }
});

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});

Now we can start our server by running $ node app.js and then open any API test tool of your choice. I will be using Postman for this. You can also download the Postman VS code extension.
The REST COUNTRIES API accepts many country's name initials, but we will use only the eest as a route parameter on the endpoint we will be testing throughout this tutorial.
In your postman, make a request to the country endpoint.
http://localhost:3300/country/eest

In the image displayed above, you'll notice that there's a highlighted section indicating that it takes approximately 1281 milliseconds for the request to be completed. This delay can be better optimized using caching, considering that the data being requested doesn't change frequently. Additionally, as your user base expands, the number of requests to this particular endpoint will increase, potentially leading to excessive strain on the database. But this issue can be resolved with caching

Step 3: Implementing Caching using Redis

Here, we will begin by starting our Redis server. In your terminal (wsl for windows users or redis-cli), type redis-server; this script will start the local redis server.
PS: You must have Redis installed in your computer.
To confirm if your local redis instance is up and running, open another terminal window and type redis-cli ping, and you will get back a response PONG.

Now that our redis server is up, we are going to connect our Express application to the local Redis server.

Firstly, inside your app.js file, import the ioredis module that was installed as a depency in step 1
const Redis = require('ioredis');

At the top of your GET route in the app.js file, create a function that connects to the Redis server.

let client;
(() => {  
    client = new Redis({
      port: 6379,
      host: '127.0.0.1',
    });

    client.on('connect', () => {
      console.log('Connected to Redis yo!');
    });

    client.on('error', (error) => {
      console.error(error);
    });
})();

In the above code, we declared a variable and then created an anonymous IIFE (Immediately Invoked Function Expression) function to connect our Express application to redis using ioredis. The variable client is declared outside of the IIFE function, so that it can be globally accessible outside of the function.

Inside the function, we called the new Redis() class to create a Redis instance and passed the connection parameters into the object. This method will come in handy if you are also trying to connect to the Redis cloud service. The new Redis instance is assigned to the client variable.

Also, call on the Node.js on() method which registers events on whichever object it is called on (in this case, the ioredis object). The on() method accepts two arguments; the name of the event and a callback function.

The client.on('connect', () => {}) function checks if the new Redis instance has successfully connected.

The client.on('error', () => {}) function checks if there is an error while trying to connect to the Redis server.

Next, we are going to implement cache hit logic. A cache hit is when data is successfully served from the cache memory. This code will look like:

let cachedData = await client.get(countryName);
    if (cachedData ) {
      return res.status(200).send(JSON.parse(cachedData));
    }

The ioredis get() method is used to get data from the redis server. We pass the countryName as an argument into the get() method.

The if(cachedData){} conditional statement checks if the cachedData variable has data, if it returns true, this is known as a cache hit. The cacheData variable is then converted to a JavaScript object using the JSON.parse() method, and the result is returned to the user.

After implementing the cache hit logic, next we will implement the cache miss logic. This is what happens when the cache hit fails, meaning there is no data with that key name in the redis server. The code logic for the cache miss would look like:

client.set(countryName, JSON.stringify(result));

The ioredis set() method plays a crucial role in storing data on our Redis cache server. This method requires two arguments: the key and data (in this case, countryName and result).

To elaborate further, the first argument, countryName, serves as the key under which the data is stored on the Redis server. It's essential to recall that countryName is a dynamic value, initially parsed in our endpoint '/country/:countryName'. Therefore, when users access the endpoint '/country/eest', the countryName dynamically becomes and is stored as eest on the Redis server.

The second argument holds the result retrieved either directly from the database or through a third-party API call. To ensure compatibility, we employ the JSON.stringify() method on the result. This step converts the obtained data into a JSON string. Also, recall that in our cache hit logic, when we retrieve data using the get() method, we subsequently use JSON.parse() on the cachedData. This action converts the data back into a JavaScript object, ensuring seamless integration into our application's logic.

Your updated code should look like:

const express = require('express');
const Redis = require('ioredis');

const app = express();
const PORT = 3300;

// Connect to the redis server
let client;
(() => {
  client = new Redis({
    port: 6379,
    host: '127.0.0.1',
  });

  client.on('connect', () => {
    console.log('Connected to Redis yo!!');
  });

  client.on('error', (error) => {
    console.error(error);
  });
})();


app.get('/country/:countryName', async (req, res) => {
  const {countryName} = req.params;

  try {
    //cache hit
    let cachedData = await client.get(countryName);

    if (cachedData) {
      return res.status(200).send(JSON.parse(cachedData));
    }

const data = await fetch(`https://restcountries.com/v3.1/name/${countryName}`);

    let result = await data.json();

    //cache miss
    client.set(countryName, JSON.stringify(result));

    return res.status(200).send(result);
  } catch (error) {
    console.log(error);
  }
});

app.listen(PORT, () => {
  console.log(`App listening on port ${PORT}`);
});

Save your code and navigate to Postman to test the endpoint

From the above image, we can see that the response time was high (1789ms). When we requested the data from the endpoint, the API call was made directly to the Rest Countries API, the cachedData variable returns null because there is no data with the key countryName stored in our Redis cache server yet. Thus there is a cache miss, then the rest codes after the if(cachedData){} condition would run, and eventually, the data is saved to the Redis cache server.

If you make the same request to the same API endpoint as shown below, you will see that the response time would significantly reduce, as shown below:

From the image above, we can see that it took 311ms to return data to the user, this is because, the data was already saved in the Redis server, thus faster retrieval.

Conclusion

The significance of caching in software development cannot be overstated. Its impact extends beyond just enhancing our application; it also plays a crucial role in cost savings, particularly when our service/application relies on data from paid API services or databases.

The complete code can be found Here