Implementing Redis Sentinel - Ensuring High Availability of Redis for Your Application

#redis #redissentinel #cachingredis #dockerize

Introduce

In today's application development environment, ensuring the availability and fault tolerance of systems is extremely important. In this context, Redis Sentinel emerges as a crucial solution for managing and protecting your Redis systems.

Redis Sentinel is an automated system management component of Redis, designed to monitor, manage, and provide fault tolerance for Redis nodes. By deploying Redis Sentinel, you can confidently build and operate Redis systems that are self-healing and highly fault-tolerant.

In this article, we will explore Redis Sentinel, including its main functions, how to deploy and configure it, and the benefits it brings to managing your Redis system. Let’s delve into how Redis Sentinel can enhance the performance and reliability of your Redis systems!

Understanding Sentinels

Redis Sentinel is a distributed system consisting of multiple Redis instances running in sentinel mode. We call these instances Sentinels.

The group of Sentinels monitors a master Redis and its replicas. If the Sentinels detect that the master Redis has failed, the sentinel processes will look for the replica with the most up-to-date data and will promote that replica to become the new master. This allows clients communicating with the database to reconnect to the new master Redis and continue functioning normally, causing minimal disruption to users.

Deciding That the Master Redis Has Failed
- For the Sentinels to decide that the master Redis has failed, we need enough Sentinels to agree that the server is inaccessible.
- A certain number of Sentinels need to agree that they need to take action, which is referred to as achieving quorum. If the Sentinels do not reach quorum, they cannot decide that the master has failed. The exact number of Sentinels required for quorum can be configured.
Activating Failover
- Once the Sentinels have decided that a master Redis has failed, they must elect and authorize a leader (one Sentinel instance) to perform the failover. A leader can only be chosen if the majority of the Sentinels agree.
- In the final step, the leader will configure the selected replica to become the master by sending the command REPLICA OF NO ONE, and it will configure the other replicas to follow the newly promoted master.

High Availability with Redis Sentinel

Redis Sentinel provides high availability for Redis when not using Redis Cluster.

It also offers other ancillary tasks such as monitoring, notification, and acting as a configuration provider for clients.

Here is a complete list of Sentinel's capabilities at a high level (i.e., the overall picture):

Monitoring: Sentinel continuously checks whether the master and replica instances are functioning as expected.
Notification: Sentinel can notify system administrators or other computer programs via an API if there is an issue with one of the monitored Redis instances.
Automatic Failover: If a master is not functioning as expected, Sentinel can initiate an automatic failover process where a replica is promoted to master, the other replicas are reconfigured to use the new master, and applications using the Redis server are informed of the new address to connect to.
Configuration Provider: Sentinel acts as a configuration source for service discovery for clients: clients connect to Sentinel to request the address of the current master Redis that provides a specific service. If an issue occurs, Sentinel will report the new address.

Sentinel as a Distributed System

Sentinel is designed to run in a configuration with multiple cooperating Sentinel processes. The benefits of having multiple cooperating Sentinel processes are:

Fault detection is performed when multiple Sentinels agree that a specific master is no longer available. This reduces the likelihood of false positives.
Sentinel continues to operate even if not all Sentinel processes are active, making the system robust against failures.

Deploying Redis Sentinel for Applications

In this section, I will demonstrate a NodeJS application using ioredis to set up Redis Sentinel.

Configuring Redis Sentinel on Docker

I will write a Docker Compose file to run Redis Sentinel as shown below. It will include 1 master, 2 slaves, and 3 sentinels.

version: '3'

services:
  redis-master:
    image: bitnami/redis:latest
    ports:
      - '6379:6379'
    environment:
      - REDIS_REPLICATION_MODE=master
      - ALLOW_EMPTY_PASSWORD=yes
      - REDIS_AOF_ENABLED=yes
    volumes:
      - redis-master-db:/bitnami/redis/data
    networks:
      redis-net:
        ipv4_address: 172.50.0.10

  redis-slave1:
    image: bitnami/redis:latest
    ports:
      - '6380:6379'
    environment:
      - REDIS_REPLICATION_MODE=slave
      - REDIS_MASTER_HOST=redis-master
      - REDIS_MASTER_PORT_NUMBER=6379
      - ALLOW_EMPTY_PASSWORD=yes
      - REDIS_AOF_ENABLED=yes
    volumes:
      - redis-slave1-db:/bitnami/redis/data
    networks:
      redis-net:
        ipv4_address: 172.50.0.11
    depends_on:
      - redis-master

  redis-slave2:
    image: bitnami/redis:latest
    ports:
      - '6381:6379'
    environment:
      - REDIS_REPLICATION_MODE=slave
      - REDIS_MASTER_HOST=redis-master
      - REDIS_MASTER_PORT_NUMBER=6379
      - ALLOW_EMPTY_PASSWORD=yes
      - REDIS_AOF_ENABLED=yes
    volumes:
      - redis-slave2-db:/bitnami/redis/data
    networks:
      redis-net:
        ipv4_address: 172.50.0.12
    depends_on:
      - redis-master

  redis-sentinel1:
    image: bitnami/redis-sentinel:latest
    ports:
      - '26379:26379'
    environment:
      - REDIS_MASTER_HOST=redis-master
      - REDIS_MASTER_PORT_NUMBER=6379
      - REDIS_SENTINEL_PORT_NUMBER=26379
    networks:
      redis-net:
        ipv4_address: 172.50.0.13
    depends_on:
      - redis-master
      - redis-slave1
      - redis-slave2

  redis-sentinel2:
    image: bitnami/redis-sentinel:latest
    ports:
      - '26380:26379'
    environment:
      - REDIS_MASTER_HOST=redis-master
      - REDIS_MASTER_PORT_NUMBER=6379
      - REDIS_SENTINEL_PORT_NUMBER=26380
    networks:
      redis-net:
        ipv4_address: 172.50.0.14
    depends_on:
      - redis-master
      - redis-slave1
      - redis-slave2

  redis-sentinel3:
    image: bitnami/redis-sentinel:latest
    ports:
      - '26381:26379'
    environment:
      - REDIS_MASTER_HOST=redis-master
      - REDIS_MASTER_PORT_NUMBER=6379
      - REDIS_SENTINEL_PORT_NUMBER=26381
    networks:
      redis-net:
        ipv4_address: 172.50.0.15
    depends_on:
      - redis-master
      - redis-slave1
      - redis-slave2

networks:
  redis-net:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.50.0.0/24

volumes:
  redis-master-db:
  redis-slave1-db:
  redis-slave2-db:

This is the result after running the compose file:

Check the logs in the containers; if they display as shown below, you have successfully run it:

Log redis-sentinel-demo-redis-master-1:

Log redis-sentinel-demo-redis-sentinel1-1:

Thus, the database setup step is complete.

Connecting the Application to Redis Sentinel

Information on the necessary environment variables:

REDIS_MASTER_HOST_PORT_DOCKER=172.50.0.10:6379
REDIS_SLAVE_HOST_PORT_DOCKER_1=172.50.0.11:6379
REDIS_SLAVE_HOST_PORT_DOCKER_2=172.50.0.12:6379
REDIS_MASTER_MACHINE_HOST=your_machine_ip
REDIS_SLAVE_MACHINE_HOST_1=your_machine_ip
REDIS_SLAVE_MACHINE_HOST_2=your_machine_ip
REDIS_MASTER_MACHINE_PORT=6379
REDIS_SLAVE_MACHINE_PORT_1=6380
REDIS_SLAVE_MACHINE_PORT_2=6381
REDIS_SENTINEL_HOST_PORT_DOCKER_1=172.50.0.13:26379
REDIS_SENTINEL_HOST_PORT_DOCKER_2=172.50.0.14:26379
REDIS_SENTINEL_HOST_PORT_DOCKER_3=172.50.0.15:26379
REDIS_SENTINEL_HOST_1=your_machine_ip
REDIS_SENTINEL_HOST_2=your_machine_ip
REDIS_SENTINEL_HOST_3=your_machine_ip
REDIS_SENTINEL_PORT_1=26379
REDIS_SENTINEL_PORT_2=26380
REDIS_SENTINEL_PORT_3=26381
REDIS_MASTER_NAME=mymaster
REDIS_DB=0

Write a config file to connect to Redis Sentinel:

const dbConfig = {
  redisSentinels: {
    natMap: {
      [`${process.env.REDIS_MASTER_HOST_PORT_DOCKER}`]: {
        // Docker IP and internal port of redis-primary
        host: process.env.REDIS_MASTER_MACHINE_HOST,
        port: parseInt(process.env.REDIS_MASTER_MACHINE_PORT), // Exposed port in docker-compose
      },
      [`${process.env.REDIS_SLAVE_HOST_PORT_DOCKER_1}`]: {
        // Docker IP and internal port of redis-replica 1
        host: process.env.REDIS_SLAVE_MACHINE_HOST_1,
        port: parseInt(process.env.REDIS_SLAVE_MACHINE_PORT_1), // Exposed port in docker-compose
      },
      [`${process.env.REDIS_SLAVE_HOST_PORT_DOCKER_2}`]: {
        // Docker IP and internal port of redis-replica 2
        host: process.env.REDIS_SLAVE_MACHINE_HOST_2,
        port: parseInt(process.env.REDIS_SLAVE_MACHINE_PORT_2), // Exposed port in docker-compose
      },
      // Docker IP of Sentinel 1
      [`${process.env.REDIS_SENTINEL_HOST_PORT_DOCKER_1}`]: {
        host: process.env.REDIS_SENTINEL_HOST_1,
        port: parseInt(process.env.REDIS_SENTINEL_PORT_1), // Exposed port in docker-compose
      },
      // Docker IP of Sentinel 2
      [`${process.env.REDIS_SENTINEL_HOST_PORT_DOCKER_2}`]: {
        host: process.env.REDIS_SENTINEL_HOST_2,
        port: parseInt(process.env.REDIS_SENTINEL_PORT_2), // Exposed port in docker-compose
      },
      // Docker IP of Sentinel 3
      [`${process.env.REDIS_SENTINEL_HOST_PORT_DOCKER_3}`]: {
        host: process.env.REDIS_SENTINEL_HOST_3,
        port: parseInt(process.env.REDIS_SENTINEL_PORT_3), // Exposed port in docker-compose
      },
    },
    sentinels: [
      { host: process.env.REDIS_SENTINEL_HOST_1, port: parseInt(process.env.REDIS_SENTINEL_PORT_1) },
      { host: process.env.REDIS_SENTINEL_HOST_2, port: parseInt(process.env.REDIS_SENTINEL_PORT_2) },
      { host: process.env.REDIS_SENTINEL_HOST_3, port: parseInt(process.env.REDIS_SENTINEL_PORT_3) },
    ],
    name: process.env.REDIS_MASTER_NAME || 'mymaster',
    redisDB: parseInt(process.env.REDIS_DB, 10) || 0,
  },
};

module.exports = dbConfig;

Create a client using ioredis to connect to the Sentinel:

const { Redis } = require('ioredis');
const config = require('../configs');

const redisClient = new Redis(
  {
    sentinels: config.redisSentinels.sentinels,
    name: config.redisSentinels.name,
    showFriendlyErrorStack: true,
    db: config.redisSentinels.redisDB,
  },
  { natMap: config.redisSentinels.natMap },
);

redisClient.on('connect', () => {
  console.info('Connect Redis successfully');
});

redisClient.on('error', (err) => {
  console.error('Redis client error', err);
});

module.exports = { redisClient };

Write a simple ExpressJS server with two routes: /write and /read to test writing and reading data to Redis via the Sentinel connection:

Results

Run the server and check the connection:

Thus, the server is running and has successfully connected to Redis Sentinel.

Check the API for writing data:

Check the API for reading data:

Conclusion

In this article, I introduced Redis Sentinel and the features it offers when applied to our applications. I also provided a detailed guide on how to configure and run a Redis Sentinel setup using Docker. Finally, there was a tutorial on connecting to a demo NodeJS application using ioredis.

I hope this article provides valuable knowledge and helps those of you looking to implement Redis Sentinel for your applications. If you found the article helpful and interesting, please vote for me (this is my first article, so it would be a great motivation for me). I am also open to listening, improving, and offering support if you need it, so feel free to leave comments. I will conclude this article here; remember to vote for me. I look forward to seeing everyone in future articles.

If the article is interesting and useful to you, you can give me a cup of coffee. Thanks for reading.