Redis Cluster is the native sharding implementation available within Redis that allows you to automatically distribute your data across multiple nodes without having to rely on external tools and utilities. At ScaleGrid, we recently added support for Redis Clusters on our platform through our fully managed Redis hosting plans. In this post, we're going to introduce you to the advanced Redis Cluster sharding opportunities, discuss its advantages and limitations, when you should deploy, and how to connect to your Redis Cluster.
Sharding with Redis Cluster
The entire keyspace in Redis Clusters is divided in 16384 slots (called hash slots) and these slots are assigned to multiple Redis nodes. A given key is mapped to one of these slots, and the hash slot for a key is computed as:
HASH_SLOT = CRC16(key) mod 16384
Multi-key operations are supported on Redis Clusters as long as all the keys involved in a single command execution belong to the same hash slot. This can be ensured using the concept of hash tags.
The Redis Cluster Specification is the definitive guide to understanding the internals of the technology, while the Redis Cluster Tutorial provides deployment and administration guidelines.
Advantages
Check out the top advantages of Redis Clusters to see how it can benefit your deployments:
-
High Performance
Redis Cluster promises the same level of performance as standalone Redis deployments. -
High Availability
Redis Cluster supports the standard Redis master-replica configuration to ensure high availability and durability. It also implements a Raft-like consensus approach to ensure availability of the entire cluster. -
Horizontal & Vertical Scalability
Adding or removing new Redis nodes from the Cluster can occur transparently and without any downtime. This makes adding and removing shards, retiring, or scaling individual nodes easy. -
Native Solution
Deploying Redis Clusters requires no external proxies or tools, so there are no new tools you need to learn or worry about. It also provides almost complete compatibility with standalone Redis deployments.
Limitations
Let's also be aware of the limitations to ensure it's right for your deployment:
-
Requires Client Support
Clients need to make changes in order to support Redis Clusters. While Redis Clusters have been around for many years now, there are still clients that don't support it. Refer to the Redis client documentation to ensure the client you use is supported before choosing to deploy Redis Clusters. -
Limited Multi-Key Operation Support
As mentioned in the previous section, multi-key operations are supported only when all the keys in a single operation belong to the same slot. This is something to be careful about when designing your data structures. -
Only Supports One Database
Unlike standalone databases, Redis Clusters support only one database (database 0), and the SELECT command isn't allowed. Since most people don't use multiple databases, this too is not a major limitation.
When Should You Deploy a Redis Cluster?
The Redis Cluster solution can be a good fit for you if you need a sharded Redis solution. Redis Cluster is a native solution that is simple and offers great performance.
Typically, people start looking at sharding their Redis deployments when they've started to saturate a standalone Redis node with writes and want to spread writes out to multiple nodes. Even though Redis is primarily single threaded, I/O typically becomes network or memory bound on a standalone before it can start saturating the CPU. Memory bound can be overcome to an extent by choosing to add more memory to a standalone system, but it starts to become prohibitive in terms of cost, backup, restart, warm up times, etc. beyond a certain point.
On the other hand, if you're looking only to spread your read across multiple nodes, it's much easier to just add read replicas to the standalone.
When compared to other sharding solutions for Redis, shard rebalancing in Redis Clusters is transparent to applications. This makes it very easy to add or remove shards without effecting the application.
Connecting To a Redis Cluster
If you choose to deploy a Redis Cluster with ScaleGrid, you get a fully featured Redis Cluster deployment that's totally compatible with the standard release.
If you're just starting out, sign up for a free 30-day trial on the ScaleGrid console, and check out this documentation on creating your first Redis deployment.
Here's what you need in order to connect to the Redis Cluster:
- List of node names
- Ports
- Authentication string
The Overview tab of your Redis deployment details page has the list of masters of each shard, along with port numbers and authentication information:
Alternatively, the list of all the nodes of the cluster is available on the Machines tab:
As you may have noticed, not a lot of examples are available showing you how to connect to an authentication-enabled Redis Cluster available online, here are a few examples using some of the popular clients.
Connecting With Java
Among the popular Redis Java clients, Jedis and Lettuce support Redis Clusters. We will take up Jedis for our example.
Jedis
Redis Cluster connections are abstracted by the JedisCluster class. The best examples of using this class to connect to Redis Clusters can be found in the Jedis tests, Jedis source code. Unfortunately at the this time, when authentication is specified, the JedisCluster constructor is not very clean. Here's an example which writes 100 keys to the Redis Cluster. Note that since the keys aren't tagged, they will end up in different slots on different nodes:
...
import java.util.HashSet;
import java.util.Set;
import redis.clients.jedis.HostAndPort;
import redis.clients.jedis.JedisCluster;
import redis.clients.jedis.JedisPoolConfig;
...
public class RedisClusterModeTest {
public static final int DEFAULT_TIMEOUT = 5000;
public static final int DEFAULT_REDIRECTIONS = 5;
public static void main(String[] args) {
Set jedisClusterNodes = new HashSet();
jedisClusterNodes.add(new HostAndPort("SG-example-1.servers.scalegrid.io, 6379));
jedisClusterNodes.add(new HostAndPort("SG-example-2.servers.scalegrid.io", 6379));
jedisClusterNodes.add(new HostAndPort("SG-example-3.servers.scalegrid.io", 6379));
jedisClusterNodes.add(new HostAndPort("SG-example-4.servers.scalegrid.io", 6379));
jedisClusterNodes.add(new HostAndPort("SG-example-5.servers.scalegrid.io", 6379));
jedisClusterNodes.add(new HostAndPort("SG-example-6.servers.scalegrid.io", 6379));
JedisCluster jedis = new JedisCluster(jedisClusterNodes, DEFAULT_TIMEOUT, DEFAULT_TIMEOUT, DEFAULT_REDIRECTIONS, <auth>, new JedisPoolConfig());
for (int i = 0; i < 100; i++) {
jedis.set("key" + i, "value" + i);
}
jedis.close();
}
}
The arguments to the constructor are documented in the Jedis API docs. We recommend that you specify all the nodes of the cluster during cluster creation with Jedis.
Connecting With Ruby
The most popular Redis client in Ruby is redis-rb. It also supports Redis Clusters so we'll use it in our example.
Redis-rb
redis-rb versions 4.1.0 and above have support for Redis Clusters. The 'cluster' option needs to be specified during connection initialization, and you can refer to this documentation for exact semantics. Here's the same program as the Java example above in Ruby:
require 'redis'
require 'pp'
NODES = ["redis://SG-example-1.servers.scalegrid.io:6379",
"redis://SG-example-2.servers.scalegrid.io:6379",
"redis://SG-example-3.servers.scalegrid.io:6379",
"redis://SG-example-4.servers.scalegrid.io:6379",
"redis://SG-example-5.servers.scalegrid.io:6379",
"redis://SG-example-6.servers.scalegrid.io:6379"]
begin
pp "Attempting connection..."
redis = Redis.new(cluster: NODES, password: <auth>)
100.times { |i| redis.set("key#{i}", "value#{i}") }
pp "Done..."
redis.close
rescue StandardError => e
puts e.message
end
Connecting With Node.js
Node_redis is the most popular Redis client in Node.js. However, it doesn't officially support Redis Clusters yet. ioredis is another popular Redis client which has Redis Clusters support so we'll use this for our Node.js example.
ioredis
The ioredis documentation describes the details of the additional parameters that must be passed for connecting to Redis Clusters, and a basic example is also provided in the README. Here's an example program which prompts the user for a key and read its value from the Redis Cluster:
const readline = require('readline');
const Redis = require('ioredis');
var cluster = new Redis.Cluster([{
port: 6379,
host: 'SG-example-1.servers.scalegrid.io'
},
{
port: 6379,
host: 'SG-example-2.servers.scalegrid.io'
},
{
port: 6379,
host: 'SG-example-3.servers.scalegrid.io'
},
{
port: 6379,
host: 'SG-example-4.servers.scalegrid.io'
},
{
port: 6379,
host: 'SG-example-5.servers.scalegrid.io'
},
{
port: 6379,
host: 'SG-example-6.servers.scalegrid.io'
}
], { redisOptions: { password: '<auth>' } });
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout,
prompt: 'enter key> '
});
console.log('Welcome to the Redis Cluster reader. Enter the key which you want to read [Ctrl D to Exit]');
rl.prompt();
rl.on('line', (line) => {
if (line.trim()) {
cluster.get(line, function (err, result) {
if (err) {
console.error(err);
} else {
console.log("value: " + result);
}
rl.prompt();
});
} else {
console.error("No input received");
rl.prompt();
}
}).on('close', () => {
console.log('\nterminating');
cluster.quit();
process.exit(0);
});
You should be able to execute any of these examples after installing the most recent versions of the Redis drivers on your client machines.
If you're ready to upgrade your Redis deployments to fully managed Redis hosting, check out the amazing features available on the ScaleGrid platform through a free 30-day trial. Our Redis on AWS plans are available across 14 different datacenters worldwide, and we're the only Redis service that allows you to manage your deployments within your own cloud account.
Top comments (0)