Exploring Redis I. Publish/Subcribe pattern

Introduction

In these articles, I'll be exploring the use of Redis beyond storing key-value data. In this particular one, I'll be using it as a Publish/Subcribe messaging paradigm.

Publish/Subcribe pattern

I'm pretty sure that if you've been learning backend development, you already heard of this messaging pattern. The general idea that I get from this pattern it's as follows. The sender of the message, let's call it Publisher, needs to send a message, or several of them, to a receiver or a group of them, let's call these guys Subscribers. This Publisher guy, is an anti-social, he doesn't care who is in the other end, he wants to reach out his subscribers without interacting directly with them. I love this guy 😎!

While the subscriber, want to know about certain contents, so he subscribes to some topics of his interest, and he doesn't care either from whom comes the message, he's hunger for content about this topic.

How is this even possible? 🤔...

Yep, I bet you already figure it out, if they cannot communicate directly, they use a middle man, let's call this guy a broker. Let me clarify that this is not the only way they can communicate, so this is not the only topology to use a more formal name. As an example of another way of Pub/Sub pattern that doesn't use a broker, check out Data Distribution Service it's pretty interesting I didn't know about this, but let's just focus on the broker.

Now that you get a broad idea of how this pattern works, well a silly idea of how it works. You may be wondering where is Redis in all this. Well, Redis can also be used as a middle man between these two anti-socials, between the Publisher and the Subscriber. Let's see Redis in action.

Prerequisites

1. Yes, of course, Redis. You could download it directly from their page (redis download). Just 2.3 mgb, and reading the Readme file, would be enough.
2. (Optional) A redis client in a programming language of your preference, I've used in Python, aioredis, in Go go-redis/redis and in C hiredis. The best documentation is with the Python guys, in my opinion of course, but the three of them are pretty straightforward, with many examples on their source code. Looking at the test folder, could also give you an idea of how to use the library.

Compiling redis

When you downloaded the source code from the previous link, decompress the file and put you in the terminal inside this folder, that at the writing of this tutorial the version of redis is redis-6.2.4. When you are in there, run this:

make

The magic of GNU Makefiles. After the compilation is done, it should show you something like this:

Hint: It's a good idea to run 'make test' ;)

A valid hint, you should, it takes a lot of time, redis is heavily tested.

No code, just redis-server and redis-cli

After the compilation you inside the src folder you should have two binary files, well more than two, but we just care about these two. One is the redis-server and the redis-cli, the names are self-explanatory. Open three terminal on this path, and run:

./redis-server

and

./redis-cli

in the other two. You will notice that redis is extremely fast. In the redis-cli instances, we are going to write out commands that goes to the redis-server.

Now let's say a subscriber want to get messages related to a topic or channel, you could imagine something in your head like this:

SUBSCRIBE fake_news

Well it's like that 👀, yep, not lying to you. But wait, and the publisher? You got me, if you just make this you won't get anything until the publisher decide to publish something on this particular topic, fake_news. Now in the other terminal running the other instance of the redis-cli, you should run this one

PUBLISH fake_news "you won't be bald at all man"

If you keep an eye on the other instance that got the Subscriber, you will see that it receives this message. Take a look, the publisher didn't specify anything about the subscriber, that's right, because he doesn't care he just wants to pass his "fake news".

Now think calmly, why is this pattern so amazing, write down your conclusion, so you don't forget them later when you actually need to use it.

Using hiredis

#include <stdio.h>
#include <hiredis/hiredis.h>

int main(int argc, char *argv[])
{
    redisContext *ctx = NULL;
    redisReply *reply = NULL;

    /*initialize redis context*/
    ctx = redisContext("127.0.0.1", 6379);
    if(!ctx || ctx->err) {
        if(ctx){
            fprintf(stderr, "ERROR: %s", ctx->err);
            redisFree(ctx);
            return 1;
        }   
        else {
            fprintf(stderr, "Couldn't allocate redis context");
            return 1;
        }    
    }
    //now to send a command
    reply = redisCommand(ctx, "SUBSCRIBE fake_news");
    printf("EXTRA EXTRA: %s", reply->str);

    //cleaning the mess
    freeReplyObject(reply);
    redisFree(ctx);
    return 0;
}

For the Publisher would be very similar, you need to change redisCommand with the Publisher's command.

Even in C is not so much code, of course there's a library for that, but that's the point you don't need to reinvent the wheel.

Conclusion

It seems that redis is more than just a "key-value" database, or at least it can be used for other purposes as well. Next time will be using Redis Streams.

Bibliography

1. Official website
2. Redis Pub/Sub
3. Pub/Sub paradigm
4. hiredis documentation