You don't know Redis

In my previous post, I touched on the point that Redis is more than just an in-memory cache.

Most people do not even consider Redis as a primary database. There are a lot of use cases where Redis is a perfect choice for non-cache related tasks.

In this article, I will demonstrate how I built a fully functional Q&A board for asking and upvoting the most interesting questions. Redis will be used as a primary database.

I will use Gatsby (React), Netlify serverless functions and Upstash Serverless Redis.

Upstash has been a good choice so far and I decided to try it out in a more serious project. I love everything serverless and how it makes things simpler for me.

Serverless will be a great choice for most tasks however you need to know the pros and cons of the tech you are using. I encourage you to learn more about serverless to get the most out of it.

Q&A board features

As you may know, I run a tech newsletter for recruiters where I explain complex tech in simple terms. I have an idea to collect questions from recruiters using a Q&A board and let them vote for questions.

All questions will eventually be answered in my newsletter, however, the most upvoted questions will be addressed first.

Anyone can upvote a question and registration is not required.

Questions will be listed in three tabs:

• Active - questions sorted by votes and available for voting.
• Most recent - questions sorted by date (newest first).
• Answered - only questions that have answers.

Upvoting will be one of the most frequently used features and Redis has a data type and optimized commands for it.

A Sorted set is ideal for this task because all its members are automatically sorted by the score.

Scores are numeric values that we will associate with votes. It is very easy to increment a score (add a vote) by using the ZINCRBY command.

We will also leverage scores for handling unmoderated questions by setting the score for them to 0. All approved questions will have a score of 1+.

It allows us to fetch all unmoderated questions by simply using the ZRANGEBYSCORE command specifying the min and max arguments as 0.

To fetch all approved questions sorted by the score (highest first) we can use the ZREVRANGEBYSCORE command setting the min score argument to 1.

This is great that by using just a few Redis commands we can also solve logical tasks along the way. Lower complexity is a huge benefit.

We will also use sorted sets for sorting questions by date or filtering questions that have answers. I will explain it in more detail in a moment.

Less frequent operations, namely creating, updating and deleting questions are also easy to accomplish using hashes.

Implementation details

The most interesting part is always the actual implementation. I use serverless functions and the ioredis library and I will link the source code explaining what it does.

This article is dedicated to client-facing functionality. Although I will explain admin-related functions, in the final source code there will be no backend interface. You will need to use Postman or a similar tool to call the admin related endpoints.

Let’s take a look at the API endpoints and what they do.

Add a question

Users can create questions. All questions require moderation before they become visible.

A question is an object and Redis hash is a perfect data type to represent objects.

This is the structure of a questions:
{"datetime":"1633992009", "question":"What are Frontend technologies?", "author":"Alex", "email":"alex@email.com", “score:” “0”, “url”: “www.answer.com” }

We will store questions in hashes using the HMSET command which takes a key and multiple key-value pairs.

The key schema is question:{ID} where ID is the question ID generated using the uuid library.

This is a new question and there is no answer yet. We skip the url property but it will be an easy task to add it later using the HSET command.

The score for a newly created question is 0 by default. By our design, it means that this question needs moderation and will not be listed because we only fetch questions with scores starting from 1.

Since we keep the score value in a hash, we’ll need to update it whenever it changes. There is a HINCRBY command that we can use to easily increment values in hashes.

As you can see, using Redis hashes solves a lot more for us than just storing data.

Now that we know how we’ll store questions, we also need to keep track of questions to be able to fetch them later.

For that, we add the ID of a question to a sorted set with a score of 0 using the ZADD command. A sorted set will allow us to fetch question IDs sorted by scores.

As you can see, we are setting the score to 0 just like we do it for the score property in the hash above. The reason why we duplicate the score in a hash is that we need it when showing the most recent questions or questions that have answers.

For instance, the most recent questions are stored in a separate sorted set with timestamp as a score hence the original score value is not available unless it’s duplicated in a hash.

Since we store the score in two places, we need to make sure that values are updated both in a hash and in a sorted set. We use the MULTI command to execute commands in a manner where either all commands are executed successfully or they are rolled back. Check Redis Transactions for more details.

We will use this approach where applicable. For example, HMSET and ZADD will also be executed in a transaction (see source code below).

ZADD command takes a key and our schema for it is questions:{boardID}

All questions are mapped to a boardID. For now, it’s a hardcoded value because I need one board only. In the future, I may decide to introduce more boards, for example, separately for Frontend, Backend, QA and so on. It’s good to have the needed structure in place.

Endpoint:
POST /api/create_question

Here is the source code for the create_question serverless function.

Approve a question

Before a question becomes available for voting, it needs to be approved. Approving a question means the following:

1. Update the score value in hash from 0 to 1 using HINCRBY command.
2. Update the score value in the questions:{boardID} sorted set from 0 to 1 using the ZADD command.
3. Add the question ID to the questions:{boardID}:time sorted set with the timestamp as the score to fetch questions sorted by date (most recent questions) using the same ZADD command.

We can get the timestamp by looking up the question by its ID using the HGET command.

Once we have it, we can execute the remaining three commands in a transaction. This will ensure that the score value is identical in the hash and the sorted set.

To fetch all unapproved questions the ZRANGEBYSCORE command is used with the min and max values as 0.

ZRANGEBYSCORE returns elements ordered by a score from low to high while ZREVRANGEBYSCORE - from high to low. We’ll use the latter to fetch questions ordered by the number of votes.

Endpoint for fetching all unapproved questions:
GET /api/questions_unapproved

Endpoint for approving a question:
PUT: /api/question_approve

Here is the source code for the questions_unapproved serverless function. For the most part, this code is similar to other GET endpoints and I will explain it in the next section.

Here is the source code for the question_approve serverless function.

Fetch approved questions

To fetch all approved questions we use the ZREVRANGEBYSCORE command setting the min argument to 1 in order to skip all unapproved questions.

As a result, we get a list of IDs only. We will need to iterate over them to fetch question details using the HGETALL command.

Depending on the number of questions fetched, this approach can become expensive and block the event loop in Node (I am using Node.js). There are a few ways to mitigate this potential problem.

For example, we can use ZREVRANGEBYSCORE with the optional LIMIT argument to only get a range of elements. However, if the offset is large, it can add up to O(N) time complexity.

Or we can use a Lua script to extend Redis by adding a custom command to fetch question details based on IDs from a stored set without us doing it manually in the application layer.

In my opinion, it would be overhead in this case. Besides that, one must be very careful with Lua scripts because they block Redis and you can’t do expensive tasks with them without introducing performance degradation. This approach may be cleaner however we would still use the LIMIT to avoid large amounts of data.

Always research the pros and cons before the final implementation. As long as you understand the potential issues and have evaluated ways to mitigate them, you are safe.

In my case, I know that it will take significant time before I will have enough questions to face this issue. No need for premature optimization.

Endpoint:
GET /api/questions

Here is the source code for the questions serverless function.

Vote for a question

The process of upvoting a question consists of two important steps that both need to be executed as a transaction.

However, before manipulating the score, we need to check if this question has no answer (url property). In other words, we do not allow anyone to vote for questions that have been answered.

The vote button is disabled for such questions. But we do not trust anyone on the internet and therefore check on the server if a given ID exists in the questions:{boardID}:answered sorted set using the ZSCORE command. If so, we do nothing.

We use the HINCRBY command to increment the score in the hash by 1 and the ZINCRBY command to increment the score in the sorted set by 1.

Endpoint:
PATCH /api/question_upvote

Here is the source code for the question_upvote serverless function.

Fetch most recent approved questions

It’s very similar to how we fetch all approved questions with the only difference being that we read another sorted set where the key schema is questions:{boardID}:time. Since we used the timestamp as a score, the ZREVRANGEBYSCORE command returns IDs sorted in descending order.

Endpoint:
PATCH /api/questions_recent

Here is the source code for the questions_recent serverless function.

Update a question with an answer

Updating or adding new properties to hashes is simple with the HSET command. However, when we add an answer, we move the question from the questions:{boardID} sorted set to the questions:{boardID}:answered one preserving the score.

To do so, we need to know the score of the question and we obtain it using the ZSCORE command. Answered questions will be sorted by score in descending order.

Then we can:

1. update the hash with the url property using the HSET command;
2. add the hash to the questions:{boardID}:answered sorted set using ZADD;
3. remove the question from the questions:{boardID} sorted set running the ZREM command.
4. remove the question from the questions:{boardID}:time sorted set running the ZREM command.

All four commands are executed in a transaction.

Endpoint:
PATCH /api/question_add_answer

Here is the source code for the question_add_answer serverless function.

Fetch questions with answers

Again, the process is similar to fetching all approved questions. This time from the questions:{boardID}:answered sorted set.

Endpoint:
PATCH /api/questions_unswered

Here is the source code for the questions_unswered serverless function.

---

Full source code.
Working DEMO on my website.

Conclusion

Redis has a lot of use-cases going way beyond cache. I’ve demonstrated only one of the multiple applications for Redis that one can consider instead of reaching for an SQL database right away.

Of course, if you already use a database, adding yet another one may be an overhead.

Redis is very fast and scales well. Most commercial projects have Redis in their tech stack and often use them as an auxiliary database, not just in-memory cache.

I strongly recommend learning about Redis data patterns and best practices to realize how powerful it is and benefit from this knowledge in the long run.

Check my previous article where I created LinkedIn-like reactions with Serverless Redis if you haven’t already.

Here is You don't know Redis (Part 2)

Follow for more.