Redis Async Generators

If you have never gone through Joshua Bloch's API Design in Bumper Stickers presentation, then you certainly should. Here are four of the guidelines he's suggesting:

APIs should be easy to use and hard to misuse.

When in Rome, do as the Romans do.

APIs should be self-documenting

Don't make the client do anything the library could do.

It sounds so obvious. However, I don't have to think too deep to come up with examples of APIs that clearly miss the mark.

To be fair, "easy to use" is a little vague. "Easy to use" might mean that it requires very little code to use the API. But it could also mean that the code you do have to write is idiomatic for the language in which you're coding. Or perhaps it should be easy to read, without even knowing the library.

Scala's Dispatch library is an example of a library that optimized for compactness of the calling code instead of readability. And with Scala's flexible syntax, nobody even knew for sure what "idiomatic" really meant in Scala's world.

JavaScript is different. It's syntax is less flexible than Scala's. It doesn't allow you to "extend" the syntax and make it behave as something else. You have to live with whatever it has baked in. As a consequence, I would expect JavaScript libraries to converge more quickly towards an idiomatic and common way of doing things.

So, moving on to Redis. Redis is — in a way — a foreign entity in JavaScript. It's API wasn't designed with JavaScript in mind in particular. It seems that some of that spilled over in its client binding. A lot of things would be much simpler if the standard redis library for node would be based on promises rather than callbacks. I'd say promises and async await constructs are the more idiomatic way to do things these days.

The same goes for iterators. If you have code that iterates over a collection, then Redis' JavaScript API makes that quite painful. That is, it requires quite some careful control by the caller. It violates not just one but all of four of the above guidelines from Josh Bloch's talk.

This is a slightly modified version of some code I found on StackOverflow to iterate over all keys matching the pattern person*.

let cursor = '0'
let redisClient = …
function scan (pattern, callback) {

  redisClient.scan(cursor, 'MATCH', pattern, 'COUNT', '1000', function (err, reply) {
    if(err){
        throw err
    }
    cursor = reply[0]
    if(cursor !== '0') {
        let keys = reply[1]
        keys.forEach(callback)
        return scan(pattern,callback)
    }
  })
}

scan('person*', console.info);

I found this code hard to digest. You need to understand about ‘0’ being the magic marker to mark the end of the matching values. That not only feels not all that idiomatic, but it also forces me to control a whole bunch of things I’d rather leave to the API instead.

So therefore, I present to you redis-async-gen — async generators for Redis. That’s quite a mouthful, but the important thing is that it compacts this code considerably, that it’s easier on the eyes, and that it relieves the caller from having to carefully manage a cursor:

const redisClient = …
const generators = require('redis-async-gen')
const { keysMatching } = generators.using(redisClient)

for await (const key of keysMatching('person*')) {
  console.info(key)
}