DEV Community

Cover image for Avoid the Promise.all pitfall! Rate limit async function calls
Mike Talbot ⭐
Mike Talbot ⭐

Posted on • Updated on

Avoid the Promise.all pitfall! Rate limit async function calls

Have you fallen into the Promise.all pitfall? You know the one; you grab a list of things from somewhere and run a parallel function against all of them:



     const list = await getMeSomeList()
     const results = await Promise.all(list.map(someAsyncFunction))


Enter fullscreen mode Exit fullscreen mode

Works a treat when the list has a few things in it, but lets say there are suddenly 10,000 records returned - this could really get messy.

You are really trying to spin too many plates, and memory or resources are going to become tight...

Man spinning many plates

The Solution

Well you could just install the async package which has lots of useful functions like mapLimit which will reduce the burden and only run a number in parallel.

If that's overkill - then you can achieve a similar result using a simple rate limiter:



class Semaphore {
    constructor(maxConcurrency) {
        this.maxConcurrency = maxConcurrency
        this.currentConcurrency = 0
        this.queue = []
    }

    async acquire() {
        return new Promise((resolve) => {
            if (this.currentConcurrency < this.maxConcurrency) {
                this.currentConcurrency++
                resolve()
            } else {
                this.queue.push(resolve)
            }
        })
    }

    release() {
        if (this.queue.length > 0) {
            const resolve = this.queue.shift()
            resolve()
        } else {
            this.currentConcurrency--
        }
    }
}

export function rateLimit(asyncFunction, rate) {
    const semaphore = new Semaphore(rate)

    return async function process(...args) {
        await semaphore.acquire()
        try {
            return await asyncFunction(...args)
        } finally {
            semaphore.release()
        }
    }
}



Enter fullscreen mode Exit fullscreen mode

With that in hand your code would change to be just this:



     const list = await getMeSomeList()
     const results = await Promise.all(list.map(rateLimit(someAsyncFunction, 20))


Enter fullscreen mode Exit fullscreen mode

This would mean that it would keep 20 running at a time until the list way finished. Every time one of the someAsyncFunctions returns another one is started until the list is exhausted. Easy right :)

Top comments (7)

Collapse
 
wakywayne profile image
wakywayne

Is 20 the recommended number? Surely you should have a much higher limit, correct?

Collapse
 
miketalbot profile image
Mike Talbot ⭐

It depends on what the async function is doing :) Normally I'd set a 100 or so. But it depends, if you are querying a database then perhaps no more than 20 or 30 in parallel to avoid too much contention or too many database connections required. Given the speed of Redis, my tests have shown 20 - 30 concurrent is optimal under my configuration.

Collapse
 
voltra profile image
Voltra

If you run HTTP requests in parallel, you'll hit that limit

Collapse
 
miketalbot profile image
Mike Talbot ⭐

Dead right, especially on HTTP rather than HTTP2

Collapse
 
artxe2 profile image
Yeom suyun

My library has a function that was written for a similar purpose.
How is it?

/**
 * Run multiple functions in parallel with a specified limit on the number of parallel executions.
 * @param {number} size
 * @param {Function[]} callbacks
 * @returns {Promise<({ value: * }|{ reason: * })[]>}
 */
const parallel = (size, ...callbacks) => {
    /** @type {({ value: * }|{ reason: * })[]} */
    const result = []
    return new Promise(resolve => {
        const length = callbacks.length
        if (length < size) size = length
        let index = 0
        const finally_callback = () => {
            if (index < length) run(index++)
            else if (++index == length + size) resolve(result)
        }
        /** @param {number} i */
        const run = (i) =>
            Promise.resolve(callbacks[i]())
                .then(value => result[i] = { value }, reason => result[i] = { reason })
                .finally(finally_callback)
        while (index < size) run(index++)
    })
}
Enter fullscreen mode Exit fullscreen mode
Collapse
 
miketalbot profile image
Mike Talbot ⭐

I see that you are returning an array of results of exceptions, that's an interesting concept and would fit certain circumstances well. If you require all of the results then the Promise.all works well as it throws an exception on the first one that fails which stops the execution of the other remaining ones.

Collapse
 
midasxiv profile image
Midas/XIV

Just added this to my code base! it's really useful, thanks!