Have you fallen into the Promise.all
pitfall? You know the one; you grab a list of things from somewhere and run a parallel function against all of them:
const list = await getMeSomeList()
const results = await Promise.all(list.map(someAsyncFunction))
Works a treat when the list has a few things in it, but lets say there are suddenly 10,000 records returned - this could really get messy.
You are really trying to spin too many plates, and memory or resources are going to become tight...
The Solution
Well you could just install the async package which has lots of useful functions like mapLimit
which will reduce the burden and only run a number in parallel.
If that's overkill - then you can achieve a similar result using a simple rate limiter:
class Semaphore {
constructor(maxConcurrency) {
this.maxConcurrency = maxConcurrency
this.currentConcurrency = 0
this.queue = []
}
async acquire() {
return new Promise((resolve) => {
if (this.currentConcurrency < this.maxConcurrency) {
this.currentConcurrency++
resolve()
} else {
this.queue.push(resolve)
}
})
}
release() {
if (this.queue.length > 0) {
const resolve = this.queue.shift()
resolve()
} else {
this.currentConcurrency--
}
}
}
export function rateLimit(asyncFunction, rate) {
const semaphore = new Semaphore(rate)
return async function process(...args) {
await semaphore.acquire()
try {
return await asyncFunction(...args)
} finally {
semaphore.release()
}
}
}
With that in hand your code would change to be just this:
const list = await getMeSomeList()
const results = await Promise.all(list.map(rateLimit(someAsyncFunction, 20))
This would mean that it would keep 20 running at a time until the list way finished. Every time one of the someAsyncFunction
s returns another one is started until the list is exhausted. Easy right :)
Top comments (7)
Is 20 the recommended number? Surely you should have a much higher limit, correct?
It depends on what the async function is doing :) Normally I'd set a 100 or so. But it depends, if you are querying a database then perhaps no more than 20 or 30 in parallel to avoid too much contention or too many database connections required. Given the speed of Redis, my tests have shown 20 - 30 concurrent is optimal under my configuration.
If you run HTTP requests in parallel, you'll hit that limit
Dead right, especially on HTTP rather than HTTP2
My library has a function that was written for a similar purpose.
How is it?
I see that you are returning an array of results of exceptions, that's an interesting concept and would fit certain circumstances well. If you require all of the results then the Promise.all works well as it throws an exception on the first one that fails which stops the execution of the other remaining ones.
Just added this to my code base! it's really useful, thanks!