Node Thread Pool ⏫ 🔛

Nodejs introduced worker_threads module to allow a better use of nodejs for CPU intensive tasks, but there is no official thread pool that allow to reuse threads.

So I develop two thread pool on top on worker_threads that you can use with an high level API https://github.com/pioardi/poolifier .

This project contain two thread pool implementations :

The first implementation is a static thread pool , with a defined number of threads that are started at creation time and will be reused.

The second implementation is a dynamic thread pool with a number of threads started at creation time ( these threads will be always active and reused) and other threads created when the load will increase ( with an upper limit, these threads will be reused when active ), the new created threads will be stopped after a configurable period of inactivity.

You have to implement your worker extending the ThreadWorker class

Installation

npm install poolifier --save

Usage

You can implement a worker in a simple way , extending the class ThreadWorker :

'use strict'
const { ThreadWorker } = require('poolifier')

function yourFunction (data) {
  // this will be executed in the worker thread,
  // the data will be received by using the execute method
  return { ok: 1 }
}

class MyWorker extends ThreadWorker {
  constructor () {
    super(yourFunction, { maxInactiveTime: 1000 * 60})
  }
}
module.exports = new MyWorker()

Instantiate your pool based on your needed :

'use strict'
const { FixedThreadPool, DynamicThreadPool } = require('poolifier')

// a fixed thread pool
const pool = new FixedThreadPool(15,
  './yourWorker.js',
  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })

// or a dynamic thread pool
const pool = new DynamicThreadPool(10, 100,
  './yourWorker.js',
  { errorHandler: (e) => console.error(e), onlineHandler: () => console.log('worker is online') })

pool.emitter.on('FullPool', () => console.log('Pool is full'))

// the execute method signature is the same for both implementations,
// so you can easy switch from one to another
pool.execute({}).then(res => {
  console.log(res)
}).catch .... 

Considerations

Contribute

Pull requests, contributors and feedbacks are welcome , join the project :)