DEV Community

Cover image for How to integrate Queue in an Express application
Le Quan Phat
Le Quan Phat

Posted on

How to integrate Queue in an Express application

To integrate a queue in an Express application, you can use a queueing library like Bull (which works with Redis) to handle background tasks and job processing. Here’s a step-by-step guide:

1. Install Bull and Redis

First, install Bull and Redis client libraries in your Express project:

npm install bull redis
Enter fullscreen mode Exit fullscreen mode

Make sure Redis is installed and running. If you don’t have Redis installed locally, you can run it using Docker:

docker run -p 6379:6379 -d redis
Enter fullscreen mode Exit fullscreen mode

2. Set Up a Redis Client Configuration

In your project/config folder, create a redisConfig.js file to configure the Redis connection.

const Redis = require('redis');

const redisClient = Redis.createClient({
  host: '127.0.0.1',
  port: 6379,
});

redisClient.on('error', (err) => console.error('Redis Client Error', err));
redisClient.connect();

module.exports = redisClient;
Enter fullscreen mode Exit fullscreen mode

3. Initialize a Queue Using Bull

Create a queue using Bull. In the project/config folder, create a queue.js file.

const Queue = require('bull');

// Initialize a new queue
const exampleQueue = new Queue('example-queue', {
  redis: {
    host: '127.0.0.1',
    port: 6379,
  },
});

module.exports = exampleQueue;
Enter fullscreen mode Exit fullscreen mode

4. Create a Queue Processing Function

In your project/services folder, create a file called queueService.js to handle processing tasks from the queue.

const exampleQueue = require('../config/queue');

// Process jobs in the queue
exampleQueue.process(async (job) => {
  // job.data contains the data passed when adding the job
  console.log(`Processing job with ID: ${job.id} and data:`, job.data);
  // Simulate a task, e.g., sending an email, resizing an image, etc.
  return `Processed data: ${job.data}`;
});

module.exports = exampleQueue;
Enter fullscreen mode Exit fullscreen mode

5. Add a Job to the Queue from a Route

Create an endpoint in project/routes/queueRoutes.js to add a job to the queue:

const express = require('express');
const router = express.Router();
const exampleQueue = require('../config/queue');

// Route to add a job to the queue
router.post('/add-job', async (req, res) => {
  const { data } = req.body;
  try {
    const job = await exampleQueue.add(data);
    res.status(200).send(`Job added with ID: ${job.id}`);
  } catch (error) {
    res.status(500).send(`Failed to add job: ${error.message}`);
  }
});

module.exports = router;
Enter fullscreen mode Exit fullscreen mode

6. Add the Route to the Express App

In project/app.js, import and use the queue routes.

const express = require('express');
const queueRoutes = require('./routes/queueRoutes');

const app = express();
app.use(express.json());

// Other routes...

app.use('/queue', queueRoutes);

module.exports = app;
Enter fullscreen mode Exit fullscreen mode

7. Run and Test Your Application

Start your server:

node app.js
Enter fullscreen mode Exit fullscreen mode

Use Postman or a similar tool to test the queue integration by making a POST request to http://localhost:3000/queue/add-job with JSON data:

{
  "data": {
    "task": "example task",
    "details": "some task details"
  }
}
Enter fullscreen mode Exit fullscreen mode

You should see the job added and processed in the console logs.

8. Monitor the Queue (Optional)

For monitoring, you can use Bull Dashboard to track your jobs:

  • Install the Bull Board library:
npm install @bull-board/express
Enter fullscreen mode Exit fullscreen mode
  • Set up the Bull Board in app.js:
const { ExpressAdapter, createBullBoard } = require('@bull-board/express');
const exampleQueue = require('./config/queue');

const serverAdapter = new ExpressAdapter();
createBullBoard({
  queues: [new BullAdapter(exampleQueue)],
  serverAdapter,
});

serverAdapter.setBasePath('/admin/queues');
app.use('/admin/queues', serverAdapter.getRouter());
Enter fullscreen mode Exit fullscreen mode

Now you can navigate to http://localhost:3000/admin/queues to view and manage your jobs in the browser.

This setup integrates a queue into your Express app, allowing you to add and process background tasks easily. Let me know if you need further customization on this!

Top comments (0)