Redis Bull Queue with Nodejs

Hello readers, this article is going to be an interesting article where we will try to understand how the queue works and how to implement it in Nodejs with Redis.

What we will cover in this article

1. What is Queue?
2. Why should we use it?
3. Understand the basics of the bull module.
4. Implementation of Queue in Node.js with Bull module.

What is Queue?

• A queue is a data structure which works on FIFO (First In First Out) principle

Queue Image

• As we can see in the queue the first job which was in the queue will come out first from the queue.

Why should we use it?

• The queue can be used for multiple scenarios like sending notifications to the users.
• Suppose there are around 1M data which need to be processed by the server but the server can handle only 1K data at a time. So, to handle this situation queue can be used.

Understand the basics of bull Module

• Bull is 3rd party library which is made on top of the Redis cache.

• Explanation of the above diagram

As we know that every queue has 2 properties. One is adding the data into a queue and the other is removing data from the queue.

• ADD Data into the queue: Adding data into the queue is the first part of the queue where all data gets added to the Redis queue.
• Remove data from Queue: To remove data from the queue is called worker. So, from where all data or jobs get dequeue from the queue.

Implementation of Queue in Node.js with Bull module.

• Clone the GitHub repo.
• After that checkout, the branch to
  git checkout QueueImplementationWithBull

Explanation of file and code

• Understand the files of the repo

• In this file structure
  queueList : https://github.com/airrakeshkumarsharma/nodejsExperiment/blob/QueueImplementationWithBull/queueList.js
  It holds all lists of a queue which is created by the bull queue.
  addJob:
  https://github.com/airrakeshkumarsharma/nodejsExperiment/blob/QueueImplementationWithBull/addJob.js
  It holds the logic for adding data into a queue
  jobWorker
  https://github.com/airrakeshkumarsharma/nodejsExperiment/blob/QueueImplementationWithBull/jobWorker.js
  job worker contains logic to dequeue the job from the queue and process it.

Run the project

npm install

node addJob.js

NOTE: No need to run jobWorker file because it is a process. So, just this file needs to access by the nodejs project. So, that is done in this file at line number 3.

Feedback: Thank you for reading this article. I hope you understood the basics of implementing queue from scratch in node.js. Please feel free to ask questions or give suggestions to improve the content quality. Visit this for all articles.