Source of article: https://github.com/Checkson/b...
Preface
Unexpectedly, in my daily development, I encountered the need for a "priority asynchronous queue".The github has similar functionality, and the libraries that integrate redis include Kue, Bull, and so on.However, I am not satisfied with the pursuit of simplicity, ease and lightness.Based on the 28 Principles, I decided to implement a lightweight open source library with only the 20% functionality of the above two: priority-async-queue.
What is the name of an open source library so long?The reason is that I think developers just need to look at the library name to see what it does, and it's not ambiguous at all.However, for fluent wording, priority-async-queue is hereinafter referred to as "paq".
You may not need paq
Following the redux authors'line, first of all, we need to be clear that you may not need paq.
You need to use paq only if you encounter N asynchronous tasks that cannot be executed in parallel and can only be executed sequentially.
Simply put, if you need to perform N asynchronous tasks without resource race and occupation, data sharing, strict logical ordering, priority comparison, etc., paq may be unnecessary and will reduce execution efficiency and affect program performance.
paq design ideas
The paq is designed in three categories:
Task is the execution logic and configuration parameters that describe each task to be executed (asynchronous/synchronous).
PriorityQueue is a priority queue that controls the basic properties and operations of each task to be executed (asynchronous/synchronous).
AsyncQueue is a queue that controls the strict sequential execution of each pending (asynchronous/synchronous) task.
Here is the program flow chart for paq:
Basic concepts of paq and API
1. addTask
addTask is the core method that creates a task and adds it to the paq queue.
paq.addTask([options, ]callback);
options is an optional object that contains the following properties:
{ id: undefined, // Task id priority: 'normal', // Task weights, for example: low, normal, mid, high, urgent context: null, // Execution Task Context start: (ctx, options) => {}, // Callback that task will be executed completed: (ctx, res) => {}, // Callback after task execution is complete failed: (ctx, err) => {}, // Callback after task execution failure remove: (ctx, options) => {} // Callback after task is deleted }
callback is a function that describes the logic of performing a task and contains two parameters: ctx and options:
- ctx is the paq instance to which the task belongs.
-
Options is the final value of the options parameter for this task.
paq.addTask((ctx, options) => {
console.log(ctx === paq); // true});
2. removeTask
The removeTask method deletes tasks from a waiting column based on the task id.
paq.removeTask(taskId);
If the task is successfully deleted, it returns true.Otherwise, it will return false.
3. pause
The pause method is to pause the paq to continue the task.
paq.pause();
Note: However, you cannot pause the task currently executing because the progress of the asynchronous task cannot be temporarily detected.
4. isPause
isPause property that returns whether the current queue is paused.
paq.isPause; // return true or false.
5. resume
resume method, which restarts the paq queue to perform tasks.
paq.resume();
6. clearTask
The cleartTask method, which clears all tasks in the paq wait queue.
paq.clearTask();
paq usage
1. Basic usage
As soon as a task is added to the paq, it is automatically executed.
const PAQ = require('priority-async-queue'); const paq = new PAQ(); paq.addTask((ctx, options) => { console.log('This is a simple task!'); }); // This is a simple task!
2. Synchronize tasks
You can use paq Perform a series of synchronization tasks, such as: const syncTask = (n) => { for (let i = 0; i < n; i++) { paq.addTask(() => { console.log('Step', i, 'sync'); return i; }); } }; syncTask(3); // Step 0 sync // Step 1 sync // Step 2 sync
3. Asynchronous Tasks
You can also use paq to perform a series of asynchronous tasks, such as:
const asyncTask = (n) => { for (let i = 0; i < n; i++) { paq.addTask(() => { return new Promise(resolve => { setTimeout(() => { console.log('Step', i, 'async'); resolve(i); }, i * 1000); }); }); } }; asyncTask(3); // Step 0 async // Step 1 async // Step 2 async
4. Mixed tasks
You can even use paq to perform a series of synchronous and asynchronous staggered tasks, such as:
const mixTask = (n) => { asyncTask(n); syncTask(n); asyncTask(n); }; mixTask(2); // Step 0 async // Step 1 async // Step 0 sync // Step 1 sync // Step 0 async // Step 1 async
5. Binding Execution Context
Sometimes if you need to specify a context to perform a task, for example:
const contextTask = (n) => { var testObj = { name: 'foo', sayName: (name) => { console.log(name); } }; for (let i = 0; i < n; i++) { paq.addTask({ context: testObj }, function () { this.sayName(this.name + i); }); } }; contextTask(3); // foo0 // foo1 // foo2
Note: this does not exist in the arrow function, or it is the context that points to its definition.
6. Delayed execution
paq also supports delayed execution of tasks, such as:
const delayTask = (n) => { for (let i = 0; i < n; i++) { paq.addTask({ delay: 1000 * i }, () => { console.log('Step', i, 'sync'); return i; }); } }; delayTask(3); // Step 0 sync // Step 1 sync // Step 2 sync
7. Priority
If the tasks you need to perform have weights, for example:
const priorityTask = (n) => {
for (let i = 0; i < n; i++) {
paq.addTask({ priority: i === n - 1 ? 'high' : 'normal' }, () => { return new Promise(resolve => { setTimeout(() => { console.log('Step', i, 'async'); resolve(i); }, i * 1000); }); });
}
};
priorityTask(5);
// Step 0 async
// Step 4 async
// Step 1 async
// Step 2 async
// Step 3 async
The default priority mapping is as follows:
{ "low": 1, "normal": 0, // default "mid": -1, "high": -2, "urgent": -3 }
8. Callback function
Sometimes you want to be able to do something when a task starts, completes, fails, or is deleted, for example
const callbackTask = (n) => { for (let i = 0; i < n; i++) { paq.addTask({ id: i, start: (ctx, options) => { console.log('start running task id is', options.id); }, completed: (ctx, res) => { console.log('complete, result is', res); }, failed: (ctx, err) => { console.log(err); } }, () => { if (i < n / 2) { throw new Error(i + ' is too small!'); } return i; }); } }; callbackTask(5); // start running task id is 0 // Error: 0 is too small! // start running task id is 1 // Error: 1 is too small! // start running task id is 2 // Error: 2 is too small! // start running task id is 3 // complete, result is 3 // start running task id is 4 // complete, result is 4
9. Delete Tasks
Sometimes you need to delete tasks, such as:
const removeTask = (n) => { for (let i = 0; i < n; i++) { paq.addTask({ id: i, remove: (ctx, options) => { console.log('remove task id is', options.id); } }, () => { return new Promise(resolve => { setTimeout(() => { console.log('Step', i, 'async'); resolve(i); }, i * 1000); }); }); } console.log(paq.removeTask(3)); console.log(paq.removeTask(5)); }; removeTask(5); // remove task id is 3 // true // false // Step 0 async // Step 1 async // Step 2 async // Step 4 async
Note: You must assign IDS when creating tasks and delete tasks based on id s.
paq events
If you need to monitor the status of a paq queue, paq provides the following event listeners:
1. addTask
paq.on('addTask', (options) => { // Triggered when the queue adds a task. });
2. startTask
paq.on('startTask', (options) => { // Triggered when a task in the queue is about to execute. });
3. changeTask
paq.on('changeTask', (options) => { // Triggered when a task in the queue changes. });
4. removeTask
paq.on('removeTask', (options) => { // Triggered when the queue remove a task. });
5. completed
paq.on('completed', (options, result) => { // Triggered when the task execution in the queue is complete. });
6. failed
paq.on('failed', (options, err) => { // Triggered when a task in the queue fails to execute. });
Finally, I would like to add that if you encounter a need that Promise.all and Proise.race cannot solve, consider paq