Usually when we talk about Node. js, we talk about asynchrony. In fact, Node.js performs asynchronous calls in different scenarios. This article will analyze how Node. JS makes asynchronous calls through libuv thread pool through libuv source code. The Node.js version described in this article is v11.15.0 The libuv version is 1.24.0 .
Take the following code as an example, which calls fs.access To asynchronously determine whether a file exists and print a log in a callback, this is a typical asynchronous call in Node.js.
const fs = require('fs') const cb = function (err) { console.log(`Is myfile exists: ${!err}`) } fs.access('myfile', cb)
Before we analyze the calling process of the above code, let's look at some libuv concepts.
What type of request libuv will put it in the thread pool to execute
Active operations initiated through libuv are referred to by libuv as [requests]request libuv's thread pool acts on the following four enumerated asynchronous requests:
- UV_FS: fs Module asynchronous functions (except uv_fs_req_cleanup )，fs.access,fs.stat And so on.
- UV_GETADDRINFO: dns Module asynchronous function, dns.lookup And so on.
- UV_GETNAMEINFO: dns Module asynchronous function, dns.lookupService And so on.
- UV_WORK: zlib Modular zlib.unzip,zlib.gzip In Addresson (C/C++) of Node.js uv_queue_work Create multithreaded requests.
Other UV_CONNECT, UV_WRITE, UDP_SEND and so on are not executed through the thread pool.
Thread pool request classification
These four enumeration requests are internally classified into three [task types]Task type:
- UV_WORK_CPU: CPU-intensive, and requests of type UV_WORK are defined as this type. Therefore, according to this classification, I/O intensive operations in uv_queue_work are not recommended.
- UV_WORK_FAST_IO: Fast IO type, which is defined for requests of type UV_FS.
- UV_WORK_SLOW_IO: Slow IO, UV_GETADDRINFO and UV_GETNAMEINFO requests are defined as this type.
The process of execution of UV_WORK_SLOW_IO is different from that of UV_WORK_CPU and UV_WORK_FAST_IO. There are some differences when libuv executes it, which will be mentioned later.
How the thread pool is initialized
libuv pass init_threads The function initializes the thread pool according to a name UV_THREADPOOL_SIZE The environment variable initializes the size of the internal thread pool. The maximum number of threads is 128 The default is 4 . If the service is deployed in a single-process architecture, the thread pool size can be set according to the number of core CPU s and business conditions of the server to maximize resource utilization. When a uv loop thread creates a worker thread, it initializes the following variables:
- Semaphore sem: synchronize with threads when creating threads. After each thread is created, the semaphore will inform the uv loop thread that it has initialized and can start processing requests. When all threads are initialized, the semaphore is destroyed, that is, the initialization of the thread pool is completed.
- Conditional variable cond: When the thread is created, it enters the blocking state through the conditional variable.( uv_cond_wait Until other threads pass uv_cond_signal Wake it up.
- mutex: mutex access to the following three critical resources.
- Request queue wq: The thread pool receives requests of type UV__WORK_CPU and UV__WORK_FAST_IO, inserts them at the end of the queue and passes through uv_cond_signal Wake up the worker thread to process, which is the main queue of thread pool requests.
- Slow I/O queue slow_io_pending_wq: The thread pool receives a request of type UV_WORK_SLOW_IO and inserts it at the end of the queue.
- Slow I/O flag node run_slow_work_message: When there is a slow I/O request, it is used as a flag bit in the request queue WQ to indicate that there are currently slow I/O requests, and the worker thread needs to pay attention to the requests of the slow I/O queue when processing the requests of the slow I/O queue; when all the requests of the slow I/O queue are processed, the flag bit will be removed from the request queue wq.
The entry functions of worker threads are worker Function, let's talk about that later. init_threads The realization is as follows:
static void init_threads(void) { unsigned int i; const char* val; uv_sem_t sem; // Lines 6-23 initialize thread pool size nthreads = ARRAY_SIZE(default_threads); val = getenv("UV_THREADPOOL_SIZE"); // Setting thread pool size based on environment variables if (val != NULL) nthreads = atoi(val); if (nthreads == 0) nthreads = 1; if (nthreads > MAX_THREADPOOL_SIZE) nthreads = MAX_THREADPOOL_SIZE; threads = default_threads; if (nthreads > ARRAY_SIZE(default_threads)) { threads = uv__malloc(nthreads * sizeof(threads[0])); if (threads == NULL) { nthreads = ARRAY_SIZE(default_threads); threads = default_threads; } } // Initialize conditional variables if (uv_cond_init(&cond)) abort(); // Initialize mutex if (uv_mutex_init(&mutex)) abort(); // Initialize queues and nodes QUEUE_INIT(&wq); // Work queue QUEUE_INIT(&slow_io_pending_wq); // Slow I/O queue QUEUE_INIT(&run_slow_work_message); // If there is a slow I/O request, insert this node as a flag bit into wq // Initialization semaphore if (uv_sem_init(&sem, 0)) abort(); // Subsequent thread synchronization relies on this semaphore, so if the semaphore creation fails, the process terminates. // Create worker threads for (i = 0; i < nthreads; i++) if (uv_thread_create(threads + i, worker, &sem)) // Initialize worker threads abort(); // One of the reasons for woker thread creation errors is EAGAIN, EINVAL, EPERM. Refer to man 3 for details. // Waiting for worker creation to complete for (i = 0; i < nthreads; i++) uv_sem_wait(&sem); // Waiting for the worker thread to be created // Recovery of semaphore resources uv_sem_destroy(&sem); }
How does the request go into the thread pool to execute
libuv has two functions to create multithreaded requests:
- uv_queue_work Developers often use functions to create multithreaded requests.
- uv__work_submit libuv creates functions for multithreaded requests internally, in fact uv_queue_work This function is also called eventually.
uv__work_submit Functions do two main things:
- call init_threads Initialize the thread pool, because the creation of the thread pool is lazy and will only be created when it is used.
- Call internal post The function inserts the request into the request queue.
The realization is as follows:
void uv__work_submit(uv_loop_t* loop, struct uv__work* w, enum uv__work_kind kind, void (*work)(struct uv__work* w), void (*done)(struct uv__work* w, int status)) { // Initialization of the thread pool begins only after a request is received, but only once uv_once(&once, init_once); w->loop = loop; w->work = work; w->done = done; post(&w->wq, kind); } static void init_once(void) { // After fork, the state of pthread variables such as mutex, condition variables of the child process is replicated when the parent process fork, so the state of the child process needs to be reset when it is created. // Refer specifically to http://man7.org/linux/man-pages/man2/fork.2.html if (pthread_atfork(NULL, NULL, &reset_once)) abort(); // Initialize thread pool init_threads(); } static void reset_once(void) { // Reset once variable uv_once_t child_once = UV_ONCE_INIT; memcpy(&once, &child_once, sizeof(child_once)); }
post Functions do two main things:
-
Determine whether the request type is UV_WORK_SLOW_IO:?
- If so, insert the request at the end of the slow_io_pending_wq request queue, and insert a run_slow_work_message node at the end of the request queue WQ as a flag bit to inform the request queue that there are currently slow I/O requests.
- If not, insert the request at the end of the request queue wq.
- If there is an idle thread, wake up one to execute the request.
The number of concurrent slow I/O requests will not exceed half the size of the thread pool. The advantage of this is to avoid multiple slow I/O requests filling up all threads in a certain period of time, resulting in other requests that can be executed quickly that need to be queued.
post The function is implemented as follows:
static void post(QUEUE* q, enum uv__work_kind kind) { // Lock up uv_mutex_lock(&mutex); if (kind == UV__WORK_SLOW_IO) { /* Insert into slow I/O queues */ QUEUE_INSERT_TAIL(&slow_io_pending_wq, q); /* If the run_slow_work_message node is not empty, it is already in the wq queue and does not need to be inserted again */ if (!QUEUE_EMPTY(&run_slow_work_message)) { uv_mutex_unlock(&mutex); return; } // If not in the wq queue, insert run_slow_work_message as a flag bit into the tail of wq q = &run_slow_work_message; } // Insert the request at the end of the request queue QUEUE_INSERT_TAIL(&wq, q); // If there is an idle thread, wake up one to execute the request if (idle_threads > 0) uv_cond_signal(&cond); // Wake up a worker thread uv_mutex_unlock(&mutex); }
Entry function of worker thread worker After the thread is created and initialized, the following steps will be followed continuously:
- Wait for awakening.
- Take out the request queue wq or slow I/O request queue head request to execute.
- Notify the uv loop thread that a request has been processed.
- Back to 1.
static void worker(void* arg) { struct uv__work* w; QUEUE* q; int is_slow_work; // Notify the uv loop thread that the worker thread has been created uv_sem_post((uv_sem_t*) arg); arg = NULL; uv_mutex_lock(&mutex); // Continuous execution of requests through this dead cycle for (;;) { /* This while has two judgments 1. In multi-core processors, pthread_cond_signal may activate more than one thread. To avoid the problem caused by this situation, use a while. See https://linux.die.net/man/3/pthread_cond_signal specifically. 2. Limit the number of slow I/O requests to less than half the number of threads */ while (QUEUE_EMPTY(&wq) || (QUEUE_HEAD(&wq) == &run_slow_work_message && QUEUE_NEXT(&run_slow_work_message) == &wq && slow_io_work_running >= slow_work_thread_threshold())) { idle_threads += 1; // The worker thread is blocked when initialization is complete or no request is executed until it is awakened by a new request uv_cond_wait(&cond, &mutex); idle_threads -= 1; } // A request to remove the head of the queue after waking up and meeting the conditions for executing the request q = QUEUE_HEAD(&wq); // If the header request exits, the worker thread is terminated by jumping out of the loop if (q == &exit_message) { // Continue to wake up other worker s to terminate threads uv_cond_signal(&cond); uv_mutex_unlock(&mutex); break; } // Remove this request node from the request queue wq QUEUE_REMOVE(q); QUEUE_INIT(q); is_slow_work = 0; // If this request is a slow I/O flag bit if (q == &run_slow_work_message) { /* Control the number of slow I/O requests, insert them at the end of the queue, and wait for the previous request to be executed. */ if (slow_io_work_running >= slow_work_thread_threshold()) { QUEUE_INSERT_TAIL(&wq, q); continue; } /* Determine if there is a request in the slow I/O request queue and the request may be cancelled */ if (QUEUE_EMPTY(&slow_io_pending_wq)) continue; is_slow_work = 1; slow_io_work_running++; // Requests to take out the head of the slow I/O request queue q = QUEUE_HEAD(&slow_io_pending_wq); QUEUE_REMOVE(q); QUEUE_INIT(q); // If there are still requests in the slow I/O request queue, insert the flag run_slow_work_message back to the end of the request queue wq if (!QUEUE_EMPTY(&slow_io_pending_wq)) { QUEUE_INSERT_TAIL(&wq, &run_slow_work_message); if (idle_threads > 0) uv_cond_signal(&cond); // Wake up a thread to continue execution } } uv_mutex_unlock(&mutex); w = QUEUE_DATA(q, struct uv__work, wq); // That's all I've done, and I'm finally starting to execute the requested function here. w->work(w); uv_mutex_lock(&w->loop->wq_mutex); w->work = NULL; // In order to ensure thread safety, the request will not be called back immediately after execution, but will be inserted into the completed request queue to complete the callback in the uv loop thread. QUEUE_INSERT_TAIL(&w->loop->wq, &w->wq); // Synchronize uv loop threads through uv_async_send: Thread pool completes a request uv_async_send(&w->loop->wq_async); uv_mutex_unlock(&w->loop->wq_mutex); uv_mutex_lock(&mutex); if (is_slow_work) { slow_io_work_running--; } } }
How does the request synchronize the thread where the uv loop is located after the worker executes
stay uv_loop_init When [wq_async] of the thread pool wq_async Handle pass uv_async_init Initialize and insert into uv loop async_handles In the queue, and then traverse in the uv loop thread async_handles Queue and complete callback.
worker threads and uv loop threads pass through uv_async_send Synchronize, and uv_async_send Just one thing: to async_wfd The handle writes a string of 1 byte long (only the 0 character).
uv_async_send The realization is as follows:
int uv_async_send(uv_async_t* handle) { if (ACCESS_ONCE(int, handle->pending) != 0) return 0; // The cmpxchgi function sets the flag bit, and if it has already been set, it will not call uv__async_send repeatedly. if (cmpxchgi(&handle->pending, 0, 1) == 0) uv__async_send(handle->loop); return 0; } static void uv__async_send(uv_loop_t* loop) { const void* buf; ssize_t len; int fd; int r; buf = ""; len = 1; fd = loop->async_wfd; #if defined(__linux__) if (fd == -1) { static const uint64_t val = 1; buf = &val; len = sizeof(val); fd = loop->async_io_watcher.fd; /* eventfd */ } #endif do r = write(fd, buf, len); // Write content to fd while (r == -1 && errno == EINTR); if (r == len) return; if (r == -1) if (errno == EAGAIN || errno == EWOULDBLOCK) return; abort(); }
Yes async_wfd Why can content be synchronized? In fact, in the worker thread pair async_wfd When writing, the uv loop thread is also constantly running loop To receive and process a variety of events or requests, including async_wfd Listening for readable events.
uv loop is uv_run Executed in the function, it's Node.js Startup time Called, uv_run The realization is as follows:
int uv_run(uv_loop_t* loop, uv_run_mode mode) { int timeout; int r; int ran_pending; r = uv__loop_alive(loop); if (!r) uv__update_time(loop); while (r != 0 && loop->stop_flag == 0) { // Update timer time uv__update_time(loop); // Callback timer, setTimeout, setInterval are called back by this function uv__run_timers(loop); // Handling some callbacks that are not completed in uv_io_poll ran_pending = uv__run_pending(loop); // Official explanation: Idle handle is needed only to stop the event loop from blocking in poll. // In fact, some functions in napi, such as napi_call_threadsafe_function, insert callbacks into idle queues and execute at this stage. uv__run_idle(loop); // Callback to process. _start Profiler IdleNotifier uv__run_prepare(loop); timeout = 0; if ((mode == UV_RUN_ONCE && !ran_pending) || mode == UV_RUN_DEFAULT) timeout = uv_backend_timeout(loop); // To calculate the time-out time of uv_io_poll, refer to https://github.com/libuv/libuv/blob/v1.24.0/src/unix/core.c#L318. // The readable monitoring of async_wfd is in the function of uv_io_poll // The second parameter, timeout, is calculated above to set the time for functions such as epoll_wait to wait for I/O events. uv__io_poll(loop, timeout); // setImmediate callback // ps: Personally, I think setImmediate and nextTick should exchange names in terms of implementation: -) uv__run_check(loop); // Closing handles is an asynchronous operation // Usually when the uv loop is terminated, uv_wall will be called to traverse all the handles and close them first, and then execute uv loop once to complete the closure through this function, and finally call uv_loop_close again, otherwise memory leak will occur. uv__run_closing_handles(loop); if (mode == UV_RUN_ONCE) { /* UV_RUN_ONCE implies forward progress: at least one callback must have * been invoked when it returns. uv__io_poll() can return without doing * I/O (meaning: no callbacks) when its timeout expires - which means we * have pending timers that satisfy the forward progress constraint. * * UV_RUN_NOWAIT makes no guarantees about progress so it's omitted from * the check. */ uv__update_time(loop); uv__run_timers(loop); } r = uv__loop_alive(loop); if (mode == UV_RUN_ONCE || mode == UV_RUN_NOWAIT) break; } /* The if statement lets gcc compile it to a conditional store. Avoids * dirtying a cache line. */ if (loop->stop_flag != 0) loop->stop_flag = 0; return r; }
It can be seen that in the uv loop, the timer is updated continuously, various types of callbacks are processed, and I/O events are polled. Node.js asynchronization is accomplished through the uv loop.
The asynchronous use of libuv is Reactor Model multiplexing, in uv__io_poll To handle I/O related events, uv__io_poll Through different platforms epoll,kqueue It can be implemented in different ways. So go ahead async_wfd When writing content, the uv__io_poll Will poll async_wfd Readable event, which is only used to notify the uv loop thread that a callback from a non-uv loop thread needs to be executed in the uv loop thread.
When polling arrives async_wfd After reading, uv__io_poll Callback the corresponding function uv__async_io It mainly does the following two things:
- Read the data and confirm if there is any uv_async_send Call, data content is not concerned.
- ergodic async_handles Handle queue to determine if there is an event and, if so, to execute its callback.
The realization is as follows:
static void uv__async_io(uv_loop_t* loop, uv__io_t* w, unsigned int events) { char buf[1024]; ssize_t r; QUEUE queue; QUEUE* q; uv_async_t* h; assert(w == &loop->async_io_watcher); // This for loop is used to confirm whether there is a uv_async_send call for (;;) { r = read(w->fd, buf, sizeof(buf)); if (r == sizeof(buf)) continue; if (r != -1) break; if (errno == EAGAIN || errno == EWOULDBLOCK) break; if (errno == EINTR) continue; abort(); } // Exchange loop - > async_handle and queue content to avoid inserting new async_handle into the queue when traversing loop - > async_handles // In the loop - > async_handles queue, there are other handles besides the thread pool QUEUE_MOVE(&loop->async_handles, &queue); while (!QUEUE_EMPTY(&queue)) { q = QUEUE_HEAD(&queue); h = QUEUE_DATA(q, uv_async_t, queue); QUEUE_REMOVE(q); // Reinsert uv_async_t into loop - > async_handles. uv_async_t needs to call uv_async_stop manually before it can be removed from the queue. QUEUE_INSERT_TAIL(&loop->async_handles, q); // Confirm whether this async_handle needs a callback if (cmpxchgi(&h->pending, 1, 0) == 0) continue; if (h->async_cb == NULL) continue; // Call the callback function bound to initialize uv_async_t through uv_async_init // The uv_async_t of the thread pool is initialized at uv_loop_init, and its bound callback is uv_work_done // So if h = loop - > wq_async, here H - > async_cb actually calls uv_work_done (h); // Refer to https://github.com/libuv/libuv/blob/v1.24.0/src/unix/loop.c#L88 for details. h->async_cb(h); } }
Calling H - > async_cb from the thread pool will return to the thread pool uv__work_done Function:
void uv__work_done(uv_async_t* handle) { struct uv__work* w; uv_loop_t* loop; QUEUE* q; QUEUE wq; int err; loop = container_of(handle, uv_loop_t, wq_async); uv_mutex_lock(&loop->wq_mutex); // Clear the completed loop - > WQ queue QUEUE_MOVE(&loop->wq, &wq); uv_mutex_unlock(&loop->wq_mutex); while (!QUEUE_EMPTY(&wq)) { q = QUEUE_HEAD(&wq); QUEUE_REMOVE(q); w = container_of(q, struct uv__work, wq); // If the uv_cancel cancel request is called before the callback, an error will still occur even if the request has been executed. err = (w->work == uv__cancelled) ? UV_ECANCELED : 0; w->done(w, err); } }
Finally, call back through W - > done (w, err) uv__fs_done And by uv__fs_done Callback JS function:
static void uv__fs_done(struct uv__work* w, int status) { uv_fs_t* req; req = container_of(w, uv_fs_t, work_req); uv__req_unregister(req->loop, req); // If cancelled, an exception is thrown if (status == UV_ECANCELED) { assert(req->result == 0); req->result = UV_ECANCELED; } // Callback JS req->cb(req); }
That's how libuv is a thread pool from creation to execution of multithreaded requests.
Analysis of fs.access Call Procedure
Back to the code mentioned at the beginning of the article, let's analyze its calling process.
const fs = require('fs') const cb = function (err) { console.log(`Is myfile exists: ${!err}`) } fs.access('myfile', cb)
Assuming that the thread pool size is 2, execution is described below fs.access The state of three threads (omitting Node.js startup and JavaScript and Native function calls) and the timeline from top to bottom:
Blank stands for blocked state, - stands for threads that have not yet started
| uv loop thread | worker thread 1 | worker thread 2 |
|---|---|---|
| [fs.access]fs.access | - | - |
| JavaScript calls Native functions through v8 | - | - |
| uv_fs_access | - | - |
| uv__work_submit | - | - |
| init_threads | worker | worker |
| uv_sem_wait | uv_sem_post | uv_sem_post |
| uv_cond_wait | uv_cond_wait | |
| uv_cond_signal | ||
| uv__io_poll | access | |
| uv__io_poll | ||
| uv__io_poll | uv_async_send | |
| uv__io_poll | uv_cond_wait | |
| uv__io_poll | ||
| uv__async_io | ||
| uv__work_done | ||
| uv__fs_done | ||
| Native calls back JavaScript functions through v8 | ||
| cb | ||
| console.log(`Is myfile exists: ${exists}`) |
You can see that the invocation process is as follows:
- By binding JavaScript functions to Native functions at Node.js startup, fs.access Eventually it goes into the Native function, which calls libuv's uv_fs_access Function to determine whether a file is accessible. (This skips how JavaScript calls the Native function through v8)
- uv_fs_access A multithreaded request is submitted to the thread pool in the uv loop thread.
- Because the thread pool is inert, the operation of initializing the thread pool is performed before executing the request.
- After the thread pool initialization is completed, worker thread 1 is awakened to execute the request, and the uv loop thread continuously polls whether the request has been completed.
- Synchronized calls to worker thread 1 access Function to determine whether the target file is readable.
- access When the function is completed, worker thread 1 passes through uv_async_send The synchronous uv loop thread request has been completed, and at the same time it enters the blocking state itself, waiting for the new request to wake it up.
- The uv loop thread finds that the request has been executed and returns through a series of callbacks uv__fs_done.
- uv__fs_done Callback JavaScript functions print logs. (skipped here) uv__fs_done How to call back to JavaScript via v8)
As there are no new requests coming in, worker thread 2 is always blocked.
Concluding remarks
Through right fs.access By analyzing the calling process, we learned how libuv makes asynchronous calls through thread pools. In addition, you can see that for different platforms, libuv pairs uv__io_poll The implementation is different. We will introduce it later. uv__io_poll How to implement asynchronous I/O.