EventLoop
In introducing the initialization and start-up process of Bootstrap, we touched NioEventLoopGroup several times. To understand this class, we also need to understand netty's thread model. NioEventLoopGroup can be understood as a set of threads, each of which can handle io events generated by multiple channel s independently.
Initialization of NioEventLoopGroup
Let's look at one of the constructions with more parameters. Other constructions with fewer parameters use some default values. The default parameters are as follows:
- SelectorProvider type, used to create socket channels, udp channels, Selector objects, etc., default value is SelectorProvider.provider(), in most cases, the default value will do. This method ultimately creates a Windows Selector Provider object.
- SelectStrategyFactory, the factory class of the Select policy class, whose default value is DefaultSelectStrategyFactory.INSTANCE, is a SelectStrategyFactory object itself, and the SelectStrategyFactory produces the DefaultSelectStrategy policy class.
- Rejected Execution Handler, a policy class that rejects tasks, decides what strategy to take when the task queue is full, similar to the role of Rejected Execution Handler in the jdk thread pool
Next, let's look at one of the commonly used constructions.
public NioEventLoopGroup(int nThreads, ThreadFactory threadFactory,
final SelectorProvider selectorProvider, final SelectStrategyFactory selectStrategyFactory) {
super(nThreads, threadFactory, selectorProvider, selectStrategyFactory, RejectedExecutionHandlers.reject());
}As you can see, there is no initialization logic in the current class and the construction method of the parent class is directly invoked. So let's look at the construction method of the parent class MultithreadEventLoopGroup:
protected MultithreadEventLoopGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, threadFactory, args);
}Similarly, without task processing, we call the parent class constructor directly, so let's look at the MultithreadEventExecutorGroup constructor, where the initialization logic is implemented.
Multithread Event Executor Group Construction Method
From the analysis of the previous summary, we know that the main logical implementation of NioEventLoop Group's construction method is in the MultithreadEventExecutorGroup class, and a default value of the parameter is added in the process of calling the construction method, namely DefaultEventExecutorChooserFactory.INSTANCE, the default value of the EventExecutorChooserFactory type parameter, which is roundrobin. In this way, threads are selected from multiple threads to register channel.
Summarize the main steps of this code:
- First of all, non-null checks and legitimacy checks of some variables
- Then, according to the number of threads passed in, several sub-executors are created, each corresponding to a thread.
- Finally, a selector is created using the selector factory class with the sub-actuator array as the parameter.
-
Finally, a listener is added to each sub-executor to monitor the termination of the sub-executor and do some bookkeeping so that the current group of executors is terminated after all the sub-executors have terminated.
protected MultithreadEventExecutorGroup(int nThreads, Executor executor,
EventExecutorChooserFactory chooserFactory, Object... args) {
// First, the non-empty inspection of variables and the judgment of legitimacy.
// nThreads has been processed by default values in the construction method of MultthreadEventLoopGroup.
if (nThreads <= 0) {
throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
}// The default values are usually used here. // The function of ThreadPerTaskExecutor is literally, one task, one thread. if (executor == null) { executor = new ThreadPerTaskExecutor(newDefaultThreadFactory()); } // An array of subexecutors, one subexecutor corresponding to one thread children = new EventExecutor[nThreads]; // Create multiple self-executors based on the number of incoming threads // Note that the sub-executor will not run immediately after it is created. for (int i = 0; i < nThreads; i ++) { boolean success = false; try { children[i] = newChild(executor, args); success = true; } catch (Exception e) { // TODO: Think about if this is a good exception type throw new IllegalStateException("failed to create a child event loop", e); } finally { // If the creation of sub-executors is unsuccessful, then all created sub-executors need to be destroyed. if (!success) { for (int j = 0; j < i; j ++) { children[j].shutdownGracefully(); } // Wait so the sub-executor stops and exits for (int j = 0; j < i; j ++) { EventExecutor e = children[j]; try { while (!e.isTerminated()) { e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS); } } catch (InterruptedException interrupted) { // Let the caller handle the interruption. Thread.currentThread().interrupt(); break; } } } } } // Create a selector for a sub-executor whose function is to select one from the sub-executor // The default way to use roundRobin chooser = chooserFactory.newChooser(children); final FutureListener<Object> terminationListener = new FutureListener<Object>() { @Override public void operationComplete(Future<Object> future) throws Exception { if (terminatedChildren.incrementAndGet() == children.length) { terminationFuture.setSuccess(null); } } }; // Add listeners to each subexecutor and do some work when the subexecutor terminates // Add the terminated Children variable to one for each child executor termination // When all subexecutors terminate, the current group of executors terminates for (EventExecutor e: children) { e.terminationFuture().addListener(terminationListener); } // Packing an immutable set Set<EventExecutor> childrenSet = new LinkedHashSet<EventExecutor>(children.length); Collections.addAll(childrenSet, children); readonlyChildren = Collections.unmodifiableSet(childrenSet);}
NioEventLoopGroup.newChild
The above method calls the newChild method to create a subexecutor, which is an abstract method. Let's look at the implementation of the NioEventLoopGroup class:
protected EventLoop newChild(Executor executor, Object... args) throws Exception {
return new NioEventLoop(this, executor, (SelectorProvider) args[0],
((SelectStrategyFactory) args[1]).newSelectStrategy(), (RejectedExecutionHandler) args[2]);
}You can see that it's just a simple creation of a NioEventLoop object.
Summary
At this point, we have analyzed the initial process of NioEventLoop Group. We can't help thinking, since Nio EventLoopGroup is an executor group, which is a group of threads, when did these threads run? If the reader is impressed, it should be remembered that when we analyze the connection process of Bootstrap, channel needs to be registered in EventLoop Group after initialization, which is actually registered on one of the EventLoops. The registration logic is ultimately implemented in the AbstractChannel.AbstractUnsafe.register method, which has a section of code:
if (eventLoop.inEventLoop()) {
register0(promise);
} else {
try {
eventLoop.execute(new Runnable() {
@Override
public void run() {
register0(promise);
}
});
} catch (Throwable t) {
logger.warn(
"Force-closing a channel whose registration task was not accepted by an event loop: {}",
AbstractChannel.this, t);
closeForcibly();
closeFuture.setClosed();
safeSetFailure(promise, t);
}
}First, call EvetLoop. inEventLoop () to determine whether the thread of the executor is the same as the current thread. If so, execute the registered code directly. If not, call EvetLoop. execute to encapsulate the registered logic into a task queue of the executor. Next, we use this method as a starting point to explore the start-up process of the sub-executor thread.
AbstractEventExecutor.inEventLoop
First, let's look at this method, which is used to determine whether the current thread and the thread of the executor are the same thread.
public boolean inEventLoop() {
return inEventLoop(Thread.currentThread());
}SingleThreadEventExecutor.inEventLoop
The code is simple, let's not say much.
public boolean inEventLoop(Thread thread) {
return thread == this.thread;
}
SingleThreadEventExecutor.execute
The method is simple. The core logic is in the startThread method.
public void execute(Runnable task) {
// Non-empty inspection
if (task == null) {
throw new NullPointerException("task");
}
// Executing to this point is usually an external caller.
boolean inEventLoop = inEventLoop();
// Add a task to the task queue
addTask(task);
// If the current thread is not the thread of the executor, you need to check whether the executor thread is running.
// If it's not running, you need to start the thread
if (!inEventLoop) {
startThread();
// Check if the thread is closed
if (isShutdown()) {
boolean reject = false;
try {
// Remove the task you just added
if (removeTask(task)) {
reject = true;
}
} catch (UnsupportedOperationException e) {
// The task queue does not support removal so the best thing we can do is to just move on and
// hope we will be able to pick-up the task before its completely terminated.
// In worst case we will log on termination.
}
if (reject) {
reject();
}
}
}
// AdTaskWakesUp doesn't know what this variable means. NioEventLoop passes in false.
// Add an empty task to the task queue so that blocked execution threads can be waken up
// In some cases, the executor thread will block on taskQueue.
// So adding an element to the blocked queue can wake up threads that are blocked because the queue is empty
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}SingleThreadEventExecutor.startThread
The main function of this method is to maintain the internal state of state. It is thread-safe to modify the state with concurrent cas instructions, and the judgment of the state ensures that the startup logic is executed only once.
private void startThread() {
// Maintenance of state variables
if (state == ST_NOT_STARTED) {
// The Atomic IntegerFieldUpdater class in jdk is used here.
// Using cas instructions of cpu to ensure that state variables can be safely maintained in concurrent situations
// Ensure that only one thread can execute the startup logic and that the startup logic is executed only once
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
boolean success = false;
try {
// Logic for actually starting threads
doStartThread();
success = true;
} finally {
if (!success) {
STATE_UPDATER.compareAndSet(this, ST_STARTED, ST_NOT_STARTED);
}
}
}
}
}SingleThreadEventExecutor.doStartThread
I'm not going to stick the code to this method, but I'll talk about its main function.
- Start a thread with an internal Executor object (typically a ThreadPerTaskExecutor) and perform tasks
- Maintaining the running state of the executor mainly guarantees thread safety through internal state quantities and cas instructions; in addition, maintaining some internal bookkeeping quantities, such as the reference of the thread itself, the start time of the thread, etc.
- At the end of the thread, do some finishing and cleaning work, such as running the remaining tasks, closing the hook, closing the underlying selector (this is the cleaning logic for specific subclasses), and updating the state quantities.
Specific business logic is still implemented in subclasses, that is, the specific implementation of the SingleThreadEventExecutor.run() method.
NioEventLoop.run
Let's still take NioEventLoop as an example to see how it implements the run method. Let me also give a brief account of its main logic:
- The preferred method is a loop that continuously receives io events by calling the selector at the bottom of jdk and processes different io events. It also handles tasks in the task queue, as well as tasks scheduled or delayed.
- Invoke jdk's api, selector receives io events
- Handling various types of io events
- Handling tasks
Here, I will not post code, which is more important to consider and deal with some concurrent situations, such as selector's wake-up time. Next, we mainly look at the processing of various io events. As for the task queue and the task in the scheduling queue, the processing is relatively simple, so we will not start.
NioEventLoop.processSelectedKeysOptimized
This method traverses the selectionKey corresponding to all received io events, and then processes them in turn.
private void processSelectedKeysOptimized() {
// SelectionKey traversing all io events
for (int i = 0; i < selectedKeys.size; ++i) {
final SelectionKey k = selectedKeys.keys[i];
// null out entry in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.keys[i] = null;
final Object a = k.attachment();
if (a instanceof AbstractNioChannel) {
// Handling events
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
// If you need to re-select, set all subsequent selectionKey to 0, and then call the selectNow method again
if (needsToSelectAgain) {
// null out entries in the array to allow to have it GC'ed once the Channel close
// See https://github.com/netty/netty/issues/2363
selectedKeys.reset(i + 1);
selectAgain();
i = -1;
}
}
}NioEventLoop.processSelectedKey
This method first deals with the invalid case of SelectionKey, which can be divided into two cases: the channel itself is invalid; the channel is still normal, but it is cancelled from the current selector, and it may still work in other selectors.
- For the first case, you need to close the channel, which is to close the underlying connection.
- In the second case, there is no need to deal with it.
Next, we will focus on the four kinds of event processing logic.
private void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final AbstractNioChannel.NioUnsafe unsafe = ch.unsafe();
// If selectionKey is invalid, then the corresponding channel is invalid, and the channel needs to be closed at this time.
if (!k.isValid()) {
final EventLoop eventLoop;
try {
eventLoop = ch.eventLoop();
} catch (Throwable ignored) {
// If the channel implementation throws an exception because there is no event loop, we ignore this
// because we are only trying to determine if ch is registered to this event loop and thus has authority
// to close ch.
return;
}
// Only close ch if ch is still registered to this EventLoop. ch could have deregistered from the event loop
// and thus the SelectionKey could be cancelled as part of the deregistration process, but the channel is
// still healthy and should not be closed.
// See https://github.com/netty/netty/issues/5125
// Close only channel s registered on the current EventLoop.
// In theory, a channel can be registered on multiple Eventloop s.
// SelectionKey may be invalid because channel has been cancelled from the current EventLoop.
// But channel itself is still normal and registered in other EventLoop s
if (eventLoop != this || eventLoop == null) {
return;
}
// close the channel if the key is not valid anymore
// Now that channel is invalid, close it.
unsafe.close(unsafe.voidPromise());
return;
}
// Here's how to deal with the normal situation
try {
// Prepared io events
int readyOps = k.readyOps();
// We first need to call finishConnect() before try to trigger a read(...) or write(...) as otherwise
// the NIO JDK channel implementation may throw a NotYetConnectedException.
// Handling connect events
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
// Process OP_WRITE first as we may be able to write some queued buffers and so free memory.
// Handling write events
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
// Handling read and accept events
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
}
} catch (CancelledKeyException ignored) {
unsafe.close(unsafe.voidPromise());
}
}connect event handling
As you can see from the code, the connect ion event is handled by calling NioUnsafe.finishConnect. Let's look at the implementation of AbstractNioUnsafe.finishConnect:
public final void finishConnect() {
// Note this method is invoked by the event loop only if the connection attempt was
// neither cancelled nor timed out.
assert eventLoop().inEventLoop();
try {
// Has the connection been successful?
boolean wasActive = isActive();
// Abstract method with subclass implementation
doFinishConnect();
// Processing a future object and marking it as successful
fulfillConnectPromise(connectPromise, wasActive);
} catch (Throwable t) {
fulfillConnectPromise(connectPromise, annotateConnectException(t, requestedRemoteAddress));
} finally {
// Check for null as the connectTimeoutFuture is only created if a connectTimeoutMillis > 0 is used
// See https://github.com/netty/netty/issues/1770
if (connectTimeoutFuture != null) {
connectTimeoutFuture.cancel(false);
}
connectPromise = null;
}
}It can be seen that the logic of connection is mainly realized by calling doFinishConnect. Specifically, in the subclass, the implementation of NioSocketChannel.doFinishConnect is as follows:
protected void doFinishConnect() throws Exception {
if (!javaChannel().finishConnect()) {
throw new Error();
}
}write event handling
The handling of write events is accomplished by calling the NioUnsafe.forceFlush method. The final implementation is in AbstractChannel.AbstractUnsafe.flush0:
Generally speaking, the logic of this method is relatively simple, but in fact the most complex and core write logic is in the subclass implementation of the doWrite method. Because the focus of this article is to sort out the backbone logic of NioEventLoop, so we will not continue to expand here. Later we will analyze the source code of this block separately. This involves the encapsulation of buffer in netty, which involves some more complex logic.
protected void flush0() {
// If you are writing data, return it directly
if (inFlush0) {
// Avoid re-entrance
return;
}
// Output Buffer
final ChannelOutboundBuffer outboundBuffer = this.outboundBuffer;
if (outboundBuffer == null || outboundBuffer.isEmpty()) {
return;
}
inFlush0 = true;
// Mark all pending write requests as failure if the channel is inactive.
if (!isActive()) {
try {
if (isOpen()) {
outboundBuffer.failFlushed(new NotYetConnectedException(), true);
} else {
// Do not trigger channelWritabilityChanged because the channel is closed already.
outboundBuffer.failFlushed(newClosedChannelException(initialCloseCause), false);
}
} finally {
inFlush0 = false;
}
return;
}
try {
// Write buffer data into channel
doWrite(outboundBuffer);
} catch (Throwable t) {
if (t instanceof IOException && config().isAutoClose()) {
/**
* Just call {@link #close(ChannelPromise, Throwable, boolean)} here which will take care of
* failing all flushed messages and also ensure the actual close of the underlying transport
* will happen before the promises are notified.
*
* This is needed as otherwise {@link #isActive()} , {@link #isOpen()} and {@link #isWritable()}
* may still return {@code true} even if the channel should be closed as result of the exception.
*/
initialCloseCause = t;
close(voidPromise(), t, newClosedChannelException(t), false);
} else {
try {
shutdownOutput(voidPromise(), t);
} catch (Throwable t2) {
initialCloseCause = t;
close(voidPromise(), t2, newClosedChannelException(t), false);
}
}
} finally {
inFlush0 = false;
}
}read event and accept event handling
At first glance, it would be strange why these two events should be dealt with together. They are obviously different events. The main consideration here is the unification of coding, because only NioSocket Channel can have read events, and only NioServer Socket Channel can have accept events, so here through the abstract method, let different subclasses to achieve their own logic, is more unified in the code structure. Let's take a look at the implementation of NioScket Channel. For the implementation of NioServer Socket Channel, I will talk about the startup process of Server Bootstrap in the subsequent analysis of the startup process of netty server.
NioByteUnsafe.read
Summarize the main logic of this approach:
- First, the buffer allocator and the corresponding processor RecvByteBufAllocator.Handle object are retrieved.
- Loop read data, each time allocate a certain size (configurable size) of the buffer, read the data to be read in the channel into the buffer
- Taking the buffer loaded with data as the message body, triggering a read event in the pipeline of channel's processing pipeline, so that the read data can be propagated in the pipeline and processed by each processor.
- Repeat this process until there is no data to read in channel
- Finally, trigger a read-completed event in pipeline
-
At last, we should decide whether to close the channel according to the amount of data we read last time. If the amount of data we read last time is less than 0, it means that the output of the opposite end has been closed, so we need to close the input here, that is, the channel is in a semi-closed state.
public final void read() { final ChannelConfig config = config(); // If the channel is closed, then no data need to be read and returned directly. if (shouldBreakReadReady(config)) { clearReadPending(); return; } final ChannelPipeline pipeline = pipeline(); // buffered distributor final ByteBufAllocator allocator = config.getAllocator(); // Buffer allocation processor, buffer allocation, read count, etc. final RecvByteBufAllocator.Handle allocHandle = recvBufAllocHandle(); allocHandle.reset(config); ByteBuf byteBuf = null; boolean close = false; try { do { // Allocate a buffer byteBuf = allocHandle.allocate(allocator); // Read the data from the channel into the buffer allocHandle.lastBytesRead(doReadBytes(byteBuf)); // If no data is read, there is no data to be read in the channel. if (allocHandle.lastBytesRead() <= 0) { // nothing was read. release the buffer. // Since no data was read, the buffer should be released byteBuf.release(); byteBuf = null; // If the amount of data read is negative, the channel is closed. close = allocHandle.lastBytesRead() < 0; if (close) { // There is nothing left to read as we received an EOF. readPending = false; } break; } // Update bookkeeping in Handle allocHandle.incMessagesRead(1); readPending = false; // Triggering an event to the channel's processor pipeline, // Let the retrieved data be processed by ChannelHandler on the pipeline pipeline.fireChannelRead(byteBuf); byteBuf = null; // Here we judge whether to continue reading according to the following conditions: // The amount of data read last time is greater than 0, and the amount of data read is equal to the maximum capacity of the allocated buffer. // This indicates the data to be read in the channel. } while (allocHandle.continueReading()); // Read completed allocHandle.readComplete(); // Triggering an event that reads completed pipeline.fireChannelReadComplete(); if (close) { closeOnRead(pipeline); } } catch (Throwable t) { handleReadException(pipeline, byteBuf, t, close, allocHandle); } finally { // Check if there is a readPending which was not processed yet. // This could be for two reasons: // * The user called Channel.read() or ChannelHandlerContext.read() in channelRead(...) method // * The user called Channel.read() or ChannelHandlerContext.read() in channelReadComplete(...) method // // See https://github.com/netty/netty/issues/2254 // Here isAutoRead defaults to true, so normally it will continue to listen for read events if (!readPending && !config.isAutoRead()) { removeReadOp(); } } } }
summary
This paper mainly analyses EventLoop's event monitoring and processing logic. In addition to handling io events, it also handles added tasks, scheduled tasks on time and delayed tasks. EventLoop is just like the engine or heart of the whole framework. It calls system calls through jdk api and monitors various io events. It also uses different methods to deal with different categories of io events. For read events, it reads network io data into the buffer and transmits the read data to the user's processor for chain processing. Channel pipeline acts as a pipeline for handling triggered events.
remaining problems
- Writing logic of NioSocketChannel.doWrite method needs further analysis.
- Detailed analysis of Channel Pipeline, how events propagate among processors, design patterns, code structure, etc.
- The analysis of buffer allocators and buffer processors, how they manage memory, is also one of the reasons for netty's high performance.