Selector registration

In the previous article, we talked about many aspects of JNI. In this article, we talked about the Java layer. We can understand JNI basically. If you want to dig deep into yourself and learn from Baidu. Let's talk about the important registration process, how to register internally, and how to handle when select returns. Let's talk about windows first.

serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

Let's take serversocketchannel. Register (selector, selectionkey. Op  accept); for example, it is used to listen to the establishment request from the client to see what's going on inside. Finally, the register of java.nio.channels.spi.AbstractSelectableChannel is called:

  public final SelectionKey register(Selector sel, int ops, Object att)
        throws ClosedChannelException
    {
    	
        if ((ops & ~validOps()) != 0)//Validation parameters
            throw new IllegalArgumentException();
        if (!isOpen())//Verify channel is not open
            throw new ClosedChannelException();
        synchronized (regLock) {//Registration lock
            if (isBlocking())//Determine whether non blocking mode
                throw new IllegalBlockingModeException();
            synchronized (keyLock) {//key lock
                // re-check if channel has been closed
                if (!isOpen())//Verify that the channel is not open here
                    throw new ClosedChannelException();
                SelectionKey k = findKey(sel);//Have you registered key
                if (k != null) {//If you have any, please set up interested events and attachments
                    k.attach(att);
                    k.interestOps(ops);
                } else {
                    // New registration otherwise re register a key to the selector
                    k = ((AbstractSelector)sel).register(this, ops, att);//Sign up if you don't
                    addKey(k);//Save the key to cancel the registration when the channel is closed
                }
                return k;
            }
        }
    }

findKey(Selector sel)

Get the same key of the key selector in the current channel as the incoming selector:

    private SelectionKey findKey(Selector sel) {
        assert Thread.holdsLock(keyLock);
        if (keys == null)
            return null;
        for (int i = 0; i < keys.length; i++)
            if ((keys[i] != null) && (keys[i].selector() == sel))//Exists with the same selector
                return keys[i];
        return null;

    }

register(AbstractSelectableChannel ch, int ops, Object attachment)

Selector register key, sun.nio.ch.SelectorImpl register:

 @Override
    protected final SelectionKey register(AbstractSelectableChannel ch,
                                          int ops,
                                          Object attachment)
    {
        if (!(ch instanceof SelChImpl))
            throw new IllegalSelectorException();
        SelectionKeyImpl k = new SelectionKeyImpl((SelChImpl)ch, this);//Create an implementation class
        k.attach(attachment);
        
        implRegister(k);//Register in selector queue

        keys.add(k);//Save key
        try {
            k.interestOps(ops);//Set up events of interest
        } catch (ClosedSelectorException e) {
            assert ch.keyFor(this) == null;
            keys.remove(k);
            k.cancel();
            throw e;
        }
        return k;
    }

interestOps(int ops)

Set the interestOps(int) of the sun.nio.ch.SelectionKeyImpl event of interest of key:

    @Override
    public SelectionKey interestOps(int ops) {
        ensureValid();
        if ((ops & ~channel().validOps()) != 0)
            throw new IllegalArgumentException();
        int oldOps = (int) INTERESTOPS.getAndSet(this, ops);
        if (ops != oldOps) {
            selector.setEventOps(this);//If the event is different from the previous one, it is added to the selector update queue
        }
        return this;
    }

setEventOps(SelectionKeyImpl ski)

setEventOps added to sun.nio.ch.WindowsSelectorImpl in the selector update queue:

    @Override
    public void setEventOps(SelectionKeyImpl ski) {
        ensureOpen();
        synchronized (updateLock) {
            updateKeys.addLast(ski);//Add updated key
        }
    }

The basic registration process is completed. The schematic diagram is roughly as follows:

windows selector

doSelect()

Go directly to the last key place sun.nio.ch.WindowsSelectorImpl's doSelect. Let's look at the following main parts:

 @Override
    protected int doSelect(Consumer<SelectionKey> action, long timeout)
        throws IOException
    {
        ...
        processUpdateQueue();//Update queue according to key
        processDeregisterQueue();//If the key is cancelled, log out of the queue
        ...
        subSelector.poll();
		...
        processDeregisterQueue();//Log off the queue again
        int updated = updateSelectedKeys(action);//Update and process key
		...
        return updated;
    }

processUpdateQueue()

This is to process the previously registered key, add the new key information to the file descriptor array of the underlying layer, and then set the event of interest:

 private void processUpdateQueue() {
        assert Thread.holdsLock(this);

        synchronized (updateLock) {
            SelectionKeyImpl ski;

            // New registrationnew registration key
            while ((ski = newKeys.pollFirst()) != null) {
                if (ski.isValid()) {
                    growIfNeeded();
                    channelArray[totalChannels] = ski;//Add new key
                    ski.setIndex(totalChannels);//Set key index
                    pollWrapper.putEntry(totalChannels, ski);//Set the file descriptor and corresponding events to the file descriptor array
                    totalChannels++;//Channel number +1
                    MapEntry previous = fdMap.put(ski);//Add to map
                    assert previous == null;
                }
            }

            // changes to interest ops
            while ((ski = updateKeys.pollFirst()) != null) {
                int events = ski.translateInterestOps();
                int fd = ski.getFDVal();
                if (ski.isValid() && fdMap.containsKey(fd)) {
                    int index = ski.getIndex();
                    assert index >= 0 && index < totalChannels;
                    pollWrapper.putEventOps(index, events);
                }
            }
        }
    }

processDeregisterQueue()

Delete some key s cancelled due to exceptions:

  protected final void processDeregisterQueue() throws IOException {
        assert Thread.holdsLock(this);
        assert Thread.holdsLock(publicSelectedKeys);

        Set<SelectionKey> cks = cancelledKeys();//Get canceled keys
        synchronized (cks) {
            if (!cks.isEmpty()) {
                Iterator<SelectionKey> i = cks.iterator();
                while (i.hasNext()) {
                    SelectionKeyImpl ski = (SelectionKeyImpl)i.next();
                    i.remove();

                    // remove the key from the selector
                    implDereg(ski);

                    selectedKeys.remove(ski);//Delete the key to be operated
                    keys.remove(ski);//Remove from selector key set

                    // remove from channel's key set
                    deregister(ski);

                    SelectableChannel ch = ski.channel();
                    if (!ch.isOpen() && !ch.isRegistered())
                        ((SelChImpl)ch).kill();//Close channel
                }
            }
        }
    }

subSelector.poll()

JNI's select polling has started. As mentioned in poll0, it's not much to say:

        private int poll() throws IOException{ // poll for the main thread
            return poll0(pollWrapper.pollArrayAddress,
                         Math.min(totalChannels, MAX_SELECTABLE_FDS),
                         readFds, writeFds, exceptFds, timeout);
        }

updateSelectedKeys(action)

This is the processing after returning to see if there are any events:

private int updateSelectedKeys(Consumer<SelectionKey> action) {
        updateCount++;
        int numKeysUpdated = 0;
        numKeysUpdated += subSelector.processSelectedKeys(updateCount, action);
        for (SelectThread t: threads) {//If there are other threads, add them up
            numKeysUpdated += t.subSelector.processSelectedKeys(updateCount, action);
        }
        return numKeysUpdated;
    }

processSelectedKeys(updateCount, action)

As you can see, the events of three sets of read-write exceptions are handled:

   private int processSelectedKeys(long updateCount, Consumer<SelectionKey> action) {
            int numKeysUpdated = 0;
            numKeysUpdated += processFDSet(updateCount, action, readFds,
                                           Net.POLLIN,
                                           false);
            numKeysUpdated += processFDSet(updateCount, action, writeFds,
                                           Net.POLLCONN |
                                           Net.POLLOUT,
                                           false);
            numKeysUpdated += processFDSet(updateCount, action, exceptFds,
                                           Net.POLLIN |
                                           Net.POLLCONN |
                                           Net.POLLOUT,
                                           true);
            return numKeysUpdated;
        }

processFDSet

The main task is to perform the following processing: traverse the set put in, take out the corresponding event key, prepare for processing, and then add the quantity + 1 to return:

 private int processFDSet(long updateCount,
                                 Consumer<SelectionKey> action,
                                 int[] fds, int rOps,
                                 boolean isExceptFds)
        {
            int numKeysUpdated = 0;
            for (int i = 1; i <= fds[0]; i++) {//ergodic
                int desc = fds[i];
                ...
                MapEntry me = fdMap.get(desc);
                ...
                SelectionKeyImpl sk = me.ski;//Remove key
				...
                int updated = processReadyEvents(rOps, sk, action);//Handling preparation events
                if (updated > 0 && me.updateCount != updateCount) {
                    me.updateCount = updateCount;//Update quantity
                    numKeysUpdated++;
                }
            }
            return numKeysUpdated;
        }

processReadyEvents

This is to check whether the triggered event is an event of interest. It will be added to selectedKeys. This is also the set we will traverse later:

protected final int processReadyEvents(int rOps,
                                           SelectionKeyImpl ski,
                                           Consumer<SelectionKey> action) {
        if (action != null) {
            ski.translateAndSetReadyOps(rOps);
            if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
                action.accept(ski);
                ensureOpen();
                return 1;
            }
        } else {
            assert Thread.holdsLock(publicSelectedKeys);
            if (selectedKeys.contains(ski)) {
                if (ski.translateAndUpdateReadyOps(rOps)) {
                    return 1;
                }
            } else {
                ski.translateAndSetReadyOps(rOps);
                if ((ski.nioReadyOps() & ski.nioInterestOps()) != 0) {
                    selectedKeys.add(ski);
                    return 1;
                }
            }
        }
        return 0;
    }

So far, the return of select has been processed.
Then we need to traverse the selectedKeys, and then get the corresponding key to do the corresponding processing.

Linux selector

doSelect()

Take a look at how Linux does this. In EPollSelectorImpl.java:

  @Override
    protected int doSelect(Consumer<SelectionKey> action, long timeout)
        throws IOException
    {
        assert Thread.holdsLock(this);

        // epoll_wait timeout is int
        int to = (int) Math.min(timeout, Integer.MAX_VALUE);
        boolean blocking = (to != 0);
        boolean timedPoll = (to > 0);

        int numEntries;
        processUpdateQueue();
        processDeregisterQueue();
        try {
            begin(blocking);

            do {
                long startTime = timedPoll ? System.nanoTime() : 0;
                numEntries = EPoll.wait(epfd, pollArrayAddress, NUM_EPOLLEVENTS, to);
                if (numEntries == IOStatus.INTERRUPTED && timedPoll) {
                    // timed poll interrupted so need to adjust timeout
                    long adjust = System.nanoTime() - startTime;
                    to -= TimeUnit.MILLISECONDS.convert(adjust, TimeUnit.NANOSECONDS);
                    if (to <= 0) {
                        // timeout expired so no retry
                        numEntries = 0;
                    }
                }
            } while (numEntries == IOStatus.INTERRUPTED);
            assert IOStatus.check(numEntries);

        } finally {
            end(blocking);
        }
        processDeregisterQueue();
        return processEvents(numEntries, action);
    }

In fact, we can see that the process is almost the same, except that EPoll.wait (EPFD, polarrayaddress, Num [epollevents, to) is used; the final processing is processEvents. As I mentioned before, EPoll.wait doesn't need to be verbose. It will put events and corresponding file descriptors into a collection and return them, and handle them directly.

processUpdateQueue

EPoll.ctl is mainly used to operate the red black tree. The nodes can be added, modified, and deleted

  /**
     * Process changes to the interest ops.
     */
    private void processUpdateQueue() {
        assert Thread.holdsLock(this);

        synchronized (updateLock) {
            SelectionKeyImpl ski;
            while ((ski = updateKeys.pollFirst()) != null) {
                if (ski.isValid()) {
                    int fd = ski.getFDVal();
                    // add to fdToKey if needed
                    SelectionKeyImpl previous = fdToKey.putIfAbsent(fd, ski);
                    assert (previous == null) || (previous == ski);

                    int newEvents = ski.translateInterestOps();
                    int registeredEvents = ski.registeredEvents();
                    if (newEvents != registeredEvents) {
                        if (newEvents == 0) {
                            // remove from epoll
                            EPoll.ctl(epfd, EPOLL_CTL_DEL, fd, 0);
                        } else {
                            if (registeredEvents == 0) {
                                // add to epoll
                                EPoll.ctl(epfd, EPOLL_CTL_ADD, fd, newEvents);
                            } else {
                                // modify events
                                EPoll.ctl(epfd, EPOLL_CTL_MOD, fd, newEvents);
                            }
                        }
                        ski.registeredEvents(newEvents);
                    }
                }
            }
        }
    }

processEvents

Event processing does not need to be divided into three sets like windows, only one is needed:

private int processEvents(int numEntries, Consumer<SelectionKey> action)
        throws IOException
    {
        assert Thread.holdsLock(this);

        boolean interrupted = false;
        int numKeysUpdated = 0;
        for (int i=0; i<numEntries; i++) {
            long event = EPoll.getEvent(pollArrayAddress, i);
            int fd = EPoll.getDescriptor(event);
            if (fd == fd0) {
                interrupted = true;
            } else {
                SelectionKeyImpl ski = fdToKey.get(fd);
                if (ski != null) {
                    int rOps = EPoll.getEvents(event);
                    numKeysUpdated += processReadyEvents(rOps, ski, action);
                }
            }
        }

        if (interrupted) {
            clearInterrupt();
        }

        return numKeysUpdated;
    }

summary

This article introduces the process of channel registration, selector selection and return, but many details are not mentioned. These processes can be seen slowly. First, there is a general framework process. Now we know that the registration is actually to generate the SelectionKeyImpl, and then add the file descriptors and interested events of the related channels to the underlying array, and then the underlying layer will take these events to wait for when selecting, and finally return them, and then traverse the collection, add the corresponding time to the selectedKeys, and then the business logic will process them. windows is troublesome to handle, and Linux is much easier to handle.

Well, I'm here today. I hope it will be helpful for learning and understanding. God can't see it. Just for his own learning and understanding, his ability is limited. Please forgive me.