Previously:

The first two articles mainly introduced the startup process of Zookeeper server under cluster mode and the process of leader election. After the leader election, each node in the cluster has corresponding roles: leader, Follower and Observer. Then, according to the corresponding mode, different services will be started respectively, that is, several service classes mentioned above, as shown below:

This paper mainly introduces the knowledge points of its basic class zookeepserver, and then introduces its subclasses respectively.

1. Properties and construction method of zookeeperserver

/**
 * This class implements a simple standalone ZooKeeperServer. It sets up the
 * following chain of RequestProcessors to process requests:
 * PrepRequestProcessor -> SyncRequestProcessor -> FinalRequestProcessor
 */
public class ZooKeeperServer implements SessionExpirer, ServerStats.Provider {
 
    // Heartbeat time
    public static final int DEFAULT_TICK_TIME = 3000;
    protected int tickTime = DEFAULT_TICK_TIME;
    // session timeout setting
    protected int minSessionTimeout = -1;
    protected int maxSessionTimeout = -1;

    // The session manager has been specially analyzed before
    protected SessionTracker sessionTracker;
    //Transaction log, snapshot log processor
    private FileTxnSnapLog txnLogFactory = null;
    // Memory database
    private ZKDatabase zkDb;
    private final AtomicLong hzxid = new AtomicLong(0);
    public final static Exception ok = new Exception("No prob");
    // Build Processor
    protected RequestProcessor firstProcessor;
    // server status
    protected volatile State state = State.INITIAL;

    protected enum State {
        INITIAL, RUNNING, SHUTDOWN, ERROR;
    }
    
    // Service port connection processor
    private ServerCnxnFactory serverCnxnFactory;
    // Server status information
    private final ServerStats serverStats;
    // shutdown hook function processing
    private ZooKeeperServerShutdownHandler zkShutdownHandler;
    
    public ZooKeeperServer() {
        serverStats = new ServerStats(this);
        listener = new ZooKeeperServerListenerImpl(this);
    }
    // The following parameter information is constructed by default
    public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
            int minSessionTimeout, int maxSessionTimeout,
            DataTreeBuilder treeBuilder, ZKDatabase zkDb) {
        serverStats = new ServerStats(this);
        this.txnLogFactory = txnLogFactory;
        this.txnLogFactory.setServerStats(this.serverStats);
        this.zkDb = zkDb;
        this.tickTime = tickTime;
        this.minSessionTimeout = minSessionTimeout;
        this.maxSessionTimeout = maxSessionTimeout;

        listener = new ZooKeeperServerListenerImpl(this);

        LOG.info("Created server with tickTime " + tickTime
                + " minSessionTimeout " + getMinSessionTimeout()
                + " maxSessionTimeout " + getMaxSessionTimeout()
                + " datadir " + txnLogFactory.getDataDir()
                + " snapdir " + txnLogFactory.getSnapDir());
    }

    public ZooKeeperServer(FileTxnSnapLog txnLogFactory, int tickTime,
            DataTreeBuilder treeBuilder) throws IOException {
        this(txnLogFactory, tickTime, -1, -1, treeBuilder,
                new ZKDatabase(txnLogFactory));
    }
}

In the comments, we can find a lot of information. As a zookeeper server, the process chain when processing requests is: preprequestprocessor - > syncrequestprocessor - > finalrequestprocessor.

2. Analysis on main methods of zookeeper server

2.1 loadData() starts loading node information

public void loadData() throws IOException, InterruptedException {
        if(zkDb.isInitialized()){
            setZxid(zkDb.getDataTreeLastProcessedZxid());
        }
        else {
            // The logic of loading data has been analyzed before, which is mainly implemented in zkDb.loadDataBase()
            setZxid(zkDb.loadDataBase());
        }
        
        // If there are expired sessions, they will be deleted directly
        LinkedList<Long> deadSessions = new LinkedList<Long>();
        for (Long session : zkDb.getSessions()) {
            if (zkDb.getSessionWithTimeOuts().get(session) == null) {
                deadSessions.add(session);
            }
        }
        zkDb.setDataTreeInit(true);
        for (long session : deadSessions) {
            // XXX: Is lastProcessedZxid really the best thing to use?
            killSession(session, zkDb.getDataTreeLastProcessedZxid());
        }
    }

This logic has been analyzed before. The snapshot log information and transaction log information are loaded through the ZKDatabase.loadDataBase() method, and finally the zxid information of the latest transaction operation is returned.

2.2 structure of firstprocessor

The construction of RequestProcessor actually uses a decorator pattern. It starts from the first process and will continue to call nextProcessor until the last one.

protected void setupRequestProcessors() {
    RequestProcessor finalProcessor = new FinalRequestProcessor(this);
    RequestProcessor syncProcessor = new SyncRequestProcessor(this,finalProcessor);
    ((SyncRequestProcessor)syncProcessor).start();
    firstProcessor = new PrepRequestProcessor(this, syncProcessor);
    ((PrepRequestProcessor)firstProcessor).start();
}

Both PrepRequestProcessor and SyncRequestProcessor have a nextProcessor attribute. The whole call chain is as follows: PrepRequestProcessor - > SyncRequestProcessor - > finalrequestprocessor.

2.3 processing client connection requests

When a connection has not been created to the client, the connection request of the client will be processed first. In the previous article on server processing session creation requests, we have analyzed it in detail, and we can go through it roughly here

public void processConnectRequest(ServerCnxn cnxn, ByteBuffer incomingBuffer) throws IOException {
    BinaryInputArchive bia = BinaryInputArchive.getArchive(new ByteBufferInputStream(incomingBuffer));
    ConnectRequest connReq = new ConnectRequest();
    connReq.deserialize(bia, "connect");
    if (LOG.isDebugEnabled()) {
        LOG.debug("Session establishment request from client "
                  + cnxn.getRemoteSocketAddress()
                  + " client's lastZxid is 0x"
                  + Long.toHexString(connReq.getLastZxidSeen()));
    }
    ...
        // If the requested lastZXID is greater than the latest ZXID on the server side, the client request is abnormal    
        if (connReq.getLastZxidSeen() > zkDb.dataTree.lastProcessedZxid) {
            String msg = "Refusing session request for client "
                + cnxn.getRemoteSocketAddress()
                + " as it has seen zxid 0x"
                + Long.toHexString(connReq.getLastZxidSeen())
                + " our last zxid is 0x"
                + Long.toHexString(getZKDatabase().getDataTreeLastProcessedZxid())
                + " client must try another server";

            LOG.info(msg);
            throw new CloseRequestException(msg);
        }
    // Negotiate the session timeout with the server, which should be between minSessionTimeout and maxSessionTimeout
    int sessionTimeout = connReq.getTimeOut();
    byte passwd[] = connReq.getPasswd();
    int minSessionTimeout = getMinSessionTimeout();
    if (sessionTimeout < minSessionTimeout) {
        sessionTimeout = minSessionTimeout;
    }
    int maxSessionTimeout = getMaxSessionTimeout();
    if (sessionTimeout > maxSessionTimeout) {
        sessionTimeout = maxSessionTimeout;
    }
    cnxn.setSessionTimeout(sessionTimeout);
    // We don't want to receive any packets until we are sure that the
    // session is setup
    cnxn.disableRecv();
    long sessionId = connReq.getSessionId();
    // If the client is connected for the first time, the sessionId has not been assigned, and the default value is 0. If it is not 0, it indicates that it has been assigned before
    // However, due to some reason, the server will reopen the corresponding session for this connection
    if (sessionId != 0) {
        long clientSessionId = connReq.getSessionId();
        LOG.info("Client attempting to renew session 0x"
                 + Long.toHexString(clientSessionId)
                 + " at " + cnxn.getRemoteSocketAddress());
        serverCnxnFactory.closeSession(sessionId);
        cnxn.setSessionId(sessionId);
        // Reopen the corresponding session
        reopenSession(cnxn, sessionId, passwd, sessionTimeout);
    } else {
        LOG.info("Client attempting to establish new session at "
                 + cnxn.getRemoteSocketAddress());
        // To connect for the first time, you need to create a Session
        createSession(cnxn, passwd, sessionTimeout);
    }
}

2.4 processing other client requests

It is also explained in the previous blog about processing client requests. Here is a brief introduction

private void submitRequest(ServerCnxn cnxn, long sessionId, int type,
            int xid, ByteBuffer bb, List<Id> authInfo) {
    	// Assemble Request object
        Request si = new Request(cnxn, sessionId, xid, type, bb, authInfo);
        submitRequest(si);
    }
    
    public void submitRequest(Request si) {
        if (firstProcessor == null) {
            // If it has not been initialized, take a rest for 1 second for the arrived request, and then judge whether the processor has been created. If it has not been completed, an error will be reported directly
            synchronized (this) {
                try {
                    while (state == State.INITIAL) {
                        wait(1000);
                    }
                } catch (InterruptedException e) {
                    LOG.warn("Unexpected interruption", e);
                }
                if (firstProcessor == null || state != State.RUNNING) {
                    throw new RuntimeException("Not started");
                }
            }
        }
        try {
            // session processing
            touch(si.cnxn);
            boolean validpacket = Request.isValid(si.type);
            if (validpacket) {
                // It is directly handed over to the PrepRequestProcessor for processing
                firstProcessor.processRequest(si);
                if (si.cnxn != null) {
                    incInProcess();
                }
            } else {
                LOG.warn("Received packet at server of unknown type " + si.type);
                new UnimplementedRequestProcessor().processRequest(si);
            }
        } catch (MissingSessionException e) {
            if (LOG.isDebugEnabled()) {
                LOG.debug("Dropping request: " + e.getMessage());
            }
        } catch (RequestProcessorException e) {
            LOG.error("Unable to process request:" + e.getMessage(), e);
        }
    }

The request processing is mainly handled by the RequestProcessor.

Summary:

There are no special points in this article. The knowledge points are basically covered in previous blogs. It is mainly a summary of zookeeper server to facilitate the subsequent introduction of its subclasses.