Hadoop yarn source code analysis AsyncDispatcher event asynchronous distributor 2021SC@SDUSC

Keywords: Big Data Hadoop

2021SC@SDUSC

1, AsyncDispatcher overview

As an asynchronous event scheduler in yarn, AsyncDispatcher is a component of scheduling events based on blocking queue in RM. It dispatches events in a specific single thread and sends the dispatched events to the corresponding EventHandler event processor registered in AsyncDispatcher for processing. A thread pool can be used to schedule time to complete the processing of multiple events.
The processing process is as follows: after the processing request enters the system, it is passed to the corresponding EventHandler by the AsyncDispatcher (asynchronous scheduler). After that, the event processor may forward the event to another event processor or to an event processor containing a finite state machine. The final processing result is transmitted to the AsyncDispatcher in the form of an event. The new event will be processed by AsyncDispatcher and forwarded again until the processing is completed.

In the event driven programming model, all objects are abstracted into event processors, and event processors are related to each other through events. Each event handler handles one type of event and triggers another as needed.

2, AsyncDispatcher property

2.1 AsyncDispatcher member variable

There are also some flag bits in AsyncDispatcher, as follows:
1.stopped: flag bit of whether to stop
2.drainEventsOnStop: enable / disable the configuration flag bit of the run out distributor event in the stop function. If the startup is successful, the AsyncDispatcher needs to process the events in the eventQueue before stopping, otherwise it will stop directly.
3.drained: flag bit indicating that all remaining distributor events in the stop function have been processed or drained
4.waitForDrained: wait lock on the drain flag bit.
5.blockNewEvents: the flag bit that blocks newly arrived events from entering the queue during AsyncDispatcher stop. It is valid only when drainEventsOnStop is enabled (i.e. true).
6.exitOnDispatchException: ensure that the scheduler crashes, but do not make the system exit flag bit.

//package org.apache.hadoop.yarn.event.AsyncDispatcher.java
  //Pending transaction blocking queue
  private final BlockingQueue<Event> eventQueue;
  //AsyncDispatcher flag bit, stop
  private volatile boolean stopped = false;

  //Controls the configuration of detail queue event printing
  private int detailsInterval;
  private boolean printTrigger = false;

  //Enable / disable the configuration flag bit of the run out distributor event in the stop function
  private volatile boolean drainEventsOnStop = false;

  //Flag bit that all remaining distributor events in the stop function have been processed or exhausted
  private volatile boolean drained = true;
  //drained wait lock
  private final Object waitForDrained = new Object();

  // The flag bit that blocks recent events from entering the queue during AsyncDispatcher stop. It is valid only when drainEventsOnStop is enabled (i.e. true)
  private volatile boolean blockNewEvents = false;
  //example
  private final EventHandler<Event> handlerInstance = new GenericEventHandler();

  //Event scheduling thread
  private Thread eventHandlingThread;
  //The event type enumeration class Enum is a collection of mappings to the EventHandler instance of the event handler
  protected final Map<Class<? extends Enum>, EventHandler> eventDispatchers;
  
  //Ensure that the scheduler crashes, but do not do system exit
  private boolean exitOnDispatchException = true;

2.2 AsyncDispatcher constructor

Parameterless Construction: the eventQueue in AsyncDispatcher is a blocking queue. By default, it is implemented as a thread safe chained blocking queue, LinkedBlockingQueue, which is reflected in its parameterless construction method.
Parameterized Construction: initialize the eventDispatchers set to HashMap, which specifically stores the mapping relationship between Enum type events and EventHandler instance of event processor. All events distributed by the scheduler must register an event handler according to eventDispatchers. If not registered, the scheduler will not distribute events.

  //Nonparametric structure
  public AsyncDispatcher() {
    //Call the parameterized construct and pass in the LinkedBlockingQueue instance
    this(new LinkedBlockingQueue<Event>());
  }

  //Parametric structure
  public AsyncDispatcher(BlockingQueue<Event> eventQueue) {
    super("Dispatcher");
    this.eventQueue = eventQueue;
    this.eventDispatchers = new HashMap<Class<? extends Enum>, EventHandler>();
    this.eventTypeMetricsMap = new HashMap<Class<? extends Enum>,
        EventTypeMetrics>();
  }

3, Source code analysis of AsyncDispatcher

Thread start

3.1 serviceInit() method

Take the parameter yarn.dispatcher.exit-on-error. If the parameter is not configured, it defaults to false

  @Override
  protected void serviceInit(Configuration conf) throws Exception{
    super.serviceInit(conf);
    this.detailsInterval = getConfig().getInt(YarnConfiguration.
                    YARN_DISPATCHER_PRINT_EVENTS_INFO_THRESHOLD,
            YarnConfiguration.
                    DEFAULT_YARN_DISPATCHER_PRINT_EVENTS_INFO_THRESHOLD);
  }

3.2 serviceStart() method

Create an event scheduling thread eventHandlingThread
, and start the thread.

  @Override
  protected void serviceStart() throws Exception {
    //start-up
    super.serviceStart();
    eventHandlingThread = new Thread(createThread());
    eventHandlingThread.setName(dispatcherThreadName);
    //Start the event scheduling thread eventHandlingThread
    eventHandlingThread.start();
  }

3.3 run() method

eventHandlingThread is defined by the createThread() method, in which there is a run() method.
The while loop in the run method determines whether the flag bit stopped is false, that is, it runs all the time when the AsyncDispatcher is not stopped and the current thread is not interrupted.
First, assign the flag bit drained to whether the eventQueue is empty. If new events are prevented from joining the queue during the stop process, set the flag bit blockNewsEvents to true. If all the events to be processed have been scheduled, assign the flag bit drained to true, and call the notify() method of waitForDrained to notify the waiting person.
When a normal event event is encountered, use the take() method to take the first object in the blockingQueue. If the blockingQueue is empty, block it, wait until new data is added, and call the dispatch() method to distribute the event.

Runnable createThread() {
    return new Runnable() {
      @Override
      public void run() {
        //If the flag bit stopped is false and the current thread is not interrupted, it will run all the time
        while (!stopped && !Thread.currentThread().isInterrupted()) {
          //Judge whether the event scheduling queue is empty, and assign the value to the flag bit drained
          drained = eventQueue.isEmpty();
          //If new events are prevented from joining the queue to be processed during the stop process, that is, the flag bit blockNewEvents is true
          if (blockNewEvents) {
            synchronized (waitForDrained) {
              if (drained) {
                waitForDrained.notify();
              }
            }
          }
          Event event;
          try {
            //Take an event from the eventQueue
            event = eventQueue.take();
          } catch(InterruptedException ie) {
            if (!stopped) {
              LOG.warn("AsyncDispatcher thread interrupted", ie);
            }
            return;
          }
          if (event != null) {
            if (eventTypeMetricsMap.
                get(event.getType().getDeclaringClass()) != null) {
              long startTime = clock.getTime();
              dispatch(event);
              eventTypeMetricsMap.get(event.getType().getDeclaringClass())
                  .increment(event.getType(),
                      clock.getTime() - startTime);
            } else {
              //Call the dispatch() method to dispatch
              dispatch(event);
            }
            if (printTrigger) {
              //Record the latest scheduled event type that may cause too many events in the queue
              LOG.info("Latest dispatch event type: " + event.getType());
              printTrigger = false;
            }
          }
        }
      }
    };
  }

3.3 dispatch() method

@SuppressWarnings("unchecked")
  protected void dispatch(Event event) {
    //All events pass through this loop
    LOG.debug("Dispatching the event {}.{}", event.getClass().getName(),
        event);
    //Determine the event type enumeration class according to event
    Class<? extends Enum> type = event.getType().getDeclaringClass();

    try{
      //Enumerate the class type according to the event type, and obtain the event handler instance handler from eventDispatchers
      EventHandler handler = eventDispatchers.get(type);
      if(handler != null) {
        handler.handle(event);
      } else {
        throw new Exception("No handler for registered for " + type);
      }
    } catch (Throwable t) {
      //Log the status of the queue
      LOG.error(FATAL, "Error in dispatcher thread", t);
      // If serviceStop is called, the thread exits
      if (exitOnDispatchException
          && (ShutdownHookManager.get().isShutdownInProgress()) == false
          && stopped == false) {
        stopped = true;
        Thread shutDownThread = new Thread(createShutDownThread());
        shutDownThread.setName("AsyncDispatcher ShutDown handler");
        shutDownThread.start();
      }
    }
  }

3.5 serviceStop() method

First judge the flag bit drainEventsOnStop. If it is true, the AsyncDispatcher needs to finish processing the events in the queue eventQueue. First set the flag bit blockNewEvents to true to prevent new events from joining and record the log information of info. Synchronize on waitForDrained through synchronized: if the events in the queue have not been processed and the eventHandlingThread thread is still alive, waitForDrained releases the lock, calls the wait method, waits for 1s, and records the log information of info. Setting the flag bit stopped to true indicates that the AsyncDispatcher service is stopped.
If the eventHandlingThread is not null, interrupt the eventHandlingThread, wait for the end of the eventHandlingThread, and call the serviceStop() method in the parent class.

@Override
  protected void serviceStop() throws Exception {
    if (drainEventsOnStop) {
      blockNewEvents = true;
      LOG.info("AsyncDispatcher is draining to stop, ignoring any new events.");
      long endTime = System.currentTimeMillis() + getConfig()
          .getLong(YarnConfiguration.DISPATCHER_DRAIN_EVENTS_TIMEOUT,
              YarnConfiguration.DEFAULT_DISPATCHER_DRAIN_EVENTS_TIMEOUT);

      synchronized (waitForDrained) {
        while (!isDrained() && eventHandlingThread != null
            && eventHandlingThread.isAlive()
            && System.currentTimeMillis() < endTime) {
          waitForDrained.wait(100);
          LOG.info("Waiting for AsyncDispatcher to drain. Thread state is :" +
              eventHandlingThread.getState());
        }
      }
    }
    stopped = true;
    if (eventHandlingThread != null) {
      eventHandlingThread.interrupt();
      try {
        eventHandlingThread.join();
      } catch (InterruptedException ie) {
        LOG.warn("Interrupted Exception while stopping", ie);
      }
    }

    //stop it
    super.serviceStop();
  }

Posted by cyberrate on Wed, 01 Dec 2021 07:10:26 -0800