Tomcat Source Code Analysis--Tomcat Startup Process

Keywords: Java Tomcat xml Apache

When it comes to starting Tomcat, we all know that we need to run the tomcat/bin/startup.sh script every time, and what exactly is the content of the script? Let's have a look.

Start script

startup.sh script

#!/bin/sh
os400=false
case "`uname`" in
OS400*) os400=true;;
esac

# resolve links - $0 may be a softlink
PRG="$0"

while [ -h "$PRG" ] ; do
  ls=`ls -ld "$PRG"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '/.*' > /dev/null; then
    PRG="$link"
  else
    PRG=`dirname "$PRG"`/"$link"
  fi
done

PRGDIR=`dirname "$PRG"`
EXECUTABLE=catalina.sh

# Check that target executable exists
if $os400; then
  # -x will Only work on the os400 if the files are:
  # 1. owned by the user
  # 2. owned by the PRIMARY group of the user
  # this will not work if the user belongs in secondary groups
  eval
else
  if [ ! -x "$PRGDIR"/"$EXECUTABLE" ]; then
    echo "Cannot find $PRGDIR/$EXECUTABLE"
    echo "The file is absent or does not have execute permission"
    echo "This file is needed to run this program"
    exit 1
  fi
fi

exec "$PRGDIR"/"$EXECUTABLE" start "$@"

Let's take a look at the script. There are two important variables in the script:

  1. PRGDIR: Represents the path of the current script
  2. EXECUTABLE: catalina.sh script name
    One of the most critical lines of code is exec "$PRGDIR"/"$EXECUTABLE" start "$@", which means that the script catalina.sh was executed, and the parameter is start.

catalina.sh script

Then let's look at the implementation in the catalina.sh script:

elif [ "$1" = "start" ] ; then

  if [ ! -z "$CATALINA_PID" ]; then
    if [ -f "$CATALINA_PID" ]; then
      if [ -s "$CATALINA_PID" ]; then
        echo "Existing PID file found during start."
        if [ -r "$CATALINA_PID" ]; then
          PID=`cat "$CATALINA_PID"`
          ps -p $PID >/dev/null 2>&1
          if [ $? -eq 0 ] ; then
            echo "Tomcat appears to still be running with PID $PID. Start aborted."
            echo "If the following process is not a Tomcat process, remove the PID file and try again:"
            ps -f -p $PID
            exit 1
          else
            echo "Removing/clearing stale PID file."
            rm -f "$CATALINA_PID" >/dev/null 2>&1
            if [ $? != 0 ]; then
              if [ -w "$CATALINA_PID" ]; then
                cat /dev/null > "$CATALINA_PID"
              else
                echo "Unable to remove or clear stale PID file. Start aborted."
                exit 1
              fi
            fi
          fi
        else
          echo "Unable to read PID file. Start aborted."
          exit 1
        fi
      else
        rm -f "$CATALINA_PID" >/dev/null 2>&1
        if [ $? != 0 ]; then
          if [ ! -w "$CATALINA_PID" ]; then
            echo "Unable to remove or write to empty PID file. Start aborted."
            exit 1
          fi
        fi
      fi
    fi
  fi

  shift
  touch "$CATALINA_OUT"
  if [ "$1" = "-security" ] ; then
    if [ $have_tty -eq 1 ]; then
      echo "Using Security Manager"
    fi
    shift
    eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      -classpath "\"$CLASSPATH\"" \
      -Djava.security.manager \
      -Djava.security.policy=="\"$CATALINA_BASE/conf/catalina.policy\"" \
      -Dcatalina.base="\"$CATALINA_BASE\"" \
      -Dcatalina.home="\"$CATALINA_HOME\"" \
      -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
      org.apache.catalina.startup.Bootstrap "$@" start \
      >> "$CATALINA_OUT" 2>&1 "&"

  else
    eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
      -classpath "\"$CLASSPATH\"" \
      -Dcatalina.base="\"$CATALINA_BASE\"" \
      -Dcatalina.home="\"$CATALINA_HOME\"" \
      -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
      org.apache.catalina.startup.Bootstrap "$@" start \
      >> "$CATALINA_OUT" 2>&1 "&"

  fi

  if [ ! -z "$CATALINA_PID" ]; then
    echo $! > "$CATALINA_PID"
  fi

  echo "Tomcat started."
The script is very long, but we only care about what we are interested in: if the parameter is start, then the logic here is executed, and the key line is to execute org.apache.catalina.startup.Bootstrap "$@" start, that is, to execute the main method we are familiar with and to carry the start parameter, so let's look at B. How is the main method of ootstrap implemented?

Bootstrap.main

First we start the main method:

public static void main(String args[]) {
    System.err.println("Have fun and Enjoy! cxs");

    // daemon Namely bootstrap
    if (daemon == null) {
        Bootstrap bootstrap = new Bootstrap();
        try {
            //Class loading mechanism, as we mentioned earlier, is not repeated here.
            bootstrap.init();
        } catch (Throwable t) {
            handleThrowable(t);
            t.printStackTrace();
            return;
        }
        daemon = bootstrap;
    } else {
        Thread.currentThread().setContextClassLoader(daemon.catalinaLoader);
    }
    try {
        // command
        String command = "start";
        // If a parameter is entered on the command line
        if (args.length > 0) {
            // command = Last command
            command = args[args.length - 1];
        }
        // If the command is started
        if (command.equals("startd")) {
            args[args.length - 1] = "start";
            daemon.load(args);
            daemon.start();
        }
        // If the command stops
        else if (command.equals("stopd")) {
            args[args.length - 1] = "stop";
            daemon.stop();
        }
        // If the command is started
        else if (command.equals("start")) {
            daemon.setAwait(true);// bootstrap and Catalina Connect in one continuous line, Here settings, Method Internal Settings Catalina Example setAwait Method
            daemon.load(args);// args Empty,Method internal call Catalina Of load Method.
            daemon.start();// identical, Reflective calls Catalina Of start Method ,thus,Startup End
        } else if (command.equals("stop")) {
            daemon.stopServer(args);
        } else if (command.equals("configtest")) {
            daemon.load(args);
            if (null==daemon.getServer()) {
                System.exit(1);
            }
            System.exit(0);
        } else {
            log.warn("Bootstrap: command \"" + command + "\" does not exist.");
        }
    } catch (Throwable t) {
        // Unwrap the Exception for clearer error reporting
        if (t instanceof InvocationTargetException &&
                t.getCause() != null) {
            t = t.getCause();
        }
        handleThrowable(t);
        t.printStackTrace();
        System.exit(1);
    }
}

Let's look at bootstrap.init(); part of the code.

public void init() throws Exception {

    // Class loading mechanism, as we mentioned earlier, is not repeated here.
    initClassLoaders();

    Thread.currentThread().setContextClassLoader(catalinaLoader);
    SecurityClassLoad.securityClassLoad(catalinaLoader);

    // Instantiation of reflection method Catalina
    Class<?> startupClass = catalinaLoader.loadClass("org.apache.catalina.startup.Catalina");
    Object startupInstance = startupClass.getConstructor().newInstance();

   
    String methodName = "setParentClassLoader";
    Class<?> paramTypes[] = new Class[1];
    paramTypes[0] = Class.forName("java.lang.ClassLoader");
    Object paramValues[] = new Object[1];
    paramValues[0] = sharedLoader;
    Method method =
        startupInstance.getClass().getMethod(methodName, paramTypes);
    method.invoke(startupInstance, paramValues);

    // Quote Catalina Example
    catalinaDaemon = startupInstance;
}

We can see that the Catalina class is instantiated by reflection and the instance reference is assigned to Catalina Daemon. Then let's look at daemon.load(args).

private void load(String[] arguments)
    throws Exception {

    // Call the load() method
    String methodName = "load";
    Object param[];
    Class<?> paramTypes[];
    if (arguments==null || arguments.length==0) {
        paramTypes = null;
        param = null;
    } else {
        paramTypes = new Class[1];
        paramTypes[0] = arguments.getClass();
        param = new Object[1];
        param[0] = arguments;
    }
    Method method =
        catalinaDaemon.getClass().getMethod(methodName, paramTypes);
    if (log.isDebugEnabled())
        log.debug("Calling startup class " + method);
    //Call through reflection Catalina Of load()Method
    method.invoke(catalinaDaemon, param);

}

Catalina.load

We can see that daemon.load(args) actually calls Catalina's load() method by reflection. So let's go to the load method of Catalina class and see:

public void load() {

    initDirs();

    // Initialization jmx Environmental variables
    initNaming();

    // Create and execute our Digester
    // Definition Analysis server.xml Configuration, tell Digester Which one? xml What classes should labels be parsed into
    Digester digester = createStartDigester();

    InputSource inputSource = null;
    InputStream inputStream = null;
    File file = null;
    try {

      // First try loading conf/server.xml,Omit part of the code......
      // If it does not exist conf/server.xml,Then load server-embed.xml(this xml stay catalina.jar in),Omit part of the code......
      // If it still can't be loaded xml,It's direct. return,Omit part of the code......

      try {
          inputSource.setByteStream(inputStream);

          // hold Catalina As a top-level example
          digester.push(this);

          // The parsing process instantiates components, such as Server,Container,Connector etc.
          digester.parse(inputSource);
      } catch (SAXParseException spe) {
          // Handling exceptions......
      }
    } finally {
        // Close IO flow......
    }

    // to Server Set up catalina information
    getServer().setCatalina(this);
    getServer().setCatalinaHome(Bootstrap.getCatalinaHomeFile());
    getServer().setCatalinaBase(Bootstrap.getCatalinaBaseFile());

    // Stream redirection
    initStreams();

    // call Lifecycle Of init stage
    try {
        getServer().init();
    } catch (LifecycleException e) {
        // ......
    }

    // ......

}

Server initialization

As you can see, here's a method we're interested in today, getServer.init(), which starts the initialization of Server, and Server is the outermost container in our diagram above. So let's look at this method, which is Lifecycle Base. init (). This method is a template method, but only a definition. The skeleton of an algorithm is implemented by putting some detailed algorithms into subclasses. Let's look at this method:

LifecycleBase.init()

@Override
public final synchronized void init() throws LifecycleException {
    // 1
    if (!state.equals(LifecycleState.NEW)) {
        invalidTransition(Lifecycle.BEFORE_INIT_EVENT);
    }
    // 2
    setStateInternal(LifecycleState.INITIALIZING, null, false);

    try {
        // Template method
        /**
         * The template approach pattern is used to manage the life cycle phases of all components supporting life cycle management.
         * Each component requiring lifecycle management only needs to inherit this base class.
         * Then covering the corresponding hook method can complete the corresponding declaration cycle stage management work.
         */
        initInternal();
    } catch (Throwable t) {
        ExceptionUtils.handleThrowable(t);
        setStateInternal(LifecycleState.FAILED, null, false);
        throw new LifecycleException(
                sm.getString("lifecycleBase.initFail",toString()), t);
    }

    // 3
    setStateInternal(LifecycleState.INITIALIZED, null, false);
}
The implementation class of Server is StandardServer. Let's analyze the StandardServer.initInternal() method. This method is used to initialize the Server. The key point is that the init method is invoked for each service by the cyclic operation of the code to the services.
[Note] Here we only paste this part of the code.
StandardServer.initInternal()
@Override
protected void initInternal() throws LifecycleException {
    super.initInternal();

    // Initialize our defined Services
    for (int i = 0; i < services.length; i++) {
        services[i].init();
    }
}

Call the init method of the Service subcontainer to initialize the Service component. Note that under the same Server, there may be multiple Service components.

Service initialization

Standard Service and Standard Server are both inherited from Lifecycle MBeanBase, so the common initialization logic is the same. Let's look directly at initInternal without much introduction.

StandardService.initInternal()

protected void initInternal() throws LifecycleException {

    // to jmx Register yourself
    super.initInternal();

    // Initialization Engine
    if (engine != null) {
        engine.init();
    }

    // existence Executor Thread pool is initialized, which is not available by default
    for (Executor executor : findExecutors()) {
        if (executor instanceof JmxEnabled) {
            ((JmxEnabled) executor).setDomain(getDomain());
        }
        executor.init();
    }

    mapperListener.init();

    // Initialization Connector,and Connector It will be right again. ProtocolHandler Initialize and open application port monitoring,
    synchronized (connectorsLock) {
        for (Connector connector : connectors) {
            try {
                connector.init();
            } catch (Exception e) {
                // Omit part of the code, logger and throw exception
            }
        }
    }
}
  1. First, register Standard Service in jmx
  2. Initialize Engine, which initializes Realm (privilege-related components)
  3. If an Executor thread pool exists, init operations are also performed, which are tomcat interfaces inherited to java.util.concurrent.Executor, org.apache.catalina.Lifecycle.
  4. Initialize Connector Connector Connector, default http1.1, ajp connector, and this Connector initialization process, will initialize ProtocolHandler, open the application port listening, will be analyzed in detail later.

Engine initialization

The code initialized by Standard Engine is as follows:

@Override
protected void initInternal() throws LifecycleException {
    getRealm();
    super.initInternal();
}

public Realm getRealm() {
    Realm configured = super.getRealm();
    if (configured == null) {
        configured = new NullRealm();
        this.setRealm(configured);
    }
    return configured;
}

Standard Engine inherits to ContainerBase, while ContainerBase rewrites the initInternal() method to initialize the start and stop thread pools, which have the following characteristics:
1. core threads are equal to max and default is 1
2. Allow core threads to exit threads when tasks are not retrieved by timeout
3. The timeout time for a thread to acquire a task is 10 seconds, that is to say, if all threads (including core threads) fail to acquire a task for more than 10 seconds, the thread will be destroyed.

What was the original intention of doing this? Because this thread pool only works when the container starts and stops, it is not necessary to process task queues all the time.

The code for ContainerBase is as follows:

// The default is one thread
private int startStopThreads = 1;
protected ThreadPoolExecutor startStopExecutor;

@Override
protected void initInternal() throws LifecycleException {
    BlockingQueue<Runnable> startStopQueue = new LinkedBlockingQueue<>();
    startStopExecutor = new ThreadPoolExecutor(
            getStartStopThreadsInternal(),
            getStartStopThreadsInternal(), 10, TimeUnit.SECONDS,
            startStopQueue,
            new StartStopThreadFactory(getName() + "-startStop-"));
    // allow core Exit when a thread timeout does not get a task
    startStopExecutor.allowCoreThreadTimeOut(true);
    super.initInternal();
}

private int getStartStopThreadsInternal() {
    int result = getStartStopThreads();

    if (result > 0) {
        return result;
    }
    result = Runtime.getRuntime().availableProcessors() + result;
    if (result < 1) {
        result = 1;
    }
    return result;
}

What's the use of this startStopExecutor thread pool?

  1. When a child container is found at start, the start operation of the child container is processed in the thread pool.
  2. In stop, stop operations are also handled in the thread pool

In the previous article, we introduced the Container component, Standard Engine as the top-level container, its direct sub-container is StardandHost, but the code analysis of Standard Engine, we did not find that it will initialize the sub-container StardandHost. Standard Engine does not play the card according to the routine, but the initial. The transformation process is in the start stage. Personally, I think that Host, Context and Wrapper are related to specific webapp applications, and the initialization process will be more time-consuming. Therefore, in the start stage, initialization and start life cycle are completed by multi-threading. Otherwise, components such as top-level Server, Service and so on need to wait for Host, Context, Wrapper to complete initialization before they can be combined. Beam initialization process, the whole initialization process is transitive

Connector initialization

Connector initialization will be explained later in a special Connector article

summary

So far, the whole initialization process is over. In the whole initialization process, the parent component controls the initialization of the child component, which is passed down layer by layer until all the OK is initialized. The following figure describes the overall delivery process.

By default, Server has only one Service component, which initializes Engine and Connector successively. Engine components do not initialize subcontainers in the initialization phase, but Host, Context and Wrapper containers are initialized in the start phase. By default, tomcat will enable Connector connectors for HTTP 1.1 and AJP, which are handled by default using Http11NioProtocol and AJPNioProtocol.

Posted by Fritz on Fri, 16 Aug 2019 02:45:17 -0700