nginx event module implementation details

Keywords: Programming Nginx Attribute socket

In Detailed structure of nginx event module In, we explain the overall workflow of the event module of nginx, and focus on the role of each method of organizing the event module. This paper mainly focuses on the entire process, and explains the implementation details of the event module of nginx from the source point of view.

1. ngx_events_block()--events configuration block resolution

When nginx parses the nginx.conf configuration file, if the currently resolved configuration item is named events and is a configuration block, the ngx_events_block() method is called to resolve the configuration block, as follows:

static char * ngx_events_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) {
  char *rv;
  void ***ctx;
  ngx_uint_t i;
  ngx_conf_t pcf;
  ngx_event_module_t *m;

  // If the configuration item that stores the event module configuration data is not empty, the configuration item has been resolved and returned directly
  if (*(void **) conf) {
    return "is duplicate";
  }

  // The main purpose here is to count the number of event modules and mark the relative order of each event module in the ctx_index property of the module.
  ngx_event_max_module = ngx_count_modules(cf->cycle, NGX_EVENT_MODULE);

  // Create a pointer to store an array of configuration items
  ctx = ngx_pcalloc(cf->pool, sizeof(void *));
  if (ctx == NULL) {
    return NGX_CONF_ERROR;
  }

  // Request Array Memory for Configuration Item Pointer
  *ctx = ngx_pcalloc(cf->pool, ngx_event_max_module * sizeof(void *));
  if (*ctx == NULL) {
    return NGX_CONF_ERROR;
  }

  // Assign array values to conf, that is, associate them to the core configuration object ngx_cycle_t
  *(void **) conf = ctx;

  for (i = 0; cf->cycle->modules[i]; i++) {
    if (cf->cycle->modules[i]->type != NGX_EVENT_MODULE) {
      continue;
    }

    m = cf->cycle->modules[i]->ctx;

    // If the create_conf() method of the current module is not empty, the method is called to create a structure that stores configuration items
    if (m->create_conf) {
      (*ctx)[cf->cycle->modules[i]->ctx_index] = m->create_conf(cf->cycle);
      if ((*ctx)[cf->cycle->modules[i]->ctx_index] == NULL) {
        return NGX_CONF_ERROR;
      }
    }
  }

  // Here the *cf structure is copied, temporarily stored in the pcf, and then the module-related parameters of the current *cf structure are initialized.
  // For further analysis
  pcf = *cf;
  cf->ctx = ctx;
  cf->module_type = NGX_EVENT_MODULE;
  cf->cmd_type = NGX_EVENT_CONF;

  // Resolve sub-configuration items in events{} configuration block
  rv = ngx_conf_parse(cf, NULL);

  // Copy pcf back to *cf for later use
  *cf = pcf;

  if (rv != NGX_CONF_OK) {
    return rv;
  }

  // Here, it means that the configuration items for the events {} configuration block have been resolved, so the init_conf() method for each module is called.
  // Initializing and merging configuration items
  for (i = 0; cf->cycle->modules[i]; i++) {
    if (cf->cycle->modules[i]->type != NGX_EVENT_MODULE) {
      continue;
    }

    m = cf->cycle->modules[i]->ctx;

    // If the init_conf() of the current module is not empty, its init_conf() method is called to initialize the configuration item
    if (m->init_conf) {
      rv = m->init_conf(cf->cycle, (*ctx)[cf->cycle->modules[i]->ctx_index]);
      if (rv != NGX_CONF_OK) {
        return rv;
      }
    }
  }

  return NGX_CONF_OK;
}

The main work done by the ngx_events_block() method is as follows:

  • Call the ngx_count_modules() method to mark the event module sequence number. Note that the order here is to mark the order of the current module in all event type modules and save the sequence number in the ctx_index property of each module, such as the ctx_index of the event type core module ngx_event_core_module here is 0;
  • Request memory space for the pointer ctx, and request an array whose address is assigned to the CTX pointer, where the array length is the number of event modules.The relative position of the current event module in all event modules corresponds to the relative position in the array, which is also the ctx_index calculated in the previous step.
  • Call the create_conf() method of each event module to create its own configuration structure and save it in the array pointed to by the ctx pointer.
  • The ngx_conf_parse() method is called to continue parsing the configuration file. As we mentioned earlier, the ngx_events_block() method is called only when it resolves to the events configuration item, so the call to the ngx_conf_parse() method here is to continue parsing the subconfigurations of the events configuration block, and the completion of the method call means that all the configuration items in the events configuration block have been parsed.
  • Initialize the configuration item by calling the init_conf() method of each module. Simply put, since only part of the configuration item's value is configured in nginx.conf, the remaining configuration items are set by the init_conf() method.

2. ngx_event_init_conf() - - Check that the event module configuration structure is properly created

After nginx has parsed all the configuration items of the nginx.conf configuration file, including the parsing of the events configuration items explained in the previous step, it will invoke the init_conf() method of all core modules to initialize the configuration items of the core modules.The core module here includes the ngx_events_module, where the init_conf() method points to the ngx_event_init_conf() method, which does nothing in essence but checks whether an array of structs storing event module configuration items has been created.The following is the source code for the ngx_event_init_conf() method:

static char *ngx_event_init_conf(ngx_cycle_t *cycle, void *conf) {
  if (ngx_get_conf(cycle->conf_ctx, ngx_events_module) == NULL) {
    ngx_log_error(NGX_LOG_EMERG, cycle->log, 0,
                  "no \"events\" section in configuration");
    return NGX_CONF_ERROR;
  }

  return NGX_CONF_OK;
}

The above two methods are the two main configurations of the ngx_events_module core module, which creates an array to store the configuration structure of each event module.Let's take a look at the main approaches of the Event Core module.

3. ngx_event_core_create_conf() - - Create event core module configuration structure

In point 1, before resolving the subconfigurations of the events configuration block, we call the create_conf() method of each event module to create the structure that stores the configuration data it uses, then call the ngx_conf_parse() method to resolve the subconfigurations, and then call the init_conf() method of each event module to initialize the structure of each module's configuration data.Here ngx_event_core_module_ctx is a module of event type, with the create_conf attribute pointing to the ngx_event_core_create_conf() method and the init_conf attribute pointing to the ngx_event_core_init_conf() method.In this section, we will first explain how the ngx_event_core_create_conf() method works:

static void *ngx_event_core_create_conf(ngx_cycle_t *cycle) {
  ngx_event_conf_t *ecf;

  ecf = ngx_palloc(cycle->pool, sizeof(ngx_event_conf_t));
  if (ecf == NULL) {
    return NULL;
  }

  ecf->connections = NGX_CONF_UNSET_UINT;
  ecf->use = NGX_CONF_UNSET_UINT;
  ecf->multi_accept = NGX_CONF_UNSET;
  ecf->accept_mutex = NGX_CONF_UNSET;
  ecf->accept_mutex_delay = NGX_CONF_UNSET_MSEC;
  ecf->name = (void *) NGX_CONF_UNSET;

  return ecf;
}

You can see that the ngx_event_core_create_conf() method here essentially creates a ngx_event_conf_t structure and sets each property to an unset state.

4. ngx_event_core_init_conf()--Initialize configuration structure

As mentioned earlier, after resolving each subconfiguration item, nginx calls the init_conf() method of each event module, where the core event module is the ngx_event_core_init_conf() method, which is the source code of the method as follows:

static char * ngx_event_core_init_conf(ngx_cycle_t *cycle, void *conf) {
  ngx_event_conf_t *ecf = conf;

#if (NGX_HAVE_EPOLL) && !(NGX_TEST_BUILD_EPOLL)
  int                  fd;
#endif
  ngx_int_t i;
  ngx_module_t *module;
  ngx_event_module_t *event_module;

  module = NULL;

#if (NGX_HAVE_EPOLL) && !(NGX_TEST_BUILD_EPOLL)

  // Test for permission to create epoll handle
  fd = epoll_create(100);

  if (fd != -1) {
    // Close the created epoll handle and point the module to the epoll module
      (void) close(fd);
      module = &ngx_epoll_module;

  } else if (ngx_errno != NGX_ENOSYS) {
      module = &ngx_epoll_module;
  }
#endif

  // Here, if there is no previously determined module type, the first module in the event module is used by default as the event processing model
  if (module == NULL) {
    for (i = 0; cycle->modules[i]; i++) {

      if (cycle->modules[i]->type != NGX_EVENT_MODULE) {
        continue;
      }

      event_module = cycle->modules[i]->ctx;

      if (ngx_strcmp(event_module->name->data, event_core_name.data) == 0) {
        continue;
      }

      module = cycle->modules[i];
      break;
    }
  }

  // If module is still NULL at this time, an exception is returned
  if (module == NULL) {
    ngx_log_error(NGX_LOG_EMERG, cycle->log, 0, "no events module found");
    return NGX_CONF_ERROR;
  }

  // The main action below is to determine if each property is an invalid value for the initial settings and, if so, that it is not configured in nginx.conf
  // About the configuration item for this property, the default value for this property is set here
  ngx_conf_init_uint_value(ecf->connections, DEFAULT_CONNECTIONS);
  cycle->connection_n = ecf->connections;

  ngx_conf_init_uint_value(ecf->use, module->ctx_index);

  event_module = module->ctx;
  ngx_conf_init_ptr_value(ecf->name, event_module->name->data);

  ngx_conf_init_value(ecf->multi_accept, 0);
  ngx_conf_init_value(ecf->accept_mutex, 0);
  ngx_conf_init_msec_value(ecf->accept_mutex_delay, 500);

  return NGX_CONF_OK;
}

The ngx_event_core_init_conf() method does two main things:

  • Select the module currently in use, and if not specified, use the first event module by default;
  • Each property value of the configuration structure that initializes the event core module is the default value.

5. ngx_event_module_init() - Configuration item initialization for core module

For the ngx_event_core_module, it also specifies two methods, one for initializing the module's ngx_event_module_init() method and the other for calling the ngx_event_process_init() method before the worker process executes the main loop logic.The ngx_event_module_init() method is called in the master process and is called after all the configuration items in the nginx.conf file have been resolved. Essentially, the function of this method is to initialize the core module (event module) of the current configuration.The following is the source code for the ngx_event_module_init() method:

/**
 * The main purpose of the current method is to request a piece of shared memory for storing statistics, then set ngx_accept_mutex_ptr,
 * ngx_connection_counter,ngx_temp_number The address of an equal variable, if slab stat is turned on,
 * The addresses of ngx_stat_accepted, ngx_stat_handled, ngx_stat_requests, etc. are also set to count more data
 */
static ngx_int_t ngx_event_module_init(ngx_cycle_t *cycle) {
  void ***cf;
  u_char *shared;
  size_t size, cl;
  ngx_shm_t shm;
  ngx_time_t *tp;
  ngx_core_conf_t *ccf;
  ngx_event_conf_t *ecf;

  // Get the configuration structure of the core event module
  cf = ngx_get_conf(cycle->conf_ctx, ngx_events_module);
  ecf = (*cf)[ngx_event_core_module.ctx_index];

  if (!ngx_test_config && ngx_process <= NGX_PROCESS_MASTER) {
    ngx_log_error(NGX_LOG_NOTICE, cycle->log, 0,
                  "using the \"%s\" event method", ecf->name);
  }

  // Get the configuration object for the core module
  ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module);

  ngx_timer_resolution = ccf->timer_resolution;

#if !(NGX_WIN32)
  {
    ngx_int_t limit;
    struct rlimit rlmt;

    if (getrlimit(RLIMIT_NOFILE, &rlmt) == -1) {
      ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                    "getrlimit(RLIMIT_NOFILE) failed, ignored");

    } else {
      // This is to check if the number of connections configured by the current event module exceeds the maximum number of file handles restricted by the operating system.
      // Or exceeds the maximum number of file handles specified in the configuration file
      if (ecf->connections > (ngx_uint_t) rlmt.rlim_cur
          && (ccf->rlimit_nofile == NGX_CONF_UNSET
              || ecf->connections > (ngx_uint_t) ccf->rlimit_nofile)) {
        limit = (ccf->rlimit_nofile == NGX_CONF_UNSET) ?
                (ngx_int_t) rlmt.rlim_cur : ccf->rlimit_nofile;

        ngx_log_error(NGX_LOG_WARN, cycle->log, 0,
                      "%ui worker_connections exceed "
                      "open file resource limit: %i",
                      ecf->connections, limit);
      }
    }
  }
#endif /* !(NGX_WIN32) */


  if (ccf->master == 0) {
    return NGX_OK;
  }

  if (ngx_accept_mutex_ptr) {
    return NGX_OK;
  }


  /* cl should be equal to or greater than cache line size */

  cl = 128;

  size = cl            /* ngx_accept_mutex */
         + cl          /* ngx_connection_counter */
         + cl;         /* ngx_temp_number */

#if (NGX_STAT_STUB)

  size += cl           /* ngx_stat_accepted */
         + cl          /* ngx_stat_handled */
         + cl          /* ngx_stat_requests */
         + cl          /* ngx_stat_active */
         + cl          /* ngx_stat_reading */
         + cl          /* ngx_stat_writing */
         + cl;         /* ngx_stat_waiting */

#endif

  // Set the size of shared memory
  shm.size = size;
  ngx_str_set(&shm.name, "nginx_shared_zone");
  shm.log = cycle->log;

  // Request memory blocks for shared memory structures
  if (ngx_shm_alloc(&shm) != NGX_OK) {
    return NGX_ERROR;
  }

  // addr is the address of the shared memory block requested
  shared = shm.addr;

  ngx_accept_mutex_ptr = (ngx_atomic_t *) shared;
  ngx_accept_mutex.spin = (ngx_uint_t) -1;

  if (ngx_shmtx_create(&ngx_accept_mutex, (ngx_shmtx_sh_t *) shared, cycle->lock_file.data) != NGX_OK) {
    return NGX_ERROR;
  }

  // Get the address of ngx_connection_counter
  ngx_connection_counter = (ngx_atomic_t *) (shared + 1 * cl);

  // Set the value of ngx_connection_counter to 1
  (void) ngx_atomic_cmp_set(ngx_connection_counter, 0, 1);

  ngx_log_debug2(NGX_LOG_DEBUG_EVENT, cycle->log, 0,
                 "counter: %p, %uA",
                 ngx_connection_counter, *ngx_connection_counter);

  // Get the address of ngx_temp_number
  ngx_temp_number = (ngx_atomic_t *) (shared + 2 * cl);

  tp = ngx_timeofday();

  // Generate a random number
  ngx_random_number = (tp->msec << 16) + ngx_pid;

#if (NGX_STAT_STUB)

  ngx_stat_accepted = (ngx_atomic_t *) (shared + 3 * cl);
  ngx_stat_handled = (ngx_atomic_t *) (shared + 4 * cl);
  ngx_stat_requests = (ngx_atomic_t *) (shared + 5 * cl);
  ngx_stat_active = (ngx_atomic_t *) (shared + 6 * cl);
  ngx_stat_reading = (ngx_atomic_t *) (shared + 7 * cl);
  ngx_stat_writing = (ngx_atomic_t *) (shared + 8 * cl);
  ngx_stat_waiting = (ngx_atomic_t *) (shared + 9 * cl);

#endif

  return NGX_OK;
}

The main work done by the ngx_event_module_init() method is as follows:

  • Gets the configured timer_resolution attribute value and assigns it to the ngx_timer_resolution attribute, which mainly specifies the execution interval of the timed task that updates the time of the nginx cache;
  • Gets the configuration of the file descriptor for nginx configuration and the file descriptor for the current operating system, and compares the two values to update the number of file descriptors that can be opened by the current process.
  • Declare a shared memory block for storing data for each attribute used by nginx for statistics.

6. ngx_event_process_init() - - Initialize worker process

The ngx_event_process_init() method is primarily invoked to initialize the worker process before it executes the main loop, as follows:

static ngx_int_t ngx_event_process_init(ngx_cycle_t *cycle) {
  ngx_uint_t m, i;
  ngx_event_t *rev, *wev;
  ngx_listening_t *ls;
  ngx_connection_t *c, *next, *old;
  ngx_core_conf_t *ccf;
  ngx_event_conf_t *ecf;
  ngx_event_module_t *module;

  // Get Configuration Objects for Core Modules
  ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module);
  // Get the configuration object for the event core module
  ecf = ngx_event_get_conf(cycle->conf_ctx, ngx_event_core_module);

  // Determine if three conditions are currently met, then mark the current way to use shared locks:
  // 1. Currently in master-worker mode;
  // 2. The number of current worker processes is greater than 1;
  // 3. The switch to use the shared lock is currently on;
  if (ccf->master && ccf->worker_processes > 1 && ecf->accept_mutex) {
    ngx_use_accept_mutex = 1;
    ngx_accept_mutex_held = 0;
    ngx_accept_mutex_delay = ecf->accept_mutex_delay;

  } else {
    // If the above conditions are not met, specify that shared locks are not used
    ngx_use_accept_mutex = 0;
  }

#if (NGX_WIN32)

  /*
   * disable accept mutex on win32 as it may cause deadlock if
   * grabbed by a process which can't accept connections
   */

  ngx_use_accept_mutex = 0;

#endif

  // The main function of these two queues here is that each worker process receives client accept events after it acquires a shared lock.
  // It is then placed in the ngx_posted_accept_events queue, events in the queue are processed, and client connections are added to the queue
  // In the ngx_posted_events queue, and then release the lock, that is, the worker process that acquires the lock only needs an accept client connection.
  // The lock is then given permission to other processes, and the read and write events for the received connections are handled on their own.

  // Create the ngx_posted_accept_events queue used to receive connection events from clients
  ngx_queue_init(&ngx_posted_accept_events);
  // Create the ngx_posted_events queue that handles read and write events for client connections
  ngx_queue_init(&ngx_posted_events);

  // Initialize a red-black tree to store events
  if (ngx_event_timer_init(cycle->log) == NGX_ERROR) {
    return NGX_ERROR;
  }

  for (m = 0; cycle->modules[m]; m++) {
    if (cycle->modules[m]->type != NGX_EVENT_MODULE) {
      continue;
    }

    // Ecf->use stores the module number of the selected event model, which is found here
    if (cycle->modules[m]->ctx_index != ecf->use) {
      continue;
    }

    // Module is the module corresponding to the selected event model
    module = cycle->modules[m]->ctx;

    // Initialization method invoked for specified event model
    if (module->actions.init(cycle, ngx_timer_resolution) != NGX_OK) {
      /* fatal */
      exit(2);
    }

    break;
  }

#if !(NGX_WIN32)

  // ngx_timer_resolution represents the time interval between sending update time events
  // This means if ngx_timer_resolution is set and no timer events are set.
  // ngx_event_flags is set in the initialization of the event module, and only the eventport and kqueue models will
  // NGX_USE_TIMER_EVENT is set to ngx_event_flags
  if (ngx_timer_resolution && !(ngx_event_flags & NGX_USE_TIMER_EVENT)) {
    struct sigaction sa;
    struct itimerval itv;

    ngx_memzero(&sa, sizeof(struct sigaction));
    // The sa here is mainly to add the following SIGALRM signal listening events, which are sent to the current process at regular intervals
    // When the current process receives a signal, the following ngx_timer_signal_handler() method will be called, which will
    // ngx_event_timer_alarm is set to 1, and when the current process is in an event loop, determine if
    // ngx_event_timer_alarm is 1, which updates the time data cached by the current process
    sa.sa_handler = ngx_timer_signal_handler;
    sigemptyset(&sa.sa_mask);

    // Add SIGALRM listening signal
    if (sigaction(SIGALRM, &sa, NULL) == -1) {
      ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                    "sigaction(SIGALRM) failed");
      return NGX_ERROR;
    }

    // Set time interval related parameters
    itv.it_interval.tv_sec = ngx_timer_resolution / 1000;
    itv.it_interval.tv_usec = (ngx_timer_resolution % 1000) * 1000;
    itv.it_value.tv_sec = ngx_timer_resolution / 1000;
    itv.it_value.tv_usec = (ngx_timer_resolution % 1000) * 1000;

    // Set timer at specified interval
    if (setitimer(ITIMER_REAL, &itv, NULL) == -1) {
      ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                    "setitimer() failed");
    }
  }

  // NGX_USE_FD_EVENT indicates that the event filter has no transparent data and requires a file descriptor table, which is mainly used for poll, /dev/poll
  if (ngx_event_flags & NGX_USE_FD_EVENT) {
    struct rlimit rlmt;

    if (getrlimit(RLIMIT_NOFILE, &rlmt) == -1) {
      ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                    "getrlimit(RLIMIT_NOFILE) failed");
      return NGX_ERROR;
    }

    // The main thing here is to initialize the maximum number of ngx_connection_t structures and save them in the files array
    cycle->files_n = (ngx_uint_t) rlmt.rlim_cur;

    cycle->files = ngx_calloc(sizeof(ngx_connection_t *) * cycle->files_n, cycle->log);
    if (cycle->files == NULL) {
      return NGX_ERROR;
    }
  }

#else

  if (ngx_timer_resolution && !(ngx_event_flags & NGX_USE_TIMER_EVENT)) {
      ngx_log_error(NGX_LOG_WARN, cycle->log, 0,
                    "the \"timer_resolution\" directive is not supported "
                    "with the configured event method, ignored");
      ngx_timer_resolution = 0;
  }

#endif

  // Request a specified number of ngx_connection_t arrays, where connection_n corresponds to configuration
  // Size specified by worker_connections in file
  cycle->connections = ngx_alloc(sizeof(ngx_connection_t) * cycle->connection_n, cycle->log);
  if (cycle->connections == NULL) {
    return NGX_ERROR;
  }

  c = cycle->connections;

  // Request the specified number of ngx_event_t arrays, which are the same length as the connections array.
  // This allows you to correlate the connections array with the read_events array
  cycle->read_events = ngx_alloc(sizeof(ngx_event_t) * cycle->connection_n, cycle->log);
  if (cycle->read_events == NULL) {
    return NGX_ERROR;
  }

  rev = cycle->read_events;
  for (i = 0; i < cycle->connection_n; i++) {
    rev[i].closed = 1;  // Initial state Default read events are closed state
    rev[i].instance = 1;  // Initialize instance1 initially
  }

  // Request the specified number of ngx_event_t arrays, which are the same length as the connections array.
  // This allows you to correlate the connections array with the write_events array
  cycle->write_events = ngx_alloc(sizeof(ngx_event_t) * cycle->connection_n, cycle->log);
  if (cycle->write_events == NULL) {
    return NGX_ERROR;
  }

  wev = cycle->write_events;
  for (i = 0; i < cycle->connection_n; i++) {
    wev[i].closed = 1;  // Write events initially are also closed by default
  }

  i = cycle->connection_n;
  next = NULL;

  do {
    i--;

    // The elements of the read_events and write_events arrays are assigned successively to the read and write attributes of the elements of the connections array.
    // And assemble the connections array into a single-chain table
    c[i].data = next;
    c[i].read = &cycle->read_events[i];
    c[i].write = &cycle->write_events[i];
    c[i].fd = (ngx_socket_t) -1;

    next = &c[i];
  } while (i);

  // At the initial state, all connections are unused and need to be stored in the free_connections chain table
  cycle->free_connections = next;
  cycle->free_connection_n = cycle->connection_n;

  /* for each listening socket */

  ls = cycle->listening.elts;
  for (i = 0; i < cycle->listening.nelts; i++) {

#if (NGX_HAVE_REUSEPORT)
    if (ls[i].reuseport && ls[i].worker != ngx_worker) {
      continue;
    }
#endif

    // Here is a ngx_connection_t structure bound to each port currently being listened on
    c = ngx_get_connection(ls[i].fd, cycle->log);

    if (c == NULL) {
      return NGX_ERROR;
    }

    c->type = ls[i].type;
    c->log = &ls[i].log;

    c->listening = &ls[i];
    ls[i].connection = c;

    rev = c->read;

    rev->log = c->log;
    // Mark accept as 1 to indicate that client connection events are currently available for reception
    rev->accept = 1;

#if (NGX_HAVE_DEFERRED_ACCEPT)
    rev->deferred_accept = ls[i].deferred_accept;
#endif

    if (!(ngx_event_flags & NGX_USE_IOCP_EVENT)) {
      if (ls[i].previous) {

        /*
         * delete the old accept events that were bound to
         * the old cycle read events array
         */

        // Delete Old Events
        old = ls[i].previous->connection;

        if (ngx_del_event(old->read, NGX_READ_EVENT, NGX_CLOSE_EVENT) == NGX_ERROR) {
          return NGX_ERROR;
        }

        old->fd = (ngx_socket_t) -1;
      }
    }

#if (NGX_WIN32)

    if (ngx_event_flags & NGX_USE_IOCP_EVENT) {
        ngx_iocp_conf_t  *iocpcf;

        rev->handler = ngx_event_acceptex;

        if (ngx_use_accept_mutex) {
            continue;
        }

        if (ngx_add_event(rev, 0, NGX_IOCP_ACCEPT) == NGX_ERROR) {
            return NGX_ERROR;
        }

        ls[i].log.handler = ngx_acceptex_log_error;

        iocpcf = ngx_event_get_conf(cycle->conf_ctx, ngx_iocp_module);
        if (ngx_event_post_acceptex(&ls[i], iocpcf->post_acceptex)
            == NGX_ERROR)
        {
            return NGX_ERROR;
        }

    } else {
        rev->handler = ngx_event_accept;

        if (ngx_use_accept_mutex) {
            continue;
        }

        if (ngx_add_event(rev, NGX_READ_EVENT, 0) == NGX_ERROR) {
            return NGX_ERROR;
        }
    }

#else

    // SOCK_STREAM represents TCP, which is generally TCP, i.e. after receiving an accept event from a client.
    // The ngx_event_accept() method is called to handle the event
    rev->handler = (c->type == SOCK_STREAM) ? ngx_event_accept
                                            : ngx_event_recvmsg;

#if (NGX_HAVE_REUSEPORT)

    // Add current event to event listening queue
    if (ls[i].reuseport) {
      if (ngx_add_event(rev, NGX_READ_EVENT, 0) == NGX_ERROR) {
        return NGX_ERROR;
      }

      continue;
    }

#endif

    if (ngx_use_accept_mutex) {
      continue;
    }

#if (NGX_HAVE_EPOLLEXCLUSIVE)

    if ((ngx_event_flags & NGX_USE_EPOLL_EVENT)
        && ccf->worker_processes > 1)
    {
        if (ngx_add_event(rev, NGX_READ_EVENT, NGX_EXCLUSIVE_EVENT)
            == NGX_ERROR)
        {
            return NGX_ERROR;
        }

        continue;
    }

#endif

    // Add current event to event listening queue
    if (ngx_add_event(rev, NGX_READ_EVENT, 0) == NGX_ERROR) {
      return NGX_ERROR;
    }

#endif

  }

  return NGX_OK;
}

The ngx_event_process_init() method here accomplishes the following main tasks:

  • Configure whether to use the shared lock field ngx_use_accept_mutex based on the process mode used and the number of worker processes;
  • Initialize the ngx_posted_accept_events queue and the ngx_posted_events queue for handling events;
  • Calling the init() method of the currently used event model module, such as the epoll model, will build an epoll handle in the init() method so that events that need to be listened to can be added to it later;
  • Determines whether the ngx_timer_resolution property is configured, which is the execution frequency field of the timed task obtained in the previous section to update the nginx cache time. If configured, a timed task is created to set the ngx_event_timer_alarm property value at a fixed time.
  • Create arrays of connections, read_events, and write_events of the same length, and point the read property of each ngx_connection_t structure in the connections array to the read event structure at the corresponding location in the read_events array, point the write property to the write event structure at the corresponding location in the write_events array, and organize all the connections into a single-chain table and store them in the ngx_cycle_t In the free_connections property of;
  • Configure a ngx_connection_t structure for each port currently listening on by nginx, and add event listeners for it to wait for client connections to arrive.

6. Summary

This paper mainly explains the initialization of the nginx event module, and explains the implementation principle of each method and its role in the whole process in turn in the calling order of each method in the initialization process.

Posted by modplod on Sun, 19 Jan 2020 18:02:36 -0800