In the previous article, we explained how nginx establishes a connection and reads data when a request arrives.After reading the data, nginx sets the callback method for reading events to ngx_http_process_request_line(), which has the following main functions:

• Read the data requested by the client, and continue to listen for client read events to read the complete data if client data is not read completely;
• Parse the read client data and store each parameter in the ngx_http_request_t structure that represents the current request.
• Set the callback method of the read event to ngx_http_process_request_headers() to continue processing header data sent by the client.

The point to note here is that the so-called request line refers to the part of the http request message that is similar to GET/index http/1.1. According to the http protocol, the data below this part is the individual header data, and the process of parsing the request line data here does not include how to parse the header data (this part will be explained in the next article).

1. Request Line Processing Main Process

The main process of request line processing is in the ngx_http_process_request_line() method, which has the following source code:

static void ngx_http_process_request_line(ngx_event_t *rev) {
  ssize_t n;
  ngx_int_t rc, rv;
  ngx_str_t host;
  ngx_connection_t *c;
  ngx_http_request_t *r;

  c = rev->data;
  r = c->data;

  if (rev->timedout) {
    ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, "client timed out");
    c->timedout = 1;
    ngx_http_close_request(r, NGX_HTTP_REQUEST_TIME_OUT);
    return;
  }

  rc = NGX_AGAIN;
  for (;;) {
    if (rc == NGX_AGAIN) {
      // The main purpose of the ngx_http_read_request_header() method here is to return the data read on the connection
      n = ngx_http_read_request_header(r);

      // When n is NGX_AGAIN, the current event is added to the event queue and returned directly, whereas NGX_ERROR indicates
      // The current read fails and returns directly
      if (n == NGX_AGAIN || n == NGX_ERROR) {
        return;
      }
    }

    // This is mainly to parse the data of the request row, such as "GET/index HTTP/1.1"
    rc = ngx_http_parse_request_line(r, r->header_in);

    // NGX_OK indicates that the requested row's data is complete and parsed
    if (rc == NGX_OK) {
      r->request_line.len = r->request_end - r->request_start;
      r->request_line.data = r->request_start;
      r->request_length = r->header_in->pos - r->request_start;
      r->method_name.len = r->method_end - r->request_start + 1;
      r->method_name.data = r->request_line.data;

      if (r->http_protocol.data) {
        r->http_protocol.len = r->request_end - r->http_protocol.data;
      }

      // Mark attributes related to parameters
      if (ngx_http_process_request_uri(r) != NGX_OK) {
        return;
      }

      if (r->host_start && r->host_end) {
        host.len = r->host_end - r->host_start;
        host.data = r->host_start;

        // Verify host and convert host to lowercase
        rc = ngx_http_validate_host(&host, r->pool, 0);

        if (rc == NGX_DECLINED) {
          ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
          return;
        }

        if (rc == NGX_ERROR) {
          ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
          return;
        }

        // Set the server and location blocks used to process the current request
        if (ngx_http_set_virtual_server(r, &host) == NGX_ERROR) {
          return;
        }

        r->headers_in.server = host;
      }

      // NGX_HTTP_VERSION_10 indicates that the current request is http 1.0, which is a request to determine if the current request is version 0.9.
      // If the request is version 0.9, then there is no request header
      if (r->http_version < NGX_HTTP_VERSION_10) {
        if (r->headers_in.server.len == 0 
            && ngx_http_set_virtual_server(r, &r->headers_in.server) == NGX_ERROR) {
          return;
        }

        // For version 0.9 requests, process requests directly
        ngx_http_process_request(r);
        return;
      }

      // Initialize headers list
      if (ngx_list_init(&r->headers_in.headers, 
                        r->pool, 20, sizeof(ngx_table_elt_t)) != NGX_OK) {
        ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
      }

      c->log->action = "reading client request headers";

      // Set the event handling method to the ngx_http_process_request_headers() method
      rev->handler = ngx_http_process_request_headers;
      ngx_http_process_request_headers(rev);
      return;
    }

    if (rc != NGX_AGAIN) {
      ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
      return;
    }

    /* NGX_AGAIN: a request line parsing is still incomplete */
    // Going here means that the return value is NGX_AGAIN, that is, the read request line has not been read completely, if there is data available in the buffer,
    // Do nothing else or continue with the next cycle
    if (r->header_in->pos == r->header_in->end) {
      // Request large memory for current request
      rv = ngx_http_alloc_large_header_buffer(r, 1);
      if (rv == NGX_ERROR) {
        ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
        return;
      }

      if (rv == NGX_DECLINED) {
        r->request_line.len = r->header_in->end - r->request_start;
        r->request_line.data = r->request_start;
        ngx_http_finalize_request(r, NGX_HTTP_REQUEST_URI_TOO_LARGE);
        return;
      }
    }
  }
}

For the mainstream processes here, we can see that the first step is to check if the read events are out of date and, if they are, to go back directly.If it does not expire, it enters an infinite for loop, as follows:

• Call the ngx_http_read_request_header() method to read the data, in which nginx checks for readable data on the current connection handle and, if not, continues to add the current event to the event framework.Since the callback method for the current event is the main process method above, that is, ngx_http_process_request_line(), the ngx_http_read_request_header() method that comes here when the read event is triggered again reads the data, which is why when the ngx_http_read_request_header() method returns NGX_AGAIN, the main process can return directly without any processing.Cause;
• Calling the ngx_http_parse_request_line() method to parse the data of the request line is a long method, but the logic is very simple. The main work is to compare each data in the request line one character at a time, and then set it to ngx_http_request_t. We will not explain the alignment in depth here.
• Processing the current request is based on the return values of the first two methods, which are mainly divided into NGX_OK, NGX_AGAIN, NGX_DECLINED, and NGX_ERROR.When the return value is NGX_DECLINED, it indicates that the request line is too long, which will return 414 status codes to the client; when the return value is NGX_ERROR, the current connection will be closed directly and a 500 status code will be returned; while NGX_AGAIN will continue to wait for the next event cycle to trigger;
• When the response code is NGX_OK, the relevant fields in ngx_http_request_t are set, and the validity of the request row data is checked.It is important to note that when this branch is last checked, there will be a situation where, if the current HTTP request version is 0.9, the ngx_http_process_request() is called directly into 11 stages of the HTTP module to process since the 0.9 version does not have a request header.If the current version is more than 1.0, the ngx_http_process_request_headers() method will be called to read and process the request header data since it is required to pass the request header.

2. Request row data read

The main thing to see here is how nginx reads the request row data, as follows is the source code for the ngx_http_read_request_header() method:

static ssize_t ngx_http_read_request_header(ngx_http_request_t *r) {
  ssize_t n;
  ngx_event_t *rev;
  ngx_connection_t *c;
  ngx_http_core_srv_conf_t *cscf;

  c = r->connection;
  rev = c->read;

  // Calculate how much data is currently unprocessed
  n = r->header_in->last - r->header_in->pos;

  // If n is greater than 0, indicating that there is still read data unprocessed, return n directly
  if (n > 0) {
    return n;
  }

  // If you go here, you know that all the data you are reading has been processed, so you can make a judgment here if the read parameter of the current event is 1,
  // Means that unread data is stored on the handle of the current connection, so the c->recv() method is called to read the data, otherwise the current event will continue
  // Add to the event queue and continue listening for read events for the current connection handle
  // The c->recv() method here actually points to the ngx_unix_recv() method, which reads the data on the specified connection
  if (rev->ready) {
    // Read data on connection file descriptors
    n = c->recv(c, r->header_in->last, r->header_in->end - r->header_in->last);
  } else {
    n = NGX_AGAIN;
  }

  // If n is NGX_AGAIN, add the current event to the event listener and continue listening for read events from the current epoll handle
  if (n == NGX_AGAIN) {
    if (!rev->timer_set) {
      cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module);
      ngx_add_timer(rev, cscf->client_header_timeout);
    }

    if (ngx_handle_read_event(rev, 0) != NGX_OK) {
      ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR);
      return NGX_ERROR;
    }

    return NGX_AGAIN;
  }

  // If n is 0, the client closed the connection
  if (n == 0) {
    ngx_log_error(NGX_LOG_INFO, c->log, 0, "client prematurely closed connection");
  }

  // Recycle the current request structure if the client closes the connection or reads an exception
  if (n == 0 || n == NGX_ERROR) {
    c->error = 1;
    c->log->action = "reading client request headers";
    ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST);
    return NGX_ERROR;
  }

  // Update the current read data pointer
  r->header_in->last += n;

  return n;
}

The overall data flow for reading requesting rows here is simple, with the following main processes:

• Check if there is unprocessed data in the current read buffer, and if there is, return it directly so that the following ngx_http_parse_request_line() method can parse the data;
• To check whether the current event is executable, the c->recv() method is called to read data from the connected handle, and the return value n indicates the size of the data read.
• If the current event is not executable, add the current event to the event framework and register the read event for the current connection on the epoll handle.
• In the second step, if the return value of the read data is NGX_ERROR or 0 (0 indicates that the client is disconnected), a 400 response code is returned to the client.

You can see that the processing of request row data here is simply to read if data exists and register the current read event if it does not.

3. Summary

This paper mainly explains how nginx reads request row data and parses it, and highlights how nginx handles it through an event model when the read data is incomplete.