Working principle and configuration of nginx
Module and working principle of nginx
Nginx consists of kernel and modules. Among them, the design of the kernel is very small and concise, and the work completed is also very simple. Only by looking up the configuration file, the client request is mapped to a location block (location is an instruction in nginx configuration for URL matching), and each instruction configured in this location will start different modules to complete the corresponding work.
Module classification of nginx
The module of nginx is divided into core module, basic module and third-party module
- HTTP module, EVENT module and MAIL module are core modules
- HTTP Access module, HTTP FastCGI module, HTTP Proxy module and HTTP Rewrite module are basic modules
- HTTP Upstream module, Request Hash module, Notice module and HTTP Access Key module belong to third-party modules
Modules developed by users according to their own needs belong to third-party modules. It is with the support of so many modules that the function of nginx will be so powerful
nginx modules are divided into three types in terms of function:
- Handlers. This kind of module directly processes the request, outputs the content and modifies the header information. Generally, there can only be one handler module
- Filters. This kind of module is mainly used to modify the output of other processor modules, and finally output by nginx
- Proxies (agent module). It is the HTTP Upstream module of nginx. These modules mainly interact with some back-end services, such as fastcgi, to realize the functions of service proxy and load balancing
nginx module is divided into: core module, event module, standard Http module, optional Http module, mail module, third-party module and patch
- Nginx basic modules: the so-called basic modules refer to the default function modules of nginx. The instructions provided by them allow you to use variables that define the basic functions of nginx. They cannot be disabled during compilation, including:
- Core module: basic functions and instructions, such as process management and security. Most of the common core module instructions are placed at the top of the configuration file
- Event module: the ability to configure network usage in Nginx. Most of the common events module instructions are placed at the top of the configuration file
- Configuration module: provides an inclusion mechanism
How nginx works
Nginx modules are directly compiled into nginx, so they belong to static compilation.
After starting nginx, the module of nginx is loaded automatically. Unlike Apache, first compile the module into a so file, and then specify whether to load it in the configuration file.
When parsing the configuration file, each module of nginx may process a request, but the same processing request can only be completed by one module.
Process architecture of nginx:
When nginx is started, a Master process will be started. This process does not process any client requests. It is mainly used to generate worker threads. A worker thread is used to process n requests.
Picture 1
The following figure shows a routine HTTP request and response process of nginx module
Picture 2
The following figure shows the basic WEB service request steps
Picture 3
Configuration of nginx
nginx configuration
Service control mode, using nginx command
Configure environment variables
[root@localhost ~]# echo 'export PATH=/usr/local/nginx/sbin:$PATH' > /etc/profile.d/nginx.sh [root@localhost ~]# . /etc/profile.d/nginx.sh
Service control mode, using nginx command
-t check the configuration file syntax
[root@localhost ~]# nginx -t nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful
-v output the version of nginx
[root@localhost ~]# nginx -v nginx version: nginx/1.20.1
-c specify the path of the configuration file
[root@localhost ~]# nginx -s stop ; nginx -c /opt/nginx.conf ##Directly stop and start [root@localhost ~]# ps -ef|grep nginx root 99290 1 0 03:32 ? 00:00:00 nginx: master process nginx -c /opt/nginx.conf nginx 99291 99290 0 03:32 ? 00:00:00 nginx: worker process nginx 99292 99290 0 03:32 ? 00:00:00 nginx: worker process root 101138 1653 0 03:33 pts/0 00:00:00 grep --color=auto nginx
-s sends a service control signal. The optional values are {stop | quit | reopen}
[root@localhost ~]# nginx -s quit [root@localhost ~]# ss -antl State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 [::]:22 [::]:*
Detailed explanation of nginx configuration file
Main configuration file: / usr/local/nginx/conf
- When nginx is started by default, the configuration file used is the installation path / conf/nginx.conf file
- You can specify the configuration file to read with the - c option when you start nginx
Common configuration files of nginx and their functions
| configuration file | effect |
|---|---|
| nginx.conf | Basic configuration file of nginx |
| mime.types | Extension files associated with MIME types |
| fastcgi.conf | fastcgi related configurations |
| proxy.conf | proxy related configuration |
| sites.conf | Configure the websites provided by nginx, including virtual hosts |
Detailed explanation of nginx.conf configuration
The content of nginx.conf is divided into the following paragraphs:
- Mainconfiguration segment: global configuration segment. The main configuration section may contain the event configuration section
- Event {}: define the working characteristics of the event model
- http {}: defines the configuration related to the http protocol
Sample Nginx configuration file
[root@localhost ~]# /usr/local/nginx/conf
# Global block
user www-data; ##user
worker_processes 2; ## The default is 1. It is generally recommended to set the number of CPU cores to 1-2 times
error_log logs/error.log; ## Error log path
pid logs/nginx.pid; ## Process id
# Events block
events {
# Use epoll's I/O model to handle polling events.
# It can not be set. nginx will select the appropriate model according to the operating system
use epoll;
# The maximum number of connections for the worker process. 1024 by default
worker_connections 2048;
# Keep alive timeout at http level
keepalive_timeout 60;
# Buffer size of the client request header
client_header_buffer_size 2k;
}
# http block
http {
include mime.types; # Import file extension and file type mapping table
default_type application/octet-stream; # Default file type
# Log format and access log path
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
# sendfile mode is allowed to transfer files. The default value is off.
sendfile on;
tcp_nopush on; # Only when sendfile is on.
# http server block
# Simple reverse proxy
server {
listen 80;
server_name domain2.com www.domain2.com;
access_log logs/domain2.access.log main;
# Forward dynamic request to web application server
location / {
proxy_pass http://127.0.0.1:8000;
deny 192.24.40.8; # Rejected ip
allow 192.24.40.6; # Allowed ip
}
# Error page
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
# load balancing
upstream backend_server {
server 192.168.0.1:8000 weight=5; # The higher the weight, the greater the weight
server 192.168.0.2:8000 weight=1;
server 192.168.0.3:8000;
server 192.168.0.4:8001 backup; # Hot standby
}
server {
listen 80;
server_name big.server.com;
access_log logs/big.server.access.log main;
charset utf-8;
client_max_body_size 10M; # Limit the file size uploaded by the user. The default is 1M
location / {
# Using proxy_pass forwards the request to a set of application servers defined through upstream
proxy_pass http://backend_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
}
}
}
Configuration instruction: to end with a semicolon, the syntax format is as follows:
derective value1 [value2 ...];
Supported variables:
- Built in variable: the module will provide built-in variable definitions
- Custom variable: set var_name value
Configuration parameters for debugging and locating problems
daemon {on|off}; //Whether to run nginx as a daemon. It should be set to off during debugging
master_process {on|off}; //Whether to run nginx in the master/worker model. It can be set to off during debugging
error_log Location level; //Configure error log
error_ The position and level in the log can have the following options:
| position | level |
|---|---|
| file stderr syslog:server=address[,parameter=value] memory:size | Debug: to use the debug level, you need to use the – with debug option when compiling nginx info notice warn error crit alert emerg |
For example, cancel the comment on the log in the configuration file, and then modify the log level. The lower the log level, the more detailed, but the more useless records, so we should choose the appropriate log level. For example, we set the crit level, but we can't see any errors, so we can reduce the level to view more detailed records
[root@localhost ~]# ls /usr/local/nginx/logs/ ##Check whether there is an error log file first nginx.pid [root@localhost ~]# vim /usr/local/nginx/conf/nginx.conf ##Edit the configuration file and open the error log #user nobody; worker_processes 1; error_log logs/error.log; ##Open error log #error_log logs/error.log notice; #error_log logs/error.log info; #pid logs/nginx.pid; ·····Omitted part [root@localhost ~]# nginx -s reload ##Reload ##A log file has been found [root@localhost ~]# ls /usr/local/nginx/logs/ error.log nginx.pid
Browser access to generate an error log
Check the error log again
[root@localhost ~]# cat /usr/local/nginx/logs/error.log ##The record inside is the error information of the page accessed above 2021/10/26 07:43:03 [error] 36057#0: *3 open() "/usr/local/nginx/html/test" failed (2: No such file or directory), client: 192.168.240.1, server: localhost, request: "GET /test HTTP/1.1", host: "192.168.240.60"
Configuration parameters necessary for normal operation
When using, directly indicate in the configuration file that the user is nginx
The pid file location is also uncommented and used directly, because it is also used by default. Of course, it can also be modified to other locations
[root@localhost conf]# vim /usr/local/nginx/conf/nginx.conf 1 2 user nginx nginx; # Uncomment, modify to nginx user, nginx group 3 worker_processes 1; ·····Omitted part········ ·····Omitted part········ 7 #error_log logs/error.log info; 8 9 pid logs/nginx.pid; # note off ·····Omitted part········ [root@localhost conf]# nginx -s reload # Reloading the configuration file takes effect immediately
Set the maximum number of files that the worker thread can open
worker_rlimit_nofile number; # Change the limit on the maximum number of open files in the worker thread. Used to increase the limit without restarting the main process. The default is 1024, but 1024 is too small, so we can set it to 10240
[root@localhost ~]# cd /usr/local/nginx/conf/ [root@localhost conf]# vim nginx.conf 1 2 user nginx nginx; 3 worker_processes 1; 4 worker_rlimit_nofile 10240; # #Add this line and set it to 10240. The default is 1024 ·····Omitted part········ [root@localhost conf]# nginx -s reload # #Reloading the configuration file takes effect immediately
Configuration parameters for performance optimization
worker_processes n;
Starting the number of worker threads and switching processes is context switching. The running position is recorded (register). The context switching information will be saved in the CPU memory until it is used again.
[root@localhost ~]# cd /usr/local/nginx/conf/ [root@localhost conf]# vim nginx.conf 1 2 user nginx nginx; 3 worker_processes 3; # The default value is 1, and changes can generate multiple worker threads to process 4 worker_rlimit_nofile 10240; ·····Omitted part········ [root@localhost conf]# nginx -s reload # Reloading the configuration file takes effect immediately
Bind the process to a cpu core to avoid frequent cache refresh. The binding rule is that the number of cores is - 1. For example, if the total number of cores is 4, three cores will be bound and the other one will be used by others
worker_cpu_affinity cpumask ...;
#cpumask: use 8-bit binary to represent the cpu core, such as:
0000 0001 / / the first cpu core
0000 0010 / / second cpu core
0000 0100 / / the third cpu core
0000 1000 / / the fourth cpu core
0001 0000 / / the fifth cpu core
0010 0000 / / the sixth cpu core
0100 0000 / / the seventh cpu core
10 million / / the eighth cpu core
[root@localhost ~]# cd /usr/local/nginx/conf/ [root@localhost conf]# vim nginx.conf user nginx nginx; worker_processes 3; worker_rlimit_nofile 10240; worker_cpu_affinity 0001 0010 0100; # Add this line to bind three cores for the three worker threads added earlier ·····Omitted part········ [root@localhost ~]# /usr/local/nginx/sbin/nginx -s stop;nginx # Restart nginx ##Check whether it is effective: top command, search nginx in uppercase L, then press lowercase f, move to uppercase P, then select the space, and finally press q to exit to see whether it is effective
Timer resolution. Lowering this value reduces the number of gettimeofday() system calls. Reduce the timer resolution in the worker thread, thereby reducing the number of system calls. By default, it is called every time a kernel event is received. As the resolution decreases, one call is made for each specified call.
timer_resolution interval; (for example, it can be set to timer_resolution 100ms;)
Indicates the nice value of the worker process. The priority is - 20 to 19. The smaller the specified number is, the higher the priority is. The relative priority (adjustable), the absolute priority, and 100 to 139 (unadjusted). You can increase the priority first
worker_priority number; [root@localhost ~]# ps -elf|grep nginx # It is found that the nice value of the worker thread is 80. We can modify and increase its priority so that it can be processed first 1 S root 34284 1 0 80 0 - 20408 - 01:35 ? 00:00:00 nginx: master process nginx 5 S nginx 34285 34284 0 80 0 - 28560 do_epo 01:35 ? 00:00:00 nginx: worker process 5 S nginx 34286 34284 0 80 0 - 28560 do_epo 01:35 ? 00:00:00 nginx: worker process 5 S nginx 34287 34284 0 80 0 - 28560 do_epo 01:35 ? 00:00:00 nginx: worker process 0 S root 99205 33695 0 80 0 - 2302 - 01:53 pts/0 00:00:00 grep --color=auto nginx [root@localhost ~]# cd /usr/local/nginx/conf/ [root@localhost conf]# vim nginx.conf 1 2 user nginx nginx; 3 worker_processes 3; 4 worker_rlimit_nofile 10240; 5 worker_cpu_affinity 0001 0010 0100; 6 worker_priority -20 # Add this line to increase the priority of the worker thread [root@localhost conf]# nginx -s reload # Send a signal to reload the configuration file with immediate effect [root@localhost ~]# ps -elf|grep nginx # It is found that the nice value of worker thread is increased to 60 0 T root 78891 1460 0 80 0 - 11875 - 08:04 pts/0 00:00:00 vim /usr/local/nginx/conf/nginx.conf 1 S root 88430 1 0 80 0 - 20408 - 08:09 ? 00:00:00 nginx: master process nginx 5 S nginx 88431 88430 0 60 0 - 28556 do_epo 08:09 ? 00:00:00 nginx: worker process 5 S nginx 88432 88430 0 60 0 - 28556 do_epo 08:09 ? 00:00:00 nginx: worker process 5 S nginx 88433 88430 0 60 0 - 28556 do_epo 08:09 ? 00:00:00 nginx: worker process 0 S root 96763 1460 0 80 0 - 3087 - 08:13 pts/0 00:00:00 grep --color=auto nginx
Event related configuration: parameter configuration in the event {} section
accept_mutex {off|on}; Mutually exclusive locks, mutually exclusive, load balancing locks used by mster when scheduling user requests to each worker process; On means that multiple workers can respond to new requests in turn and serially, and keep closed by default
accept_mutex {off|on};
lock_file logs/nginx.lock; # accept_mutex file path used by mutex (main section)
Indicate the event model used, and it is recommended that nginx choose by itself. Specify the connection processing to use. There is usually no need to specify, because nginx uses the most effective method by default
use [epoll | rtsig | select | poll];
The maximum number of connections each process can accept
worker_connections 1024;
[root@localhost conf]# vim nginx.conf
·····Omitted part········
events {
worker_connections 20480; # The default value is 1024
}
·····Omitted part········
# worker_connections 20480； Number of concurrent connections = 20480 * number of worker threads (3) / 2
# Use ab command to test concurrency and install tools * * Yum - y install httpd tools**
[root@localhost ~]# ab -n 30000 http://192.168.240.60/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 192.168.240.60 (be patient)
Completed 3000 requests
Completed 6000 requests
Completed 9000 requests
Completed 12000 requests
Completed 15000 requests
Completed 18000 requests
Completed 21000 requests
Completed 24000 requests
Completed 27000 requests
Completed 30000 requests
Finished 30000 requests
Server Software: nginx/1.20.1
Server Hostname: 192.168.240.60
Server Port: 80
Document Path: /
Document Length: 612 bytes
Concurrency Level: 1
Time taken for tests: 4.177 seconds
Complete requests: 30000
Failed requests: 0
Total transferred: 25350000 bytes
HTML transferred: 18360000 bytes
Requests per second: 7182.64 [#/sec] (mean)
Time per request: 0.139 [ms] (mean)
Time per request: 0.139 [ms] (mean, across all concurrent requests)
Transfer rate: 5927.08 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 1
Processing: 0 0 0.1 0 3
Waiting: 0 0 0.1 0 3
Total: 0 0 0.1 0 3
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 0
99% 1
100% 3 (longest request)
Network connection related configuration parameters
Timeout length of long connection. The default value is 65s
[root@localhost conf]# vim /usr/local/nginx/conf/nginx.conf
·····Omitted part········
#keepalive_timeout 0;
keepalive_timeout 65; # Default value
#gzip on;
·····Omitted part········
Set the maximum number of requests that can be served through a live connection. After the maximum number of requests, the connection will be closed.
Closing connections periodically is a necessary condition for memory allocation for each connection. Therefore, using an excessive maximum request may lead to excessive memory usage, which is not recommended
keepalive_requests number; (it can be added in http section, server section and location section)
[root@localhost]# cd /usr/local/nginx/conf
[root@localhost conf]# vim nginx.conf
·····Omitted part········
server {
listen 80;
server_name localhost;
keepalived_requests 10000 # Can be added in the server section
#charset koi8-r;
·····Omitted part········
Parameters that often need to be adjusted
worker_processes # worker threads
worker_connections # concurrent connections
worker_cpu_affinity # cpu core number binding
worker_priority # worker thread priority
Configuration used when nginx is used as a web server: configuration parameters of http {} section
Http {...}: configure HTTP related, by ngx_http_core_module module. The HTTP configuration of nginx mainly includes four blocks, and the structure is as follows:
http {//Protocol level
include mime.types;
default_type application/octet-stream;
keepalive_timeout 65;
gzip on;
upstream {//Load balancing configuration
...
}
server {//At the server level, each server is similar to a < virtualhost > in httpd
listen 80;
server_name localhost;
location / {//Request level, similar to < location > in httpd, is used to define the mapping relationship between URL and local file system
root html;
index index.html index.htm;
}
}
}
http {} segment configuration instruction:
server {}: define a virtual host, for example:
server{
listen 81；(port)
server_name www.lp.com;(Domain names can be given to multiple users)
location / {
root html/test; (Site location) or alias /var/www/html/;(alias,The alias must write an absolute path, that is, put the resource in another location. When someone accesses it, the path they see is/Under the root, in fact, we have put it in another position to prevent being attacked)
index index.html；
}
listen: specify the listening address and port
listen address[:port]; listen port;
server_name NAME […]; It can be followed by multiple hosts, and the name can use regular expressions or wildcards
When there are multiple server s, the matching order is as follows:
1. Check the exact matching first
2. Wildcard matching check on the left, such as *. idfsoft.com
3. Wildcard matching check on the right, such as mail*
4. Regular expression matching check, such as ~ ^. *. idfsoft.com$
5.default_server
root path; Set resource path mapping to indicate the starting path on the file system where the resource corresponding to the requested URL is located
alias path; Used to configure the location section and define the path alias
index file; Default master page
index index.php index.html;
error_page code [...] [= code] URI | @name indicates the special error page according to the http response status code, such as error_page 404 /404_customed.html
[= code]: respond with the specified response code instead of the default original response. By default, it means that the response code of the new resource is its response code, such as error_page 404 =200 /404_customed.html
log_format defines the log format
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
The location section, which matches the URI requested by the client by specifying a pattern
Function: it is allowed to match each defined location according to the URI requested by the user. When matching, the request will be processed by the configuration in the corresponding location configuration block, such as access control
Syntax: location [modifier] pattern {...}
e indicate the special error page according to the http response status code, such as error_page 404 /404_customed.html
[= code]: respond with the specified response code instead of the default original response. By default, it means that the response code of the new resource is its response code, such as error_page 404 =200 /404_customed.html
log_format defines the log format
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
The location section, which matches the URI requested by the client by specifying a pattern
Function: it is allowed to match each defined location according to the URI requested by the user. When matching, the request will be processed by the configuration in the corresponding location configuration block, such as access control
Syntax: location [modifier] pattern {...}