Nginx learning notes - basic data structure

Keywords: Nginx Unix Programming Python

Some features of Nginx

High performance
 Event driven model can handle massive concurrent connections without blocking
high stability
 Memory pool avoids the common resource leakage problem of c program
 Modular architecture makes all functional modules completely decoupled
 one master / mutil workers process pool is designed to realize self-monitoring management, so as to ensure that a worker can quickly recover services after being hung up
Low resource consumption
 No traditional process or thread server model, no switching cost
 Use a lot of programming skills to save system resources
High scalability
 Modular architecture, which can develop modules suitable for your own business logic according to your needs

Nginx custom integer type
Cross platform compatibility

// Defined in core / NGX ﹤ config. H
typedef intptr_t    ngx_int_t;      // Signed integer
typedef uintptr_t   ngx_uint_t;     // Unsigned integer
typedef intptr_t    ngx_flag_t;     // Flag integer type
  • IntPtr? And uintptr? Are integer types in the C/C + + standard that are large enough to hold pointers
// Define the key type in the core/ngx_rbtree.h red black tree
typedef ngx_uint_t  ngx_rbtree_key_t;
typedef ngx_int_t   ngx_rbtree_key_int_t;

// Define the integer type in OS / Unix / NGX ﹣ time. H milliseconds
typedef ngx_rbtree_key_t    ngx_msec_t;
typedef ngx_rbtree_key_int_t    ngx_msec_int_t;

Invalid value:
In Nginx, a python like None is defined, and the initialization variable, UNSET = -1, means uninitialized
Because C/C + + is a strongly typed language, Nginx defines macros with different types of conversions for - 1

//Defined in core / NGX? Conf? File. H
 #Define NGX? Conf? Unset - 1 / / Universal invalid value
 #Define NGX? Conf? Unset? Uint (NGX? Uint) - 1 / / invalid value of an unsigned integer
 #Define NGX? Conf? Unset? PTR (void *) - 1 / / invalid value of pointer type
...

With the concept of UNSET, Nginx provides initialization and conditional assignment functions in the form of macros:

// Defined in core / NGX? Conf? File. H
#define ngx_conf_init_value(conf, default)      \
    if(conf == NGX_CONF_UNSET)                  \
    {                                           \
        conf = default;                         \
    }

When conf is not initialized, it is initialized to default

Exception mechanism error handling
Separate the normal part and abnormal part in the code logic to make the structure of the code clearer

Nginx uses macros to define seven common error codes, of which the type is NGX ﹣ int ﹣ t

//Defined in core/ngx_core.h
 #Define ngx_ok 0 / / execution succeeded, no error
 #Define ngx_error - 1 / / execution failed, the most common error
 #Define ngx_again - 2 / / not ready, need to retry
 #Define NGX? Busy - 3 / / backend service is busy
 #Define ngx_done - 4 / / execution succeeded, but subsequent operations are required
 #Define ngx_declined - 5 / / execution succeeded, but no processing was done
 #Define ngx_abort - 6 / / a critical error has occurred

Memory pool
It can reduce the number of system calls and avoid memory fragmentation and leakage

// Defined in NGX · u core. H
typedef struct ngx_pool_s   ngx_pool_t;     // Simplified definition

// Defined in NGX ﹣ palloc. H
struct ngx_pool_s
{
    ...
    ngx_pool_cleanup_t  *cleanup;           // Clearing action during deconstruction
    ngx_log_t           *log;               // Associated log objects
}

Nginx will create an independent memory pool for each TCP/HTTP request -- that is, NGX pool object
Automatically destroy the NGX pool object when the request ends, freeing the memory pool and all the memory it owns

// Memory alignment is used, which is fast, but there may be a small amount of memory waste
void * ngx_palloc(ngx_pool_t  * pool, size_t size);
// Memory alignment not used
void * ngx_pnalloc(ngx_pool_t * pool, size_t size);
// Internal call to NGX Φ palloc() and clear the memory block
void * ngx_pcalloc(ngx_pool_t * pool, size_t size);
// Free memory
ngx_int_t ngx_free(ngx_pool_t * pool, void * p);

Clean up mechanism:
The Nginx framework automatically manages the life cycle of the memory pool. When the request ends, the memory in the memory pool will be completely returned to the system;
Memory is only one aspect of system resources. Other system resources (such as file handles) will not be released along with the destruction of memory pool;
If you do not do special operations, you may cause resource leakage.

This cleaning mechanism is the idea of destructors in C + +. Destructors are called automatically when objects are destroyed

Nginx defines a structure for saving cleanup information, which is used to perform cleanup actions when memory is destroyed

// Defined in NGX palloc.h
typedef void (*ngx_pool_cleanup_t) void * data;     // Clean up function pointer prototype

struct ngx_pool_cleanup_s                           // Clean up information structure
{
    ngx_pool_cleanup_pt     handler;                // Cleanup action, function pointer
    void                    *data;                  // Clean up required data
    ngx_pool_cleanup_t      *next;                  // Follow up list pointer
};

// Defined in NGX ﹣ palloc. H
ngx_pool_cleanup_pt * ngx_pool_cleanup_add(ngx_pool_t * p, size_t size);

This function uses size to allocate memory for NGX ﹐ pool ﹐ cleanup ﹐ T:: data, and returns the cleanup information NGX ﹐ pool ﹐ cleanup ﹐ object,
By setting its handler and data, you can register the cleanup function with the memory pool

std::vector has two template parameters:

template<class T,                               // Type of element contained 
        class Alloctor = std::allocator<T>      // Memory Configurator
        > class vector

We can customize the memory configurator to replace the second parameter, and vector can use the memory pool of Nginx

Character string
NGX? STR? T is not a traditional string, but rather a memory block reference

// Defined in ngx_string.h
typedef struct
{
    size_t      len;    // String length
    u_char *    data;   // Address of string
}ngx_str_t;

The advantage of this design is that the string operation is very cheap, with only two integers,
It does not need to copy a large amount of data, so it is very efficient to copy and modify it, and also saves the use of memory
(similar to boost::string_ref or std::string_view C++17)

The disadvantages are obvious. Since NGX ﹣ str ﹣ t only refers to memory, it should be used as read-only as possible
Multiple NGX? STR? T share a block of memory. Unauthorized modification will affect other references
At the same time, the referenced memory address may be invalid, and the wrong memory area may be accessed

Initialization and assignment:

#define ngx_string(str)     {sizeof(str) - 1, (u_char *) str}
#define ngx_null_string     {0, NULL}

#define ngx_str_set(str, text)  \
...
#define ngx_str_null(str)       \
...

Basic operation:

#define ngx_strcmp(s1, s2)      // Case sensitive comparison, the parameter is u u char*
#define ngx_strncmp(s1, s2, n)  // Case sensitive comparison with length parameter

#define ngx_strstr(s1, s2)      // Find substring
#define ngx_strlen(s)           // Use '\ 0' to calculate string length

// Case insensitive string comparison, parameter is u ﹣ char*
ngx_int_t ngx_strcasecmp(u_char * s1, u_char * s2);
ngx_int_t ngx_strncasecmp(u_char * s1, u_char * s2, size_t n);

// String to integer type, parameter is u char*
ngx_int_t ngx_atoi(u_char * line, size_t n);

// Memory pool copy string, parameter is NGX ﹣ str ﹣ t*
u_char * ngx_pstrdup(ngx_pool_t * pool, ngx_str_t * src);

Format function:

// Output formatting directly to buf without checking the validity of the buffer
u_char * ngx_sprintf(u_char * buf, const char * fmt, ...);
// The parameters max and last indicate the end of the buffer
u_char * ngx_snprintf(u_char * buf, size_t max, const char * fmt, ...);
u_char * ngx_slprintf(u_char * buf, u_char * last, const char * fmt, ...);

After the function is executed, the U ﹐ char * pointer is returned, indicating the end position in buf after formatting the output, which can be used to determine the length of the result

Time and date
Nginx defines a special time data structure

// Defined in core / NGX ﹣ times. H
typedef struct
{
    time_t      sec;        // Seconds since epoch, i.e. time stamp
    ngx_uint_t  msec;       // The fraction of a second, in milliseconds
    ngx_int_t   gmtoff;     // GMT time zone offset
} ngx_time_t;

Nginx Used internally cache Mechanism to store time values, using a global pointer ngx_cached_time Indicates the time of the current cache

volatile ngx_time_t * ngx_cached_time;      // Current cached time

// Get the seconds of the current time
#define ngx_time()      ngx_cached_time->sec
// Get the complete time structure
#define timeofday()     (ngx_time_t *)ngx_cached_time

// Force update cache time (lock required, high cost, called when current exact time must be obtained)
void ngx_time_update(void)

Date structure is tm in standard C structure < CTime >

// Defined in os/unix/ngx time.h
typedef struct tm   ngx_tm_t;

Date operation function

// Defined in core / NGX ﹣ times. H
void ngx_gmtime()time_t t, ngx_tm_t * tp);
void ngx_localtime(time_t t, ngx_tm_t * tp);

Convert time? To Greenwich mean time / local time

u_char * ngx_http_time(u_char * buf, time_t t);
u_char * ngx_http_cookie_time(u_char * buf, time_t t);

The above two functions call NGX ﹐ gmtime() and NGX ﹐ sprintf(), and convert time ﹐ t to date string

// Defined in core / NGX? Parse? Time. H
time_t ngx_parse_http_time(u_char * value, size_t len);

Parse the date time in the form of string, and convert it to time

Meanwhile, Nginx uses global variables to provide cached date strings, reducing the cost of frequent calls:

ngx_str_t ngx_cached_err_log_time;      // Date string for error log
ngx_str_t ngx_cached_http_time;         // Date string in HTTP format
ngx_str_t ngx_cached_http_log_time;     // Date string for HTTP log
ngx_str_t ngx_cached_http_log_iso8601;  // Date string in ISO8601 format
ngx_str_t ngx_cached_syslog_time;       // System log format date string

Operation log
Nginx uses the structure NGX ﹣ log ﹣ t to represent the operation log

// Defined in core/ngx_log.h
struct ngx_log_s
{
    ...
    ngx_uint_t      log_level;      // log level
    ngx_log_t *     next;           // Log object list pointer
};

// Defined in core/ngx_log.h
void ngx_log_error_core(ngx_uint_t level, ngx_log_t * log, ngx_err_t err,
                        const char * fmt, ...);

The format syntax of string message is the same as that of NGX ﹣ sprintf(), which uses the NGX ﹣ log ﹣ object to record the level level log

The log level parameter level depends on the following macros, which correspond to the
debug | info | notice | warn | error | crit | alert | emerg

#define NGX_LOG_STDERR      0   // highest level
#define NGX_LOG_EMERG       1
#define NGX_LOG_ALERT       2
#define NGX_LOG_CRIT        3
#define NGX_LOG_ERR         4   // Common grade
#define NGX_LOG_WARN        5
#define NGX_LOG_NOTICE      6
#define NGX_LOG_INFO        7
#define NGX_LOG_DEBUG       8   // Lowest level

STDERR is a higher error level than emerg. If you use this level to log,
Then Nginx will output the log directly to the standard error output (usually the terminal screen) instead of writing to the log file.

The commonly used log levels are NGX? Log? Err and NGX? Log? Warn

The err parameter indicates the error code returned from the call failure

// Defined in os/unix/errno.h
typedef int     ngx_err_t;

Log macro

#define ngx_log_error(level, log, ...)      \
...

Only when the log level of the message is higher than the log object level (that is, the level value of the message is small), the function will be called to log

  • Macro should be used to record logs in actual development*

C + + encapsulation implementation: https://github.com/chen892704/Nginx-Learning

Published 1 original article, praised 6, visited 1870
Private letter follow

Posted by ahmedkl on Tue, 21 Jan 2020 22:27:41 -0800