When testing http services, you usually use curl to send http requests to the server. Curl will simulate the browser's request and help you establish a tls/ssl connection with the server. Sometimes we want to get the time-consuming in the request process to optimize the client experience. At this time, we usually use the - w parameter of curl.

Statistical indicators of curl

according to curl manual curl allows us to specify a text after the - w parameter. The text can contain parameters in the form of% {MACRO}. curl will replace the actual request time into these macros.

Although there are some articles describing this function, the description of time-consuming is often unclear or even wrong. In order to verify the true meaning of time-consuming, I used the - w parameter and wireshark packet capture to verify it, and sorted out the common time-consuming related statistical values and descriptions as follows:

Each parameter includes the time-consuming of all the previous parameters, which is not easy to understand. The figure above is easy to explain:

So time_connect means the time for TCP to establish a connection. In fact, it includes the time for domain name resolution. Therefore, if you only want to obtain the time-consuming three handshakes of TCP, you should use time_connect - time_namelookup, the following parameters are the same. If you want to get the time-consuming data round-trip transmission and server processing, you need to use time_starttransfer - time_pretransfer.

practice

Let's look at the actual example. First, write a text file containing parameters

\n
time_namelookup: %{time_namelookup}\n
time_connect: %{time_connect}\n
time_appconnect: %{time_appconnect}\n
time_pretransfer: %{time_pretransfer}\n
time_redirect: %{time_redirect}\n
time_starttransfer: %{time_starttransfer}\n
\n

Save the file as curl.txt, use curl simulation request www.baidu.com, and specify the text file with the - w parameter

$ curl http://www.baidu.com/ -w '@curl.txt'
            time_namelookup:  0.010484
               time_connect:  0.023659
            time_appconnect:  0.000000
           time_pretransfer:  0.023833
              time_redirect:  0.000000
         time_starttransfer:  0.039934

The https protocol is not specified, so the time_ The time of appconnect is 0. The TCP connection establishment time = 0.023659 - 0.010484 = 0.013175 seconds. Similarly, the network transmission and server processing time is 0.016101 seconds.
These times help identify performance problems in the client experience and further optimize it.

Richer parameters

Of course, curl's - w supports many other parameters in addition to the performance parameters. If you want to save trouble, you can also directly use the parameter% {json} and it will output all the values supported by the - w parameter in the form of json

$ curl -s http://www.baidu.com/ -w '%{json}' -o /dev/null | jq
{
  "content_type": "text/html",
  "errormsg": null,
  "exitcode": 0,
  "filename_effective": "/dev/null",
  "ftp_entry_path": null,
  "http_code": 200,
  "http_connect": 0,
  "http_version": "1.1",
  "local_ip": "172.16.120.205",
  "local_port": 59399,
  "method": "GET",
  "num_connects": 1,
  "num_headers": 11,
  "num_redirects": 0,
  "proxy_ssl_verify_result": 0,
  "redirect_url": null,
  "referer": null,
  "remote_ip": "180.101.49.11",
  "remote_port": 80,
  "response_code": 200,
  "scheme": "HTTP",
  "size_download": 2381,
  "size_header": 400,
  "size_request": 77,
  "size_upload": 0,
  "speed_download": 52439,
  "speed_upload": 0,
  "ssl_verify_result": 0,
  "time_appconnect": 0,
  "time_connect": 0.026548,
  "time_namelookup": 0.011452,
  "time_pretransfer": 0.026612,
  "time_redirect": 0,
  "time_starttransfer": 0.045227,
  "time_total": 0.045405,
  "url": "http://www.baidu.com/",
  "url_effective": "http://www.baidu.com/",
  "urlnum": 0,
  "curl_version": "libcurl/7.79.1 (SecureTransport) OpenSSL/1.1.1l zlib/1.2.11 brotli/1.0.9 zstd/1.5.0 libidn2/2.3.2 libssh2/1.10.0 nghttp2/1.46.0 librtmp/2.3 OpenLDAP/2.5.7"
}

This time - w does not specify a file, but directly writes the expected output variables in the - w parameter, so there is no need to specify the @ symbol. The jq command is used to format the output json, - o /dev/null is to prevent the returned data from interfering with the output of json statistics.