Author: roast chicken Prince

Source: Hang Seng LIGHT cloud community

brief introduction

This year's company developer conference is in the form of online. According to the Convention, in order to ensure the service, it is necessary to evaluate the performance of the whole system temporarily. The more commonly used tools are jmeter and Apache Bench. Finally, Apache Bench (AB for short) was selected between them, and some conclusions were made for ab tool.

AB introduction

Open the official website to see the following paragraph:

ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression 



of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation 

is capable of serving.

Apache bench is a web stress testing tool provided with Apache server, abbreviated as ab. AB is also a line tool with low local requirements for initiating load. According to AB, many concurrent access threads can be created to simulate multiple visitors accessing a URL address at the same time. Therefore, it can be used to test the load pressure of the target server. In general, AB tool is small and simple, and can learn quickly. It can provide the required basic performance indicators, but it has no graphical results and cannot be monitored.

Comparison of jmeter and ab

There are a lot of introductions on the Internet. I collected and found the summary of a great God (I won't say much about it, you can baidu by yourself)

1. JMeter is a complete request and return, while AB only sends the request and does not process the return, but the request is sent successfully or failed. Therefore, in terms of accuracy, JMeter is more accurate, while AB is faster, which can generate more access requests with the least machine resources;

2. Jmeter itself supports assertions, variable parameters and input of CSV data sets, and can set more flexible test scenarios, while AB does not support it (I didn't expect it at the moment);

3. Jmeter can provide more detailed statistical result data, such as interface error information, single thread request time, etc., while AB does not support it;

4. Jmeter does not support accurate time pressure measurement, such as pressure measurement for 10 minutes, but AB supports it;

5. Jmeter supports distributed pressure measurement clusters and functions, but AB does not;

6. The software itself consumes resources: Jmeter takes more time and resources than AB because it is heavy and counts a lot of result data. AB is ultra lightweight and is very suitable for single interface pressure testing in the process of development and testing.

Because this test is only conducted for a single interface, and there is just a free linux machine in hand, AB is selected after comprehensive consideration. There is no more nonsense. Here are some explanations on the use of ab.

Use of AB

The official website introduces the use of ab in detail. We can check the official website address:

https://httpd.apache.org/docs...

Here are some excerpts. English is relatively simple, so there is no translation.

ab parameter

ab [ -A auth-username:password ] [ -b windowsize ] [ -B local-address ] [ -c concurrency ] [ -C cookie-name=value ] [ -d ] 

[ -e csv-file ] [ -E client-certificate file ] [ -f protocol ] [ -g gnuplot-file ] [ -h ] [ -H custom-header ] [ -i ] [ -k ]

[ -l ] [ -m HTTP-method ] [ -n requests ] [ -p POST-file ] [ -P proxy-auth-username:password ] [ -q ] [ -r ] [ -s timeout ] 

[ -S ] [ -t timelimit ] [ -T content-type ] [ -u PUT-file ] [ -v verbosity] [ -V ] [ -w ] [ -x <table>-attributes ] 

[ -X proxy[:port] ] [ -y <tr>-attributes ] [ -z <td>-attributes ] [ -Z ciphersuite ] [http[s]://]hostname[:port]/path

Parameter description

• -A auth-username:password

Supply BASIC Authentication credentials to the server. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the server needs it (i.e., has sent an 401 authentication needed).

• -b windowsize

Size of TCP send/receive buffer, in bytes.

• -B local-address

Address to bind to when making outgoing connections.

• -c concurrency

Number of multiple requests to perform at a time. Default is one request at a time.

• -C cookie-name=value

Add a Cookie: line to the request. The argument is typically in the form of a name=value pair. This field is repeatable.

• -d

Do not display the "percentage served within XX [ms] table". (legacy support).

• -e csv-file

Write a Comma separated value (CSV) file which contains for each percentage (from 1% to 100%) the time (in milliseconds) it took to serve that percentage of the requests. This is usually more useful than the 'gnuplot' file; as the results are already 'binned'.

• -E client-certificate-file

When connecting to an SSL website, use the provided client certificate in PEM format to authenticate with the server. The file is expected to contain the client certificate, followed by intermediate certificates, followed by the private key. Available in 2.4.36 and later.

• -f protocol

Specify SSL/TLS protocol (SSL2, SSL3, TLS1, TLS1.1, TLS1.2, or ALL). TLS1.1 and TLS1.2 support available in 2.4.4 and later.

• -g gnuplot-file

Write all measured values out as a 'gnuplot' or TSV (Tab separate values) file. This file can easily be imported into packages like Gnuplot, IDL, Mathematica, Igor or even Excel. The labels are on the first line of the file.

• -h

Display usage information.

• -H custom-header

Append extra headers to the request. The argument is typically in the form of a valid header line, containing a colon-separated field-value pair (i.e., "Accept-Encoding: zip/zop;8bit").

• -i

Do HEAD requests instead of GET.

• -k

Enable the HTTP KeepAlive feature, i.e., perform multiple requests within one HTTP session. Default is no KeepAlive.

• -l

Do not report errors if the length of the responses is not constant. This can be useful for dynamic pages. Available in 2.4.7 and later.

• -m HTTP-method

Custom HTTP method for the requests. Available in 2.4.10 and later.

• -n requests

Number of requests to perform for the benchmarking session. The default is to just perform a single request which usually leads to non-representative benchmarking results.

• -p POST-file

File containing data to POST. Remember to also set -T.

• -P proxy-auth-username:password

Supply BASIC Authentication credentials to a proxy en-route. The username and password are separated by a single : and sent on the wire base64 encoded. The string is sent regardless of whether the proxy needs it (i.e., has sent an 407 proxy authentication needed).

• -q

When processing more than 150 requests, ab outputs a progress count on stderr every 10% or 100 requests or so. The -q flag will suppress these messages.

• -r

Don't exit on socket receive errors.

• -s timeout

Maximum number of seconds to wait before the socket times out. Default is 30 seconds. Available in 2.4.4 and later.

• -S

Do not display the median and standard deviation values, nor display the warning/error messages when the average and median are more than one or two times the standard deviation apart. And default to the min/avg/max values. (legacy support).

• -t timelimit

Maximum number of seconds to spend for benchmarking. This implies a -n 50000 internally. Use this to benchmark the server within a fixed total amount of time. Per default there is no timelimit.

• -T content-type

Content-type header to use for POST/PUT data, eg. application/x-www-form-urlencoded. Default is text/plain.

• -u PUT-file

File containing data to PUT. Remember to also set -T.

• -v verbosity

Set verbosity level - 4 and above prints information on headers, 3 and above prints response codes (404, 200, etc.), 2 and above prints warnings and info.

• -V

Display version number and exit.

• -w

Print out results in HTML tables. Default table is two columns wide, with a white background.

• -x -attributes

String to use as attributes for table. Attributes are inserted table here.

• -X proxy[:port]

Use a proxy server for the requests.

• -y -attributes

String to use as attributes for tr.

• -z -attributes

String to use as attributes for tb

• -Z ciphersuite

Specify SSL/TLS cipher suite (See openssl ciphers)

Parameter description

Server Software

The value, if any, returned in the server HTTP header of the first successful response. This includes all characters in the header from beginning to the point a character with decimal value of 32 (most notably: a space or CR/LF) is detected.

Server Hostname

The DNS or IP address given on the command line

Server Port

The port to which ab is connecting. If no port is given on the command line, this will default to 80 for http and 443 for https.

SSL/TLS Protocol

The protocol parameters negotiated between the client and server. This will only be printed if SSL is used.

Document Path

The request URI parsed from the command line string.

Document Length

This is the size in bytes of the first successfully returned document. If the document length changes during testing, the response is considered an error.

Concurrency Level

The number of concurrent clients used during the test

Time taken for tests

This is the time taken from the moment the first socket connection is created to the moment the last response is received

Complete requests

The number of successful responses received

Failed requests

The number of requests that were considered a failure. If the number is greater than zero, another line will be printed showing the number of requests that failed due to connecting, reading, incorrect content length, or exceptions.

Write errors

The number of errors that failed during write (broken pipe).

Non-2xx responses

The number of responses that were not in the 200 series of response codes. If all responses were 200, this field is not printed.

Keep-Alive requests

The number of connections that resulted in Keep-Alive requests

Total body sent

If configured to send data as part of the test, this is the total number of bytes sent during the tests. This field is omitted if the test did not include a body to send.

Total transferred

The total number of bytes received from the server. This number is essentially the number of bytes sent over the wire.

HTML transferred

The total number of document bytes received from the server. This number excludes bytes received in HTTP headers

Requests per second

This is the number of requests per second. This value is the result of dividing the number of requests by the total time taken

Time per request

The average time spent per request. The first value is calculated with the formula concurrency timetaken 1000 / done while the second value is calculated with the formula timetaken * 1000 / done

Transfer rate

The rate of transfer as calculated by the formula totalread / 1024 / timetaken

give an example

As can be seen from the above, ab supports many parameters, but generally speaking, only - n (total number of requests sent) and - c (number of concurrent threads requested) parameters are necessary.

As can be seen from the columns in the figure, 200 threads were started concurrently this time, running a total of 2000 times (in order to avoid disclosing information security, the interface was treated with privacy).

As a result, we observed

Requests for second is 722.77, that is, 722.77 requests are processed every second;

Time per request is 276.714, that is, the average time of the request is 276.714ms;

In addition, the time-consuming indicators of 99 lines, 98 lines and 95 lines can also be used;

In particular, if the NON-2xx responses are 2000 and the total number of requests is 2000, it means that all the 2000 requests returned are non-2000. It is necessary to consider whether the requests returned are all wrong;