1. What is benchmarking
Benchmarking, in short, is a kind of stress testing for system design. Usually the goal is to master the behavior of the system. Benchmarking can observe the behavior of the system under different pressures, assess the capacity of the system, grasp what are important changes, or observe how the system processes different data. Benchmarking can create fictional scenarios for testing outside of the actual system load.
2. Benchmarking Strategy
(1) Integrated benchmarking: for the whole system
(2) Single-component benchmarking: for databases
3. Test Indicators
(1) Throughput: Number of transactions per unit time
(2) Response time or latency: the overall time required to test the task
For example, to measure the speed of a particular MySQL expression or function, you can call the BENCHMARK() function
mysql> select benchmark(10000000,2*5); +-------------------------+ | benchmark(10000000,2*5) | +-------------------------+ | 0 | +-------------------------+ 1 row in set (0.26 sec)
It shows that MySQL can execute 10,000,000 simple multiplication expressions on the system in 0.26 seconds.
(3) Concurrency: Concurrency concerns the number of concurrent operations being performed, or the number of threads or connections being worked at the same time.
Note that database connectivity and concurrency are different. A good application system can open hundreds of database connections at the same time, but there may be only a few connections in the execution of queries.
(4) Scalability: Simply put, scalability means doubling the system's work and ideally doubling its throughput.
4. Benchmarking Tools
(1)sysbench installation
sysbench can perform a variety of benchmarks, which can be used not only to test database performance, but also to test server performance running the database.
Download: https://packagecloud.io/akopytov/sysbench
(2)sysbench use and help
[root@relay3.mobvista.com:101.251.254.6 ~]#curl -s https://packagecloud.io/install/repositories/akopytov/sysbench/script.rpm.sh | sudo bash [root@relay3.mobvista.com:101.251.254.6 ~]#yum install sysbench -y
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --help // view parameters Usage: sysbench [options]... [testname] [command] //Grammatical Format ... ... Compiled-in tests: //Built-in test items fileio - File I/O test cpu - CPU performance test memory - Memory functions speed test threads - Threads subsystem performance test mutex - Mutex performance test See 'sysbench <testname> help' for a list of options for each test.
By sysbench < testname > help, you can see the test options for specific tests, such as file I/O test options:
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --test=fileio help
fileio options:
--file-num=N number of files to create [128]
--file-block-size=N block size to use in all IO operations [16384]
--file-total-size=SIZE total size of files to create [2G]
--file-test-mode=STRING test mode {seqwr, seqrewr, seqrd, rndrd, rndwr, rndrw}
--file-io-mode=STRING file operations mode {sync,async,mmap} [sync]
--file-async-backlog=N number of asynchronous operatons to queue per thread [128]
--file-extra-flags=[LIST,...] list of additional flags to use to open files {sync,dsync,direct} []
--file-fsync-freq=N do fsync() after this number of requests (0 - don't use fsync()) [100]
--file-fsync-all[=on|off] do fsync() after each write operation [off]
--file-fsync-end[=on|off] do fsync() at the end of test [on]
--file-fsync-mode=STRING which method to use for synchronization {fsync, fdatasync} [fsync]
--file-merged-requests=N merge at most this number of IO requests if possible (0 - don't merge) [0]
--file-rw-ratio=N reads/writes ratio for combined test [1.5]
(3) Use examples
Example 1: File i/o benchmark
// In the test preparation stage, 16 threads are used to create test files with a total size of 1GB and the test mode is random reading and writing.
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw prepare sysbench 1.0.17 (using system LuaJIT 2.0.4) 128 files, 8192Kb each, 1024Mb total Creating files for the test... Extra file open flags: (none) Creating file test_file.0 Creating file test_file.1 Creating file test_file.2 Creating file test_file.3 ... ...
// Test Running Phase
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-m ode=rndrw run
sysbench 1.0.17 (using system LuaJIT 2.0.4)
Running the test with following options:
Number of threads: 16 //Number of threads
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 8MiB each
1GiB total file size //Total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50 //Read-write ratio
Periodic FSYNC enabled, calling fsync() each 100 requests. //Synchronize memory and hard disk data with fsync() every 100 requests
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Initializing worker threads...
Threads started!
File operations:
reads/s: 241.51
writes/s: 160.12
fsyncs/s: 705.43
Throughput: //throughput
read, MiB/s: 3.77
written, MiB/s: 2.50
General statistics:
total time: 10.1590s
total number of events: 9201
Latency (ms):
min: 0.00
avg: 17.64
max: 317.89
95th percentile: 227.40 //Over 95% response time
sum: 162323.47
Threads fairness:
events (avg/stddev): 575.0625/35.67
execution time (avg/stddev): 10.1452/0.00
// During the test clearance phase, delete all test files:
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --test=fileio --num-threads=16 --file-total-size=1G --file-test-mode=rndrw cleanup sysbench 1.0.17 (using system LuaJIT 2.0.4) Removing test files...
Example 2: CPU benchmark
The time required to test and calculate the prime number to a certain maximum
[root@relay3.mobvista.com:101.251.254.6 ~]#sysbench --test=cpu --cpu-max-prime=2000 run // prime generator upper limit (default value is 10000)
sysbench 1.0.17 (using system LuaJIT 2.0.4)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Prime numbers limit: 2000
Initializing worker threads...
Threads started!
CPU speed:
events per second: 8164.94
General statistics:
total time: 10.0002s
total number of events: 81685
Latency (ms):
min: 0.12
avg: 0.12
max: 1.17
95th percentile: 0.13
sum: 9965.18
Threads fairness:
events (avg/stddev): 81685.0000/0.00
execution time (avg/stddev): 9.9652/0.00