1.Redis single command usage scenario

The following steps are required for a Redis client to connect to a Redis server to execute a command:

The above process is called Round Trip Time (RTT), and the mget and mset commands save RTT, but most commands do not support batch operations.

Redis provides services through TCP, Client initiates requests through a Socket connection, and each request will block the Redis server for processing after the command is issued, and the results will be returned to the client after processing.

Redis's Client and Server communicate in the form of a question-and-answer, requesting one response at a time.When this process excludes the time-consuming Redis service itself takes to perform complex operations, you can see that the most time-consuming process is the network transmission process.Each command corresponds to sending and receiving two network transmissions. If a process takes 0.1 seconds, only 10 requests can be processed in one second, which severely limits Redis's performance.

In many scenarios, if we want to complete a business, we may need to perform a series of operations on redis, such as inventory reduction, order addition, balance reduction, etc. There are many steps that need to be performed sequentially.

2.Redis single command execution time-consuming

Normally, the execution time of a Redis single command is at the millisecond level.Most of the Reedis commands can be returned within 2ms.So Redis executes very quickly.However, in high concurrency scenarios, operating a large number of Redis KEY s at the same time may not meet the requirements of high performance.

Suppose you have a request to batch check if a user can participate in an existing 10,000 return activities and if the user can participate in a return activity using Redis KEY records and storage.Normally, a for loop is used to iterate through each activity to verify that the user can participate in each activity.

Assuming that each Redis command takes 1 ms to execute and 10,000 activities to verify = 10,000 * 1 MS = 10 s, an interface needs 10 s to respond to the results, this is obviously not realistic.

3.Redis Connection Pool

In addition to the slow interface response time problem described in point 2, there is another problem: multiple Redis KEY operations in an interface result in a long period of unrelease of connections to the Redis connection pool, which may result in connections in the connection pool being quickly occupied by the first few threads, delayed release, and large numbers of threads not getting access to the connection pool at high volumesConnections cause a large number of threads to wait for timeouts.

4. Use of Redis Pipeline Pipeline command

The Pipeline command works as illustrated below.

Redis is a TCP service based on the client-server model and request/response protocol.This means that normally a request follows the following steps:

1. Clients send a query request to the server and listen for Socket returns, usually in blocking mode, waiting for the server to respond.2. The server handles the command and returns the result to the client.

Redis pipeline technology allows clients to continue sending requests to the service side when the service side does not respond and eventually read all the service side responses at once.This maximizes Redis's performance and saves unnecessary network IO overhead.

Execute a single set command 100000 times without using the Pipeline command

/**
 * Do not use the Pipeline command
 * @param count Number of commands to operate on
 * @return execution time
 */
@GetMapping("redis/no/pipeline/{count}")
public String testWithNoPipeline(@PathVariable("count") String count) {
	// start time
	long start = System.currentTimeMillis();
	// Parameter Check
	if (StringUtils.isEmpty(count)) {
		throw new IllegalArgumentException("Parameter Exception");
	}
	// for loop performs N Redis operations
	for (int i = 0 ; i < Integer.parseInt(count); i++) {
		// Set K-V
		stringRedisTemplate.opsForValue().set(String.valueOf(i), 
				String.valueOf(i), 1, TimeUnit.HOURS);
	}
	// End time
	long end = System.currentTimeMillis();
	// Return Total Execution Time
	return "Execution time equals=" + (end - start) + "Millisecond";
}

The browser enters the following URL:

http://localhost:8080/redis/no/pipeline/10000

Verify that Redis executes as follows.

The result is 219 milliseconds.

Execute the following command to empty the result saved to Redis by the code just executed.

flushall

Save 10,000 pieces of data with the Pipeline command

/**
 * Use the Pipeline command
 * @param count Number of commands to operate on
 * @return execution time
 */
@GetMapping("redis/pipeline/{count}")
public String testWithPipeline(@PathVariable("count") String count) {
	// start time
	long start = System.currentTimeMillis();
	// Parameter Check
	if (StringUtils.isEmpty(count)) {
		throw new IllegalArgumentException("Parameter Exception");
	}
	/* Insert Multiple Data */
	stringRedisTemplate.executePipelined(new SessionCallback<Object>() {
		@Override
		public <K, V> Object execute(RedisOperations<K, V> redisOperations) throws DataAccessException {
			for (int i = 0 ; i < Integer.parseInt(count); i++) {
				stringRedisTemplate.opsForValue().set(String.valueOf(i), String.valueOf(i), 1, TimeUnit.HOURS);
			}
			return null;
		}
	});
	// End time
	long end = System.currentTimeMillis();
	// Return Total Execution Time
	return "Execution time equals=" + (end - start) + "Millisecond";
}

The browser enters the following URL:

http://localhost:8080/redis/no/pipeline/10000

Execute the above code and the result is 161 milliseconds.

From the above test results, Redis Pipeline can significantly improve performance when multiple key s interact.

The above is a comparison of a single set command with a set command executed by Pipline, and the following is a single get command and a get command executed by Pipline.

Test single get command, for loop 10000 times

/**
 * Single get command without Pipeline command
 *
 * @param count Number of commands to operate on
 * @return execution time
 */
@GetMapping("redis/no/pipeline/get/{count}")
public Map<String, Object> testGetWithNoPipeline(@PathVariable("count") String count) {
    // start time
    long start = System.currentTimeMillis();
    // Parameter Check
    if (StringUtils.isEmpty(count)) {
        throw new IllegalArgumentException("Parameter Exception");
    }
    List<String> resultList = new ArrayList<>();
    // for loop performs N Redis operations
    for (int i = 0; i < Integer.parseInt(count); i++) {
        // Get K-V
        resultList.add(stringRedisTemplate.opsForValue().get(String.valueOf(i)));
    }
    // End time
    long end = System.currentTimeMillis();
    Map<String, Object> resultMap = new HashMap<>(4);
    resultMap.put("execution time", (end - start) + "Millisecond");
    resultMap.put("results of enforcement", resultList);
    // Return to resultMap
    return resultMap;
}

Execute the following URL:

http://localhost:8080/redis/no/pipeline/get/10000

The results are as follows.

Test Pipeline to execute get command and get 10,000 pieces of data

/**
 * Use the Pipeline command
 *
 * @param count Number of commands to operate on
 * @return execution time
 */
@GetMapping("redis/pipeline/get/{count}")
public Map<String, Object> testGetWithPipeline(@PathVariable("count") String count) {
    // start time
    long start = System.currentTimeMillis();
    // Parameter Check
    if (StringUtils.isEmpty(count)) {
        throw new IllegalArgumentException("Parameter Exception");
    }
    // for loop performs N Redis operations
    /* Get multiple pieces of data in batch */
    List<Object> resultList = stringRedisTemplate.executePipelined(new RedisCallback<String>() {
        @Override
        public String doInRedis(RedisConnection redisConnection) throws DataAccessException {
            StringRedisConnection stringRedisConnection = (StringRedisConnection) redisConnection;
            for (int i = 0; i < Integer.parseInt(count); i++) {
                stringRedisConnection.get(String.valueOf(i));
            }
            return null;
        }
    });
    // End time
    long end = System.currentTimeMillis();
    Map<String, Object> resultMap = new HashMap<>(4);
    resultMap.put("execution time", (end - start) + "Millisecond");
    resultMap.put("results of enforcement", resultList);
    // Return to resultMap
    return resultMap;
}

Execute the following URL:

http://localhost:8080/redis/pipeline/get/10000

The results are as follows.

The total execution time is 18 milliseconds.

5. Summary

Pipelines are used not only to reduce RTT latency costs, but also to substantially increase the total number of operations executed per second in the Redis server. This is because, without pipes, this frequent I/O operation can be costly, even though it is easy to operate a single command, involving system read-write calls.It means from user domain to kernel domain. Context switching can cause a great loss of speed.

When pipeline operations are used, many commands are usually read using a single read() system call and multiple replies are passed through a single write() system call. As a result, the total number of queries executed per second initially increases linearly with the longer pipeline and eventually reaches 10 times that obtained without pipeline technology, as shown in the following figure:

This text source address

https://github.com/online-demo/redis-pipeline.git