1, What is a distributed lock?

We all know the concept of lock when learning Java, such as synchronized lock based on JVM implementation and a set of code level locking mechanism lock provided by jdk. We often use these two locks in concurrent programming to ensure the correctness of code running in multi-threaded environment. However, these locking mechanisms are not applicable in the distributed scenario, because in the distributed business scenario, our code runs on different JVMs or even different machines, and synchronized and lock can only work in the same JVM environment. Therefore, distributed locks are needed at this time.

For example, there is a scene that is to grab the consumer voucher at all. The reason for the epidemic is that Alipay recently opened up 8 points and 12 o'clock o'clock to open up the first come first served and the consumer tickets had a fixed amount.

So at this time, we have to use distributed locks to ensure the correctness of access to shared resources.

2, Why use distributed locks, huh?

Assuming that distributed locks are not used, let's see if synchronized can guarantee? Actually, we can't. let's demonstrate it.

Now I wrote the first mock exam of the voucher, a simple springboot project. The code is very simple. It means that the number of remaining coupons is obtained from Redis first, and then the judgment is greater than 0. Then one simulation is grabbed by one user, then one is reduced to one, then the remaining Redis consumption is increased. I didn't get it. The whole code is synchronized, and the inventory quantity set by redis is 50.

//Assume stock number is 00001
private String key = "stock:00001";
@Autowired
private StringRedisTemplate stringRedisTemplate;
/**
 * Deduct inventory synchronized lock
*/
@RequestMapping("/deductStock")
public String deductStock(){
    synchronized (this){
        //Get current inventory
        int stock = Integer.parseInt(stringRedisTemplate.opsForValue().get(key));
        if(stock>0){
            int afterStock = stock-1;
            stringRedisTemplate.opsForValue().set(key,afterStock+"");//Modify inventory
            System.out.println("Inventory deduction succeeded, remaining inventory"+afterStock);
        }else {
            System.out.println("Inventory deduction failed");
        }
    }
    return "ok";
}

Then start two spring boot projects with ports 8081 and configure load balancing in nginx

upstream redislock{
    server 127.0.0.1:8080;
    server 127.0.0.1:8081;
}
server {
    listen       80;
    server_name  127.0.0.1;
    location / {
        root   html;
        index  index.html index.htm;
        proxy_pass http://redislock;
    }
}

Then test with jmeter pressure measuring tool

Then we take a look at the console output. We can see that the two web instances we are running, many of the same coupons are robbed by different threads, which proves that synchronized does not work in this case, so we need to use distributed locks to ensure the correctness of resources.

3, How to implement distributed locks with Redis?

Before implementing distributed locks, we first consider how to implement them and what functions of locks should be implemented.

1. Distributed feature (instances deployed on multiple machines can access this lock)
2. Exclusivity (only one thread can hold a lock at a time)
3. Timeout automatic release feature (the thread holding the lock needs to give a certain maximum time to hold the lock to prevent the thread from dying and unable to release the lock, resulting in deadlock)
4. . . .

Based on the basic features of distributed locks listed above, let's think about how to implement Redis?

1. The first distributed feature, Redis, has been supported. Multiple instances can be combined with one Redis
2. The second exclusivity, that is, to implement an exclusive lock, can be implemented using Redis's setnx command
3. The third timeout automatic release feature allows Redis to set the expiration time for a key
4. Release the distributed lock after execution

Popular science time

Redis Setnx command

Redis Setnx (SET if Not eXists) command sets the specified value for the specified key when the specified key does not exist

grammar

The basic syntax of redis Setnx command is as follows:

redis 127.0.0.1:6379> SETNX KEY_NAME VALUE

Available versions: > = 1.0.0

Return value: set successfully, return 1, set failed, return 0

@RequestMapping("/stock_redis_lock")
public String stock_redis_lock(){
    //The bottom layer uses the setnx command
    Boolean aTrue = stringRedisTemplate.opsForValue().setIfAbsent(lock_key, "true");
    stringRedisTemplate.expire(lock_key,10, TimeUnit.SECONDS);//Set the expiration time to 10 seconds
    if (!aTrue) {//If the setting fails, the distributed lock is not obtained
        return "error";//Here you can give users a friendly prompt
    }
    //Get current inventory
    int stock = Integer.parseInt(stringRedisTemplate.opsForValue().get(key));
    if(stock>0){
        int afterStock = stock-1;
        stringRedisTemplate.opsForValue().set(key,afterStock+"");
        System.out.println("Inventory deduction succeeded, remaining inventory"+afterStock);
    }else {
        System.out.println("Inventory deduction failed");
    }
    stringRedisTemplate.delete(lock_key);//Release the distributed lock after execution
    return "ok";
}

Still set the inventory quantity to 50. Let's test it again with jmeter and change the test address of jmeter to
127.0.0.1/stock_redis_lock, test again with the same settings.

After testing for 5 times, there is no dirty data. Change the sending time to 0. There is no problem after testing for 5 times. Then change the number of threads to 600. The time is 0. Cycle for 4 times. It is normal to test for several times.

The above code for implementing distributed lock is a relatively mature implementation of distributed lock, which has met the needs of most software companies. However, the above code still has room for optimization, for example:

1. In the above code, we do not consider exceptions. In fact, the code is not so simple. There may be many other complex operations, and exceptions may occur. Therefore, the code for releasing the lock needs to be placed in the finally block to ensure that even if the code throws an exception, the code for releasing the lock will still be executed.
2. In addition, have you noticed that the code for obtaining and setting the expiration time of our distributed lock above is a two-step operation, that is, the non atomic operation. It is possible that the machine hangs before the execution of line 4 and line 5. Then the timeout time is not set for this lock, and other threads cannot obtain it unless manual intervention, Therefore, this is the place for one-step optimization. Redis also provides atomic operations, that is, SET key value EX seconds NX

Popular science time

SET key value [EX seconds] [PX milliseconds] [NX|XX]   Associate the string value value with the key

Optional parameters

Starting from redis version 2.6.12, the behavior of the SET command can be modified through a series of parameters:

EX second: set the expiration time of the key to second seconds. The effect of SET key value EX second is the same as that of set key second value

PX millisecond: set the expiration time of the key to millisecond. SET key value PX millisecond effect is equivalent to PSETEX key millisecond value

Nx: set the key only when the key does not exist. SET key value NX effect is equivalent to SETNX key value

20: Set the key only when the key already exists

The StringRedisTemplate of SpringBoot also has corresponding method implementation, as shown in the following code:

//Assume stock number is 00001
private String key = "stock:00001";
private String lock_key = "lock_key:00001";
@Autowired
private StringRedisTemplate stringRedisTemplate;
@RequestMapping("/stock_redis_lock")
public String stock_redis_lock() {try {
        //Atomic key setting and timeout
        Boolean aTrue = stringRedisTemplate.opsForValue().setIfAbsent(lock_key, "true", 30, TimeUnit.SECONDS);
        if (!aTrue) {
            return "error";
        }
        int stock = Integer.parseInt(stringRedisTemplate.opsForValue().get(key));
        if (stock > 0) {
            int afterStock = stock - 1;
            stringRedisTemplate.opsForValue().set(key, afterStock + "");
            System.out.println("Inventory deduction succeeded, remaining inventory" + afterStock);
        } else {
            System.out.println("Inventory deduction failed");
        }
    } catch (NumberFormatException e) {
        e.printStackTrace();
    } finally {
        //Release lock
        stringRedisTemplate.delete(lock_key);
    }
    return "ok";
}

Is this perfect? Well, for scenarios with low concurrency requirements or non large concurrency, this is OK. However, in the scenario of rush buying and second kill, when the traffic is large, the server network card, disk IO and CPU load may reach the limit, and the server's response time to a request is bound to be much slower than under normal circumstances. Suppose that the timeout of the lock just set is 10 seconds, If a thread fails to finish executing the lock within 10 seconds after it gets the lock for some reason, the lock will become invalid. At this time, other threads will seize the distributed lock to execute the business logic, and then the previous thread will execute the code to release the lock in finally, and the lock of the thread occupying the distributed lock will be released, In fact, if the thread just occupying the lock has not finished executing, other threads will have the opportunity to obtain the lock again... In this way, the whole distributed lock will fail, which will have unexpected consequences. The following figure simulates this scenario.

To sum up, the problem is that the lock expiration time is not set properly, or the code execution time is greater than the lock expiration time for some reasons, resulting in concurrency problems, and the lock is released by other threads, resulting in distributed lock confusion. In short, there are two problems,

1. His lock was released by others
2. Lock timeout cannot be extended.

The first problem is easy to solve. When setting a distributed lock, we produce a unique string in the current thread, set value to this unique value, and then judge that the value of the current lock is the same as that set by ourselves in the finally block, and then execute delete, as follows:

String uuid = UUID.randomUUID().toString();
try {
    //Atomic setting key and timeout, lock unique value
    Boolean aTrue = stringRedisTemplate.opsForValue().setIfAbsent(lock_key,uuid,30,TimeUnit.SECONDS);
    //...
} finally {
    //If the lock is set by yourself, then execute delete
    if(uuid.equals(stringRedisTemplate.opsForValue().get(lock_key))){
        stringRedisTemplate.delete(lock_key);//Avoid deadlock
    }
}

Once the problem is solved (imagine what else is wrong with the above code, which will be discussed later), the lock timeout is very critical. It can't be too large or too small. Therefore, it is necessary to evaluate the execution time of the business code, such as setting 10 seconds or 20 seconds. Even if you set an appropriate timeout for your lock, the above analysis can not be avoided. For some reasons, the code is not executed within the normal evaluation time, so the solution at this time is to extend the timeout for the lock. The general idea is that the business thread acts as a separate sub thread to regularly monitor whether the distributed lock set by the business thread still exists. If it exists, it means that the business thread has not finished executing, then extend the timeout of the lock. If the lock does not exist, the business thread completes executing, and then ends itself.

The logic of "lock life extension" is true and a little complex. There are too many problems to consider. If you don't pay attention, there will be bug s. Don't look at the few lines of code for implementing distributed locks above, and think that the implementation is very simple. If you don't have actual high concurrency experience when implementing them, you will certainly step on many holes, for example,

1. The setting of lock and expiration time are non atomic operations, which may lead to deadlock.
2. There is also the one left over from the above. Judge whether the lock is set by yourself in the finally block. If yes, delete the lock. These two steps are not atomic. Assuming that the service hangs just after it is judged to be true, the code to delete the lock will not be executed, resulting in deadlock. Even if the expiration time is set, it will also deadlock before expiration. Therefore, it is also a point of attention here. To ensure atomic operation, Redis provides the function of executing Lua script to ensure the atomicity of operation. How to use it will not be expanded.

Therefore, the implementation of the logic of "lock for life" is still a little complicated. Fortunately, there is an existing open source framework on the market to help us implement it, that is, Redisson.

4, Implementation principle of Redisson distributed lock

Implementation principle:

1. Firstly, Redisson will try to lock. The principle of locking is to use the setnx command atom similar to Redis to lock. If the locking is successful, a child thread will be opened inside
2. The sub thread is mainly responsible for monitoring. In fact, it is a timer to regularly monitor whether the main thread still holds the lock. If it holds the lock, it will delay the time, otherwise the thread will end
3. If locking fails, spin keeps trying to lock
4. After executing the code, the main thread actively releases the lock

Let's take a look at the code after Redisson is used.

① First, add the maven coordinates of Redisson in the pom.xml file

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.12.5</version>
</dependency>

② We need to get the Redisson object and configure the Bean as follows

@SpringBootApplication
public class RedisLockApplication {
    public static void main(String[] args) {
        SpringApplication.run(RedisLockApplication.class, args);
    }
    @Bean
    public Redisson redisson(){
        Config config = new Config();
        config.useSingleServer().setAddress("redis://localhost:6379")
                .setDatabase(0);
        return (Redisson) Redisson.create(config);
    }
}

③ Then we get the instance of Redisson and use its API to perform lock adding and lock releasing operations

//Assume stock number is 00001
private String key = "stock:00001";
private String lock_key = "lock_key:00001";
@Autowired
private StringRedisTemplate stringRedisTemplate;
/**
 * Implementing distributed locks using Redisson
 * @return
 */
@RequestMapping("/stock_redisson_lock")
public String stock_redisson_lock() {
    RLock redissonLock = redisson.getLock(lock_key);
    try {
        redissonLock.lock();
        int stock = Integer.parseInt(stringRedisTemplate.opsForValue().get(key));
        if (stock > 0) {
            int afterStock = stock - 1;
            stringRedisTemplate.opsForValue().set(key, afterStock + "");
            System.out.println("Inventory deduction succeeded, remaining inventory" + afterStock);
        } else {
            System.out.println("Inventory deduction failed");
        }
    } catch (NumberFormatException e) {
        e.printStackTrace();
    } finally {
        redissonLock.unlock();
    }
    return "ok";
}

Is the API provided by Redisson's distributed Lock very simple? Just like the AQS Lock mechanism in Java concurrency, obtain a RedissonLock as follows

RLock redissonLock = redisson.getLock(lock_key); The RedissonLock object returned by default implements the RLOCK interface, which inherits the Lock interface in the JDK concurrent programming packet

When using Redisson locking, it also provides many API s, as follows

Now we choose to use the simplest nonparametric lock method. Simply click in and look at its source code. We find the final code for locking as follows:

We can see that the underlying layer uses Lua script to ensure atomicity, locks implemented by Redis's hash structure, and reentrant locks.

It seems simpler than our own implementation of distributed locks, but he has all the lock functions we write, and he also has those we don't have. For example, the distributed lock implemented by him supports reentry and waiting, that is, try to wait for a certain time and return false if you don't get the lock. redissonLock.lock() in the above code; Is always waiting, internal spin attempts to lock.

5, Conclusion

Here, the actual combat of Redis distributed lock is basically finished. Let's summarize Redis distributed lock.

1. If you want to realize it yourself, you should pay special attention to four points:

• Atomic locking
• Set lock timeout
• Who adds the lock will release it, and the atomic operation at the time of release
• Lock life extension problem.

1. If Redisson, an existing distributed lock framework, is used, you need to be familiar with its common API s and implementation principles, or choose other open source distributed lock frameworks to fully investigate and select one suitable for your business needs.