Analysis of Redis Technology Series: Redission's implementation of distributed locks

Keywords: lua redisson

Introduction to redistribution

Redission is a distributed solution for Redis official website

Function distribution of redistribution

Maven configuration

<!--Maven-->
<dependency>
   <groupId>org.redisson</groupId>
   <artifactId>redisson</artifactId>
   <version>3.10.4</version>
</dependency>

Basic use

// 1. Create config object
Config = ...
// 2. Create Redisson instance
RedissonClient redisson = Redisson.create(config);
// 3. Get Redis based object or service you need
RMap<MyKey, MyValue> map = redisson.getMap("myMap");
RLock lock = redisson.getLock("myLock")
lock.lock();
//Business code
lock.unlock();

Official source API

RedissionLock class

RLock red lock

Redistribution uses Lua script to execute shackle logic

Redismission accesses Redis through lua script to ensure the atomicity of business logic execution.

The following is the code for locking lua in redistribution

if (redis.call('exists', KEYS[1]) == 0) then 
        redis.call('hset', KEYS[1], ARGV[2], 1);
         redis.call('pexpire', KEYS[1], ARGV[1]); 
         return nil;
         end;
if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then
        redis.call('hincrby', KEYS[1], ARGV[2], 1);
        redis.call('pexpire', KEYS[1], ARGV[1]); 
        return nil;
        end;
return redis.call('pttl', KEYS[1]);

  • KEYS[1]: it represents the key of the lock added. I only need to judge whether the key value exists to know whether the lock is held by the thread.

  • ARGV[1]: indicates the validity period of the lock. The default is 30s

  • ARGV[2]: indicates the ID of the locked client

  • First, judge whether the key value of the lock exists. If it does not exist, you can lock it directly. If it already exists, it is necessary to judge whether the thread holding the lock is the current thread. Therefore, use hexist to judge whether the ID of the current thread exists in the hash. If it exists, it means that the current thread holds the lock, and you can enter again.

  • Increase the value by 1 and extend the validity of the lock. If it is not the ID of the current thread, the remaining lifetime will be returned, and the current thread will enter a loop and constantly try to obtain the lock.

The following is the lua code for Redisson to release the lock

if (redis.call('exists', KEYS[1]) == 0) then
       redis.call('publish', KEYS[2], ARGV[1]);
        return 1; 
        end;
if (redis.call('hexists', KEYS[1], ARGV[3]) == 0) then 
     return nil;
     end;
local counter = redis.call('hincrby', KEYS[1], ARGV[3], -1); 
if (counter > 0) then
     redis.call('pexpire', KEYS[1], ARGV[2]); 
     return 0; 
else redis.call('del', KEYS[1]); 
     redis.call('publish', KEYS[2], ARGV[1]); 
     return 1;
     end;
return nil;
  • If the key does not exist, the lock has been released. Directly execute the publish command to release the lock message and return 1.
  • The key exists, but the field does not exist in the Hash. It means that you are not the lock holder and have no right to release the lock. nil is returned.
  • Because the lock is reentrant, you cannot release all the acquired locks when releasing the lock. Only one lock can be released at a time. Therefore, execute hincrby to reduce the value of the lock by one.
  • After releasing a lock, if there are still remaining locks, refresh the lock expiration time and return 0; If the last lock has been released just now, execute the del command to delete the key of the lock, publish the lock release message, and return 1.

Basic characteristics of distributed lock

Mutex

Mutex is a very common feature of all pessimistic locks or synchronous locks. It means that only one thread can obtain a lock at the same time, which is the most basic and important point.

Automatic release lock (anti deadlock)

In the case of distributed high concurrency, suppose a thread acquires a lock, but it cannot execute the command to release the lock because of system failure or other reasons. Then it will always hold this lock, and other threads will always wait, resulting in deadlock. Therefore, we need to set the expiration time for it. Even if this happens, the lock can be released automatically after the lock expires.

High performance

The granularity of distributed locks should be as small as possible. For example, if you want to lock the inventory, the name of the lock can be the commodity ID, not any name. In this way, the lock will be added only when you operate the commodity

Small particle size.

The scope of the lock should be as small as possible: for example, if we can lock only two lines of code, then don't lock ten lines. Therefore, when implementing distributed locking with redistribution, conventional locking is better than custom annotation locking. Because the user-defined annotation can only be locked on the method, the granularity of the lock is large.

Reentrant

Like ReentrantLock, redistribution can also achieve this, which is conducive to the efficient utilization of resources.

Distributed lock execution process of redistribution

  • Competing for distributed locks

    • Once the thread obtains the lock, it succeeds, executes the lua script, and saves the data to the redis database.
    • Thread 2 failed to acquire the lock. It has been trying to acquire the lock through the while loop. After successful acquisition, execute the lua script and save the data to the redis database.

  • The function of Watch dog is to extend the effective time of the lock when the effective time of the lock is coming, and when the business logic has not been executed yet.

  • Normally, the Watch dog thread is not started. In addition, after the Watch dog is started, it will also have a certain impact on the overall performance, so it is not recommended to open the watchdog.

  • Complex business logic is encapsulated in lua script and sent to redis, and redis is atomic, which ensures the atomicity of this logic.

Redisson can implement reentrant locking mechanism

If thread 2 enters again when it already holds the lock, it does not need to change the thread ID, just change the value value. This is a bit like the locking process of bias lock: when the lock flag bit is checked successfully, the thread ID in Mark Word will be changed to its own thread ID through CAS operation, and the bias lock position will be 1. If thread 2 enters the synchronization zone next time, you don't need to perform this step.

Disadvantages of redistribution (red lock is introduced)

In Redis sentinel mode, when a thread writes a redistribution lock to the master node, it will be asynchronously copied to the slave node. If the master node fails and goes down at this time, the active / standby switch will occur, and the slave node becomes the master node. At this time, thread 2 can also write the reission lock to the new master node. In this way, multiple clients can lock the same distributed lock at the same time, which may lead to dirty data.

Redission source code analysis introduction

Locking mechanism

//org.redisson.RedissonLock#tryLockInnerAsync
return this.commandExecutor.evalWriteAsync(this.getName(), LongCodec.INSTANCE, command, 
    "if (redis.call('exists', KEYS[1]) == 0) then 
        redis.call('hset', KEYS[1], ARGV[2], 1); 
        redis.call('pexpire', KEYS[1], ARGV[1]); 
            return nil; 
    end; 
    if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then 
        redis.call('hincrby', KEYS[1], ARGV[2], 1); 
        redis.call('pexpire', KEYS[1], ARGV[1]); 
        return nil; 
    end; 
return redis.call('pttl', KEYS[1]);"
Collections.singletonList(this.getName()), new Object[]{this.internalLockLeaseTime, this.getLockName(threadId)});

Why use lua?

Because a lot of complex business logic can be encapsulated in lua script and sent to redis to ensure the atomicity of this complex business logic.

lua field explanation:
  • KEYS[1] represents the key you locked, for example:
RLock lock = redisson.getLock("myLock");

Here, you set the key of the lock, which is "myLock".

  • ARGV[1] represents the default lifetime of the lock key, which is 30 seconds by default.

  • ARGV[2] represents the ID of the locked client, similar to the following:

The first if judgment statement is to use the "exists myLock" command to judge. If the lock key you want to lock does not exist, you will lock it.

hset myLock
8743c9c0-0795-4907-87fd-6c719a6b4586:1 1

Set a hash data structure through this command. After this command is executed, a data structure similar to the following will appear:

myLock:
 {
        8743c9c0-0795-4907-87fd-6c719a6b4586:1 1
}

Then, execute the command "pexpire myLock 30000" and set the lifetime of the key of myLock to 30 seconds (default)

Lock mutual exclusion mechanism

  • At this time, what happens if client 2 tries to lock and executes the same lua script?

  • Very simply, the first if judgment will execute "exists myLock", and it is found that the lock key myLock already exists.

  • The second if judgment is to judge whether the hash data structure of the myLock lock key contains the ID of client 2, but it is obviously not, because it contains the ID of client 1.

  • Therefore, client 2 will get a number returned by pttl myLock. This number represents the remaining lifetime of the myLock key. For example, there are 15000 milliseconds left.

  • At this point, client 2 will enter a while loop and keep trying to lock.

Automatic delay mechanism of watch dog

  • The default lifetime of the lock key locked by client 1 is only 30 seconds. If it exceeds 30 seconds, client 1 still wants to hold the lock all the time. What should I do?

  • Simple! Once client 1 locks successfully, it will start a watch dog. It is a background thread and will check every 10 seconds. If client 1 still holds the lock key, it will continue to prolong the survival time of the lock key.

Reentrant locking mechanism

If client 1 already holds this lock, what happens to the reentrant lock?

RLock lock = redisson.getLock("myLock")
lock.lock();
//Business code
lock.lock();
//Business code
lock.unlock();
lock.unlock();

Analyze the lua script above.

  • The first if judgment is definitely not true. "exists myLock" will show that the lock key already exists.

  • The second if judgment holds because the ID contained in the hash data structure of myLock is the ID of client 1, that is, "8743c9c0-0795-4907-87fd-6c719a6b4586:1"

At this time, the logic of reentrant locking will be executed. He will use:

incrby myLock

8743c9c0-0795-4907-87fd-6c71a6b4586:1 1

Through this command, the number of locks on client 1 is accumulated by 1.

At this point, the myLock data structure changes to the following:

myLock:
    {
        8743c9c0-0795-4907-87fd-6c719a6b4586:1 2
    }

Release lock mechanism

if (redis.call('exists', KEYS[1]) == 0) then
       redis.call('publish', KEYS[2], ARGV[1]);
        return 1; 
        end;
if (redis.call('hexists', KEYS[1], ARGV[3]) == 0) then 
     return nil;
     end;
local counter = redis.call('hincrby', KEYS[1], ARGV[3], -1); 
if (counter > 0) then
     redis.call('pexpire', KEYS[1], ARGV[2]); 
     return 0; 
else redis.call('del', KEYS[1]); 
     redis.call('publish', KEYS[2], ARGV[1]); 
     return 1;
     end;
return nil;

  • If you execute lock.unlock(), you can release the distributed lock. The business logic at this time is also very simple.

  • In fact, to put it bluntly, the number of locks in the myLock data structure is reduced by 1 every time.

  • If it is found that the number of locks is 0, it means that the client no longer holds the lock. At this time, it will use:

  • "del myLock" command to delete this key from redis.

  • Then, another client 2 can try to complete locking.

  • This is the implementation mechanism of the so-called distributed lock open source Redisson framework.

Advantages of redistribution

It supports various deployment architectures such as Redis single instance, Redis sentry, redis cluster and Redis master slave. Based on Redis, it has a complete package for Redis functions. Many companies can use it in enterprise level projects after trial, and the community is highly active.

Disadvantages of redistribution

  • The biggest problem is that if you write the value of a lock key such as myLock to a redis master instance, it will be asynchronously copied to the corresponding master slave instance.

  • However, during this process, once the redis Master goes down, the active and standby switches, and the redis slave becomes the redis master.

  • Then, when client 2 tries to lock, it completes locking on the new redis master, and client 1 thinks it has successfully locked.

  • This will cause multiple clients to lock a distributed lock.

  • At this time, the system will have problems in business semantics, resulting in dirty data.

  • Therefore, this is the biggest defect of redis distributed locks caused by the master-slave asynchronous replication of redis cluster or redis master slave architecture: when the redis master instance goes down, multiple clients may complete locking at the same time.

Redisssion also supports the implementation of RedLock algorithm, but it is controversial at present. You can query it yourself and use it with caution

Posted by spenceddd on Wed, 03 Nov 2021 00:32:18 -0700

Hot Keywords

©2024 Programmer Group