As mentioned in the previous article Redis Installation and Operation . Today let's look at Redis's master-slave replication, Sentinel;

I. Master-slave duplication

1. configuration

Modify redis.conf on Master

// Do not want to use password, so set protection mode to no
protected-mode no
// In fact, there is no need to configure anything on the master. It just cancels the protection mode.

Modify redis.conf on Slave1 and Slave2

// Similarly turn off protection mode
protected-mode no
// Set whose slave this machine is
slaveof master Of ip 6379
// When slaveof is configured, the following control is read-only on the local machine
slave-read-only yes

2. Jedis operation

    // Simply set up three connection pools
    private static final JedisPool masterPool;
    private static final JedisPool slavePool1;
    private static final JedisPool slavePool2;
    static {
        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
        // There can be up to 10 connections
        jedisPoolConfig.setMaxTotal(10);
        jedisPoolConfig.setMaxIdle(5);
        jedisPoolConfig.setMinIdle(5);
        masterPool = new JedisPool(jedisPoolConfig, "111.111.111.111");
        slavePool1 = new JedisPool(jedisPoolConfig, "111.111.111.112");
        slavePool2 = new JedisPool(jedisPoolConfig, "111.111.111.113");
    }

    public static void main(String[] args) throws Exception {
        // Simple use, through try-with-resource
        try (Jedis jedis = masterPool.getResource()) {
            jedis.get("key1");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

3. The Significance of Subordinates and Masters

1. Does Redis need to read and write separately?
   Perhaps you've thought about the question of whether read-write separation, which is commonly used in MySQL, will be needed in memory DB like Redis?

   Opponents argue that Redis is memory storage and reads and writes very fast. If it is read and separated from Master and Slaves, it will not only cause the trouble of master from synchronization, but also not necessarily improve the processing power and speed of the whole DB.

   Those in favor of it are: Master and Slave replication capabilities provided by Redis; introduction on the official website

   Replication can be used both for scalability, in order to have multiple slaves for read-only queries (for example, slow O(N) operations can be offloaded to slaves), or simply for data redundancy.

   Even the parameter slave-read-only yes in the configuration file prompts the user, and Redis gives you the ability to read and write separately. So why not use it?

   After thinking about it, I think:

   1. If the user's business data is not large enough, there is no need to separate reading and writing at all. Both reading and writing can be done on Master. But master-slave replication is still needed, Slave can be used as a simple data dump.
   2. If the user's business data is large and only one physical machine Master is used to read and write can not meet the needs of business or performance, then it can be read and separated. That is, encapsulate Redis operations in project code (such as Jedis operations), map write operations to Master, and map read operations to a Slave according to your own defined allocation strategy.

The most important indicators of an enterprise system are "availability" and "high performance".

Obviously, the above master-slave replication, read-write separation can simply provide "high performance", but it only improves the "read" performance, and can not expand the "write". (Written extensions are not listed here for the time being)

On the other hand, "availability" is also extremely important. If the usability of the above structure is not high, Redis will not be written immediately once Master is down, Slave will only have old data, and the system will not be available immediately.
Inevitably, Redis offers a high Availability solution, one of which is Sentinel-Sentinel.

Sentinel - High Availability

1. What is Sentinel?

Knowing by name, it is Sentinel Program provided by Redis. It is a distributed program, which can be described as follows:

Sentinel's main work is as follows:

1. Monitoring: Sentinel constantly checks whether your master and slave servers are functioning properly.

2. Notification: Sentinel can send notifications to administrators or other applications through the API when a Redis server being monitored has problems.

3. Automatic failover: When a primary server fails to work properly, Sentinel starts an automatic failover operation, which upgrades one of the failed primary servers from a slave server to a new primary server, and changes the other slave servers of the failed primary server to replicate a new primary server; when the client tries to connect to the failed primary server, Sentinel starts an automatic failover operation. When the server is served, the cluster will also return the address of the new primary server to the client, so that the cluster can use the new primary server instead of the invalid server.

2. Configure Sentinel

Sentinels should be placed on separate servers, preferably at least three sentries (three servers).

1. Configuration file sentinel.conf

   // 26379 mode is sentinel's operating port, 6379 is redis-server's
   port 26379
   // As a daemon
   daemonize yes
   // Work catalogue, set to your unified planning place
   dir /tmp
   // log file
   dir /....
   
   // protected-mode must be set, not set
   protected-mode no
   
   // Configure the Master for monitoring. Notice that there is no need to configure its Slave. Sentinel will ask Master himself.
   // sentinel monitor master-name ip redis-port quorum
   // The quorum sentry believed that the master needed a legal person to die objectively. Regardless of whether this value is set or not, most sentries must agree to start failover.
   sentinel monitor yewu01 127.0.0.1 6379 2
   
   // If Master does not respond after 30,000 ms, it means Master is in Subjectively Down.
   sentinel down-after-milliseconds yewu01 30000
   
   // When failover occurs, a slave starts synchronizing with the new master. Meaning: This slave begins to receive master's RDB files and can't provide services to the outside world, while other slaves can also provide services to the outside world (see the second ///), so the fewer the slaves, the faster redis can restore services to the outside world.
   // It also matches the slave-serve-stale-data parameter in redis.conf that configures slave to specify whether expired data is available or not.
   sentinel parallel-syncs yewu01 1
   
   // How long does failover take to count failover timeouts
   sentinel failover-timeout yewu01 180000

2. Running sentinel
   Although Redis Sentinel is released as a separate executable redis-sentinel, in fact it is only a Redis server running in a special mode. You can start Redis Sentinel by giving the -- sentinel option when starting a regular Redis server.

   // 1. Start with redis-server plus parameter-sentinel
   redis-server /path/to/sentinel.conf --sentinel
   // 2. Start with redis-sentinel mode
   redis-sentinel /path/to/sentinel.conf
   
   //Be careful! Be careful! Be careful! Don't forget to add ports to the firewall. My name is CentOS 7, so here's the following
   firewall-cmd --zone=public --add-port=26379/tcp --permanent
   

3. View state
   Login to Sentinel with redis-cli ip-p 26379;

   Sentinel's acceptable order:

   • PING: Return to PONG.
   • SENTINEL masters: Lists all monitored primary servers and their current status.
   • SENTINEL slaves: Lists all slave servers for a given master server and the current status of these slave servers.
   • SENTINEL get-master-addr-by-name: Returns the IP address and port number of the primary server with a given name. This command returns the IP address and port number of the new primary server if the primary server is performing a failover operation or if the failover operation for the primary server has been completed.
   • SENTINEL reset: Reset all primary servers whose names match the given pattern. The pattern parameter is a Glob-style pattern. The reset operation clarifies all the current state of the primary server, including the failover in progress, and removes all the slave servers and entinels that have been found and associated with the primary server.
   • SENTINEL failover: When the primary server fails, an automatic failover is forced to start without consulting other Sentinels (though Sentinel, which initiates the failover, sends a new configuration to other Sentinels, and other Sentinels update accordingly).

4. sentinel.conf configuration changed

   Whenever a Sentinel starts, it modifies and notifies other Sentinels to modify its sentinel.conf file at the same time, such as:

   Generate a myid

   sentinel myid 0f9bd55b18aa54a5f5efc6fb7b3371da56d48d4a

   At the end of the document, the following will be added:

   # Generated by CONFIG REWRITE
   
   sentinel known-sentinel yewu01 192.168.0.1 26379 58a141a0f97669925bcc84e3a3b3dbc8602dea99
   sentinel known-sentinel yewu01 192.168.0.2 26379 a0fbf10df21374f8b5cac1f410d9df3b26618575
   sentinel current-epoch 0

3. failover - Failover

1. Execute pkill redis-sever to turn off Master and observe Sentinel log as follows:

   This is a sentinel log selected as failover implementer. It is clear in English and will not be translated.

   6480:X 14 Feb 19:46:54.746 # +sdown master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:54.798 # +odown master yewu01 10.173.244.98 6379 #quorum 3/2
   6480:X 14 Feb 19:46:54.798 # +new-epoch 1
   6480:X 14 Feb 19:46:54.798 # +try-failover master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:54.807 # +vote-for-leader 214bd3df8363327488cd8c430166cf48cd2ab33a 1
   6480:X 14 Feb 19:46:54.819 # f7462491e6881da2c1efbfd6465ece6380c653cf voted for 214bd3df8363327488cd8c430166cf48cd2ab33a 1
   6480:X 14 Feb 19:46:54.823 # 6c95942bbcc39a0703ec5d54a76d6a696a500a17 voted for 214bd3df8363327488cd8c430166cf48cd2ab33a 1
   6480:X 14 Feb 19:46:54.907 # +elected-leader master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:54.907 # +failover-state-select-slave master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:54.974 # +selected-slave slave 10.174.249.145:6379 10.174.249.145 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:54.974 * +failover-state-send-slaveof-noone slave 10.174.249.145:6379 10.174.249.145 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:55.057 * +failover-state-wait-promotion slave 10.174.249.145:6379 10.174.249.145 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:55.534 # +promoted-slave slave 10.174.249.145:6379 10.174.249.145 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:55.534 # +failover-state-reconf-slaves master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:55.581 * +slave-reconf-sent slave 10.251.22.210:6379 10.251.22.210 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:55.957 # -odown master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:56.041 * +slave-reconf-inprog slave 10.251.22.210:6379 10.251.22.210 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:56.042 * +slave-reconf-done slave 10.251.22.210:6379 10.251.22.210 6379 @ yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:56.096 # +failover-end master yewu01 10.173.244.98 6379
   6480:X 14 Feb 19:46:56.096 # +switch-master yewu01 10.173.244.98 6379 10.174.249.145 6379
   6480:X 14 Feb 19:46:56.097 * +slave slave 10.251.22.210:6379 10.251.22.210 6379 @ yewu01 10.174.249.145 6379
   6480:X 14 Feb 19:46:56.097 * +slave slave 10.173.244.98:6379 10.173.244.98 6379 @ yewu01 10.174.249.145 6379
   6480:X 14 Feb 19:47:26.148 # +sdown slave 10.173.244.98:6379 10.173.244.98 6379 @ yewu01 10.174.249.145 6379

   This is Sentinel's log of a non-failover executor:

   30052:X 14 Feb 19:46:54.725 # +sdown master yewu01 10.173.244.98 6379
   30052:X 14 Feb 19:46:54.816 # +new-epoch 1
   30052:X 14 Feb 19:46:54.823 # +vote-for-leader 214bd3df8363327488cd8c430166cf48cd2ab33a 1
   30052:X 14 Feb 19:46:55.584 # +config-update-from sentinel 214bd3df8363327488cd8c430166cf48cd2ab33a 10.251.22.210 26379 @ yewu01 10.173.244.98 6379
   30052:X 14 Feb 19:46:55.584 # +switch-master yewu01 10.173.244.98 6379 10.174.249.145 6379
   30052:X 14 Feb 19:46:55.584 * +slave slave 10.251.22.210:6379 10.251.22.210 6379 @ yewu01 10.174.249.145 6379
   30052:X 14 Feb 19:46:55.584 * +slave slave 10.173.244.98:6379 10.173.244.98 6379 @ yewu01 10.174.249.145 6379
   30052:X 14 Feb 19:47:25.602 # +sdown slave 10.173.244.98:6379 10.173.244.98 6379 @ yewu01 10.174.249.145 6379

   The log shows that one Slave was promoted to Master.

2. Configuration changed after failover

   Sentinel first makes redis-server and sentinel configuration changes by command. The configuration is then persisted to redis.conf and sentinel.conf files.

   1. In redis.conf
      When selected as Master, the slaveof key value is deleted directly.
      Still Slave, whose slaveof value is specified as the address of the new Master;

   2. Sentinel sentinel.conf
      sentinel monitor is designated as the address of the new Master;

      // epoch "period" (meaning version) is self-increasing by 1
      sentinel current-epoch 1

4. Jedis operation

Obviously, after failover, the master-slave structure has changed and the master-slave structure has died. If Redis is connected in the way of writing dead IP as before, errors will inevitably occur. As you can imagine, under the entinel structure, you have to ask the sentry who is the Master.

    private static final JedisSentinelPool pool;
    static {
        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
        jedisPoolConfig.setMaxTotal(10);
        jedisPoolConfig.setMaxIdle(5);
        jedisPoolConfig.setMinIdle(5);

        Set<String> sentinels = new HashSet<>(Arrays.asList(
                "111.111.111.111:26379",
                "111.111.111.112:26379",
                "111.111.111.113:26379"
        ));
        GenericObjectPoolConfig poolConfig = new GenericObjectPoolConfig();
        poolConfig.setMaxTotal(10);
        poolConfig.setMaxIdle(5);
        poolConfig.setMinIdle(5);
        pool = new JedisSentinelPool("yewu01", sentinels, jedisPoolConfig);
    }

    public static void main(String[] args) throws Exception {
        String key1 = "key1";
        try (Jedis jedis = pool.getResource()) {
            jedis.set(key1, "222");
            System.out.println(jedis.get(key1));
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

Above.

Reference:

[ 1 ] https://redis.io/topics/sentinel
[ 2 ] http://www.redis.cn/topics/sentinel.html