Redis advanced application

Keywords: Database Docker Redis

1, Master slave replication

Quick start practice

Based on Redis, a Master-Slave architecture is designed, including one Master and two slaves. The Master is responsible for reading and writing Redis and synchronizing data to the Slave. The Slave is only responsible for reading. The steps are as follows:

First, duplicate redis01, for example:

cp -r redis01/ redis02
cp -r redis01/ redis03

Step 2: if there are existing redis services, first stop all the original redis services (docker rm -f redis container name) and start a new redis container, for example:

docker run -p 6379:6379 --name redis6379 \
-v /usr/local/docker/redis01/data:/data \
-v /usr/local/docker/redis01/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes
docker run -p 6380:6379 --name redis6380 \
-v /usr/local/docker/redis02/data:/data \
-v /usr/local/docker/redis02/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes
docker run -p 6381:6379 --name redis6381 \
-v /usr/local/docker/redis03/data:/data \
-v /usr/local/docker/redis03/conf/redis.conf:/etc/redis/redis.conf \
-d redis redis-server /etc/redis/redis.conf \
--appendonly yes

Step 3: check the redis service role

Start the three clients, log in to the three redis container services respectively, and view the roles through the info command. By default, the three newly started redis service roles are master> info replication
\# Replication

Step 4: check the ip settings of redis6379

docker inspect redis6379
"Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "c33071765cb48acb1efed6611615c767b04b98e6e298caa0dc845420e6112b73",
                    "EndpointID": "4c77e3f458ea64b7fc45062c5b2b3481fa32005153b7afc211117d0f7603e154",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null

Step 5: set Master/Slave architecture

Log in to redis6380/redis6381 respectively, and then execute the following statement

slaveof 6379
Note: if the master has a password, you need to add the statement "masterauth your password" in the redis.conf configuration file of the slave, restart redis, and then execute the slaveof command

Step 6: log in to redis6379 again and check info

[root@centos7964 ~]# docker exec -it redis6379 redis-cli> info replication
\# Replication

Step 7: log in to redis6379 to test. The master can read and write

root@centos7964 ~]# docker exec -it redis6379 redis-cli> set role master6379
OK> get role

Step 8: log in to redis6380 to test. slave can only read.

[root@centos7964 ~]# docker exec -it redis6380 redis-cli> get role
"master6379"> set role slave6380
(error) READONLY You can't write against a read only replica.>

Read / write test analysis in Java. The code is as follows:

public class MasterSlaveTests {
    private RedisTemplate redisTemplate;

    void testMasterReadWrite(){//The profile port is 6379
        ValueOperations valueOperations = redisTemplate.opsForValue();
        valueOperations.set("role", "master6379");
        Object role = valueOperations.get("role");

    void testSlaveRead(){//The profile port is 6380
        ValueOperations valueOperations = redisTemplate.opsForValue();
        Object role = valueOperations.get("role");


Redis and sentinel

Sentry quick start

Step 1: enter the three redis containers respectively for configuration, and create sentinel.conf file in the specified directory of the container / etc/redis. The file content is:

sentinel monitor redis6379 6379 1

Among them, the above instruction indicates the master to be monitored, redis6379 is the service name, and 6379 are the ip and port of the master, and 1 indicates how many sentinel s think a master is invalid

If bash: vi: command not found appears here, the following two instructions can be executed in sequence

apt-get update 
apt-get install vim

Step 2: execute the following instructions in the / etc/redis directory inside each redis container (preferably in multiple client windows) to start the sentinel service

redis-sentinel sentinel.conf

Step 3: open a new client connection window and close redis6379 service (this service is the master service)

docker stop redis6379 

Other client windows to detect log output, such as

410:X 11 Jul 2021 09:54:27.383 # +switch-master redis6379 6379 6379
410:X 11 Jul 2021 09:54:27.383 * +slave slave 6379 @ redis6379 6379
410:X 11 Jul 2021 09:54:27.383 * +slave slave 6379 @ redis6379 6379

Step 4: log in to the corresponding service with ip for info detection, for example:> info replication
\# Replication

From the above information output, it is found that the redis6381 service has now become a master.

Sentinel configuration advanced
For the contents in sentinel.conf file, we can also perform enhanced configuration based on actual requirements, such as:

sentinel monitor redis6379 6379 1 
daemonize yes #Background operation
logfile "/var/log/sentinel_log.log" #Run log
sentinel down-after-milliseconds redis6379 30000 #Default 30 seconds

Of which:
1) Daemon yes indicates background operation (no by default)
2)logfile is used to specify the location and name of the log file
3) Sentinel down after milliseconds indicates how long it takes for the master to fail

Analysis of sentry working principle
1) : each Sentinel sends a PING command to its known Master, Slave and other Sentinel instances once a second.

2) : if the time of an instance from the last valid reply to the PING command exceeds the value specified by the down after milliseconds option (this configuration item specifies the required expiration time, a master will be subjectively considered unavailable by the Sentinel. The unit is milliseconds, and the default is 30 seconds), Then this instance will be marked as subjective offline by Sentinel.

3) : if a Master is marked as subjective offline, all sentinels monitoring the Master should confirm that the Master has indeed entered the subjective offline state once per second.

4) : when a sufficient number of sentinels (greater than or equal to the value specified in the configuration file) confirm that the Master has indeed entered the subjective offline state within the specified time range, the Master will be marked as objective offline.

5) : in general, each Sentinel will send INFO commands to all known masters and Slave every 10 seconds.

6) : when the Master is marked as objectively offline by Sentinel, the frequency of Sentinel sending INFO commands to all Slave of the offline Master will be changed from once in 10 seconds to once per second.

7) : if not enough Sentinel agree that the Master has been offline, the objective offline status of the Master will be removed.
8) : if the Master returns a valid reply to Sentinel's PING command again, the Master's subjective offline status will be removed.

Redis cluster high availability
The reliability guarantee of redis single machine mode is not very good, and it is prone to single point of failure. At the same time, its performance is also limited by the processing capacity of CPU. Redis must be highly available in actual development, so the single machine mode is not our destination. We need to upgrade the current redis architecture mode.
Sentinel mode achieves high availability, but only one master is providing services in essence (in the case of read-write separation, the master is also providing services in essence). When the machine memory of the master node is insufficient to support the system data, the cluster needs to be considered.
Redis cluster architecture realizes the horizontal expansion of redis, that is, start n redis nodes, distribute and store the whole data in the n redis nodes, and each node stores 1/N of the total data. Redis cluster provides a certain degree of availability through partition. Even if some nodes in the cluster fail or cannot communicate, the cluster can continue to process command requests.

Basic architecture
For a redis cluster, it is generally set to at least 6 nodes, 3 masters and 3 slave. Its simple architecture is as follows:
Insert picture description here

Create cluster
Step 1: prepare the network environment
The creation of virtual network card is mainly used for redis cluster to communicate with the outside world. Generally, the bridge mode is commonly used.

docker network create redis-net

To view the network card information of docker, you can use the following instructions

docker network ls

To view docker network details, you can use the command

docker network inspect redis-net

Step 2: prepare redis configuration template

mkdir -p /usr/local/docker/redis-cluster
cd /usr/local/docker/redis-cluster
vim redis-cluster.tmpl

Enter the following in redis-cluster.tmpl

port ${PORT}
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-port ${PORT}
cluster-announce-bus-port 1${PORT}
appendonly yes

Of which:

Each node is explained as follows:

Port: node port, that is, the port that provides external communication
Cluster enabled: whether to enable the cluster
Cluster config file: cluster configuration file
Cluster node timeout: connection timeout
Cluster announcement ip: host ip
Cluster announcement port: cluster node mapping port
Cluster announcement bus port: cluster bus port
appendonly: persistent mode
Step 3: create a node profile

Execute the following command in redis cluser

for port in $(seq 8010 8015); \
do \
  mkdir -p ./${port}/conf  \
  && PORT=${port} envsubst < ./redis-cluster.tmpl > ./${port}/conf/redis.conf \
  && mkdir -p ./${port}/data; \

Of which:

for variable in $(seq var1 var2);do ...; done by linux One of shell Loop script, For example:
[root@centos7964 ~]# for i in $(seq 1 5);
do echo $i; done;

[root@centos7964 ~]#

The instruction envsubst < source file > target file is used to update the contents of the source file into the target file
View the contents of the configuration file through the cat instruction

cat /usr/local/docker/redis-cluster/801{0..5}/conf/redis.conf

Step 4: create a redis node container in the cluster

for port in $(seq 8010 8015); \
do \
   docker run -it -d -p ${port}:${port} -p 1${port}:1${port} \
  --privileged=true -v /usr/local/docker/redis-cluster/${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
  --privileged=true -v /usr/local/docker/redis-cluster/${port}/data:/data \
  --restart always --name redis-${port} --net redis-net \
  --sysctl net.core.somaxconn=1024 redis redis-server /usr/local/etc/redis/redis.conf; \

Where, -- privileged=true means that the container user to be started has true root permission, -- sysctl net.core.somaxconn=1024, which is a linux kernel parameter used to set the size of the request queue. The default value is 128. Subsequent instructions to start redis need to be placed in the request queue first, and then started in turn
After the creation is successful, view the node content through the docker ps instruction.

Step 5: create redis cluster configuration

docker exec -it redis-8010 bash
redis-cli --cluster create --cluster-replicas 1

The above instructions should be executed in one line as far as possible, and the last 1 represents the master-slave ratio. When the selection prompt message appears, enter yes.

Step 6: connect the redis cluster and add data to redis

redis-cli -c -h -p 8010

During the build process, you may need to stop or directly delete the docker container after a problem occurs. You can use the following reference commands

Batch stop docker container:

docker ps -a | grep -i "redis-801*" | awk '{print $1}' | xargs docker stop

Batch delete docker container:

docker ps -a | grep -i "redis-801*" | awk '{print $1}' | xargs docker rm -f

Batch delete files

rm -rf 801{0..5}/conf/redis.conf

Jedis read / write data test

void testJedisCluster()throws Exception{
      Set<HostAndPort> nodes = new HashSet<>();
      nodes.add(new HostAndPort("",8010));
      nodes.add(new HostAndPort("",8011));
      nodes.add(new HostAndPort("",8012));
      nodes.add(new HostAndPort("",8013));
      nodes.add(new HostAndPort("",8014));
      nodes.add(new HostAndPort("",8015));
      JedisCluster jedisCluster = new JedisCluster(nodes);
      //redis using jedisCluster
      jedisCluster.set("test", "cluster");
      String str = jedisCluster.get("test");
      //Close connection pool

RedisTemplate read / write data test
Step 1: configure application.yml, for example:

    cluster: #redis cluster configuration
      max-redirects: 3 #Maximum number of jumps
    timeout: 5000 #Timeout
    database: 0
    jedis: #Connection pool
        max-idle: 8
        max-wait: 0

Step 2: write the unit test class. The code is as follows:

public class RedisClusterTests {
    private RedisTemplate redisTemplate;
    void testMasterReadWrite(){
        //1. Get data operation object
        ValueOperations valueOperations = redisTemplate.opsForValue();
        //2. Read and write data
        Object city=valueOperations.get("city");

Posted by rhunter007 on Mon, 11 Oct 2021 17:12:38 -0700