redis cluster building + spring integration

Keywords: Redis Spring Ruby Jedis

step.1 Compiles redis

wget http://download.redis.io/releases/redis-3.2.0.tar.gz
tar xzf redis-3.2.0.tar.gz
cd redis-3.2.0
make

Step 2. Install ruby environment and ruby package management tool rubygems

yum install ruby
yum install rubygems
whereis gem

Step 3. Create a redis copy

The smallest redis cluster has at least three master s, so there are six copies according to the official website

mkdir redis-cluster
cd redis-cluster
mkdir 7000
mkdir 7001
mkdir 7002
mkdir 7003
mkdir 7004
mkdir 7005
#Copy redis-server and redis.conf to each folder
cp redis-3.2.0/src/redis-server redis-cluster/7000
cp redis-3.2.0/src/redis-server redis-cluster/7001
cp redis-3.2.0/src/redis-server redis-cluster/7002
cp redis-3.2.0/src/redis-server redis-cluster/7003
cp redis-3.2.0/src/redis-server redis-cluster/7004
cp redis-3.2.0/src/redis-server redis-cluster/7005
cp redis-3.2.0/redis.conf redis-cluster/7000
cp redis-3.2.0/redis.conf redis-cluster/7001
cp redis-3.2.0/redis.conf redis-cluster/7002
cp redis-3.2.0/redis.conf redis-cluster/7003
cp redis-3.2.0/redis.conf redis-cluster/7004
cp redis-3.2.0/redis.conf redis-cluster/7005

Modify various port numbers, because they are on a stand-alone computer, to ensure that they are not duplicated. There are also various pid, log, sock file paths, can not be repeated.

#Record what I changed.
#Bid 127.0.0.1 commented out, after all, I'm landing on the host.
protect-mode no  #After all, I'm just playing.
port 7000 #Each redis instance is different
deamonize yes #Background operation
pid ./redis.pid
logfile "./redis.log"
databases 1

cluster-enabled yes #Open cluster
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
#Don't forget to open it. It's a bit like mysql's binlog.
appendonly yes

After the configuration is completed, you can try to start and connect. But don't insert data. Make sure it's empty.

redis-3.2.0/src/redis-cli -p 7000
redis-3.2.0/src/redis-cli -p 7000 shutdown
cat redis-cluster/7000/redis.log

Start all nodes sequentially

cd 7000
./redis-server redis.conf 
cd ../7001
./redis-server redis.conf
cd ../7002
./redis-server redis.conf 
cd ../7003
./redis-server redis.conf 
cd ../7004
./redis-server redis.conf 
cd ../7005
./redis-server redis.conf 

#See if it's all up?
ps -ef

Seen in redis.log for each instance, ok ay

No cluster configuration found, I'm e496af24b91cddf6a7f2f309f7c6e9b4fa52a043

Step 4. Create clusters with ruby scripts

gem install redis
redis-3.2.0/src/redis-trib.rb create --replicas 1 \
127.0.0.1:7000 127.0.0.1:7001 \
127.0.0.1:7002 127.0.0.1:7003 \
127.0.0.1:7004 127.0.0.1:7005

The script will prompt, recommend which is master, which is slave, enter yes to determine (there are pits here, should fill in external ip). replicas specify that each master has a slave.

Seeing [OK] All 16384 slots covered.

Step 5. Testing

redis-3.2.0/src/redis-cli -c -p 7000
127.0.0.1:7000> set foo bar
-> Redirected to slot [12182] located at 127.0.0.1:7002
OK
127.0.0.1:7002> 

- c indicates cluster mode, and then you see a key redirected to 7002 nodes.

redis-3.2.0/src/redis-cli -p 7000 cluster nodes
e496af24b91cddf6a7f2f309f7c6e9b4fa52a043 127.0.0.1:7001 master - 0 1489025337216 2 connected 5461-10922
e0d03ea551c8c1e79c92a7bf16fb966dbd714e25 127.0.0.1:7002 master - 0 1489025335193 3 connected 10923-16383
7a214b35eec50ecdff88a91d5393280f2e189692 127.0.0.1:7003 slave 613f756650e5756b7d4aac2272b2d336e9716810 0 1489025336205 4 connected
1eed98997d240f1d21a819d19c84705d86549ab0 127.0.0.1:7005 slave e0d03ea551c8c1e79c92a7bf16fb966dbd714e25 0 1489025331140 6 connected
27706d69146fdcd5ee6040b82ca11a780ad0eaa2 127.0.0.1:7004 slave e496af24b91cddf6a7f2f309f7c6e9b4fa52a043 0 1489025334182 5 connected
613f756650e5756b7d4aac2272b2d336e9716810 127.0.0.1:7000 myself,master - 0 0 1 connected 0-5460

You can see which master s are.

The embarrassment has come, and the authorities have not said how to stop the cluster! However, cluster-config-file records the state of the cluster, and a single instance can be restarted. Instances can also be added and removed dynamically

#Re-slicing guarantees cluster balance by specifying only one node, and the script will find other nodes automatically.
./redis-trib.rb reshard 127.0.0.1:7000
#To add a new node, the first node is to be added, and the second randomly designated one can be used for configuration.
./redis-trib.rb add-node 127.0.0.1:7006 127.0.0.1:7000
#Add a node as slave 
./redis-trib.rb add-node --slave 127.0.0.1:7006 127.0.0.1:7000
#Delete a node
./redis-trib del-node 127.0.0.1:7000 `<node-id>`
#Change a slave master
redis> CLUSTER REPLICATE <master-node-id>

Step 6. Integrating spring

Introducing dependency

<properties>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<jackson.version>2.1.4</jackson.version>
	<spring.version>4.2.9.RELEASE</spring.version>
</properties>
<dependency>
	<groupId>org.springframework.data</groupId>
	<artifactId>spring-data-redis</artifactId>
	<version>1.7.6.RELEASE</version>
</dependency>
<dependency>
	<groupId>redis.clients</groupId>
	<artifactId>jedis</artifactId>
	<version>2.9.0</version>
</dependency>
<!-- json Parser -->
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-annotations</artifactId>
	<version>${jackson.version}</version>
</dependency>
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-core</artifactId>
	<version>${jackson.version}</version>
</dependency>
<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-databind</artifactId>
	<version>${jackson.version}</version>
</dependency>

Add spring configuration spring-redis.xml, where serialization with json is configured, so jackson is used

<bean id="redisClusterConfiguration"
		class="org.springframework.data.redis.connection.RedisClusterConfiguration">
		<property name="maxRedirects" value="${redis.maxRedirects}"></property>
		<property name="clusterNodes">
			<set>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host1}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port1}"></constructor-arg>
				</bean>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host2}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port2}"></constructor-arg>
				</bean>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host3}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port3}"></constructor-arg>
				</bean>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host4}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port4}"></constructor-arg>
				</bean>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host5}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port5}"></constructor-arg>
				</bean>
				<bean class="org.springframework.data.redis.connection.RedisClusterNode">
					<constructor-arg name="host" value="${redis.host6}"></constructor-arg>
					<constructor-arg name="port" value="${redis.port6}"></constructor-arg>
				</bean> 
			</set>
		</property>
	</bean>
	<bean id="jedisPoolConfig" class="redis.clients.jedis.JedisPoolConfig">
		<property name="maxIdle" value="${redis.maxIdle}" />
		<property name="maxTotal" value="${redis.maxTotal}" />
	</bean>
	<bean id="jeidsConnectionFactory"
		class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory">
		<constructor-arg ref="redisClusterConfiguration" />
		<constructor-arg ref="jedisPoolConfig" />
	</bean>

	<bean id="redisTemplate" class="org.springframework.data.redis.core.RedisTemplate" 
		p:connection-factory-ref="jeidsConnectionFactory" p:default-serializer-ref="serializer"/>
	
	<bean name="ignoreUnrecognizedPropertyMapper" class="cn.bluemobi.hardware.core.utils.IgnoreUnrecognizedPropertyMapper"></bean>
	<bean name="serializer" class="org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer">
		<constructor-arg ref="ignoreUnrecognizedPropertyMapper"></constructor-arg>
	</bean>

You also need a redis.properties

redis.maxRedirects=3
redis.maxIdle=100
redis.maxTotal=600


redis.host1=192.168.137.2
redis.port1=7000
redis.host2=192.168.137.2
redis.port2=7001
redis.host3=192.168.137.2
redis.port3=7002
redis.host4=192.168.137.2
redis.port4=7003
redis.host5=192.168.137.2
redis.port5=7004
redis.host6=192.168.137.2
redis.port6=7005

java code testing

// Initialize spring
ClassPathXmlApplicationContext ctx = new ClassPathXmlApplicationContext("conf/applicationContext.xml");
RedisTemplate<String, Object> template = (RedisTemplate<String, Object>) ctx.getBean("redisTemplate");
        System.out.println(template.keys("*"));

The first time running java failed. When the cluster was created, the ip was 127.0.0.1. Ha-ha, you can change it to the external address. If the cluster is rebuilt, the dump and node-700x files generated before will be deleted and emptied.

redis-3.2.0/src/redis-trib.rb create --replicas 1 \
192.168.137.2:7000 192.168.137.2:7001 \
192.168.137.2:7002 192.168.137.2:7003 \
192.168.137.2:7004 192.168.137.2:7005

Unfortunately, it seems that the jedis cluster configuration cannot set passwords? So it can only be used on the intranet.

Clusters provide sharding, which I need. Duplication seems to be of little use.

redis-cluster does not guarantee 100% data loss! Only slices and availability are provided.

Sharing is based on the hash value of the key. What if I have some keys that I want to hash together?

1. Use transactions.

2. Using curly brackets {} in key, the cluster will only use the string in curly brackets as hash.

Posted by tskweb on Fri, 19 Apr 2019 17:42:33 -0700