1, Demand
On the C-side App, whether the user can select distribution within a certain time period requires real-time statistics of the number of orders in each period (every hour or every half an hour) in the background, and determines the distribution period that the user can select by considering the pressure requests of picking and packaging in the warehouse and distribution outside the warehouse.
2, Scheme
The zset (sorted set) data structure of redis is used for real-time statistical analysis.
The redis ordered collection zset, like the collection set, is also a collection of string type elements, and duplicate members are not allowed. The difference is that each element of zset is associated with a score (the score can be repeated). Redis uses the score to sort the members of the set from small to large.
Each warehouse uses a zset key to store every day, orderId as the member of zset, and the expected completion time stamp corresponding to the order as the score of zset;
When the user places an order, the orderId and the expected completion time stamp (zadd) corresponding to the order are recorded.
When the order is cancelled or the delivery is completed, the order (zrem) is deleted from the zset.
To query the number of orders within a time period, you only need to take the start and end timestamp of the time period as the score of zset, and use zcount to count the number of orders
3, zset test
127.0.0.1:6379> # 1. Add. If the value exists, it will be reordered. zadd 127.0.0.1:6379> zadd user 1 wuxiaolong (integer) 1 127.0.0.1:6379> zadd user 10 sukailiang 8 yufeng 5 shilei 6 huangchao (integer) 4 127.0.0.1:6379> # 2. View the number of members of the zset set. zcard 127.0.0.1:6379> zcard user (integer) 5 127.0.0.1:6379> # 3. View the members of the range specified by Zset, with scores for the output results. zrange 127.0.0.1:6379> zrange user 3 8 1) "yufeng" 2) "sukailiang" 127.0.0.1:6379> zrange user 3 8 withscores 1) "yufeng" 2) "8" 3) "sukailiang" 4) "10" 127.0.0.1:6379> zrange user 0 -1 withscores 1) "wuxiaolong" 2) "1" 3) "shilei" 4) "5" 5) "huangchao" 6) "6" 7) "yufeng" 8) "8" 9) "sukailiang" 10) "10" # 4. Get the subscript position of zset member. If the value does not exist, return null. zrank 127.0.0.1:6379> zrank user yufeng (integer) 3 127.0.0.1:6379> zrank user wuxiaolong (integer) 0 127.0.0.1:6379> zrank user nobody (nil) 127.0.0.1:6379> # 5. Get the number of members between the scores specified in the zset set. zcount 127.0.0.1:6379> zcount user 3 8 (integer) 3 127.0.0.1:6379> zcount user 5 8 (integer) 3 # 6. Delete one or more specified members. zrem 127.0.0.1:6379> zrem user yufeng (integer) 1 127.0.0.1:6379> zrem user nobody (integer) 0 127.0.0.1:6379> # 7. Get the score of the specified value. zscore 127.0.0.1:6379> zscore user wuxiaolong "1" 127.0.0.1:6379> zscore user nobody (nil) 127.0.0.1:6379> zscore user shilei "5" 127.0.0.1:6379> # 8. Increase or decrease the score of the specified element. Negative values are minus and positive values are plus. zincrby 127.0.0.1:6379> 127.0.0.1:6379> zincrby user 3 shilei "8" 127.0.0.1:6379> zscore user shilei "8" 127.0.0.1:6379> 127.0.0.1:6379> zrange user 0 -1 withscores 1) "wuxiaolong" 2) "1" 3) "huangchao" 4) "6" 5) "shilei" 6) "8" 7) "sukailiang" 8) "10" 127.0.0.1:6379> 127.0.0.1:6379> # 9. Get the value according to the range of the specified score. zrangebysocre 127.0.0.1:6379> zrangebyscore user 0 8 1) "wuxiaolong" 2) "huangchao" 3) "shilei" 127.0.0.1:6379> # 10. In reverse order, sort from top to bottom and output data in the specified range. zrevrange，zrevrangebyscore 127.0.0.1:6379> zrevrange user 0 -1 1) "sukailiang" 2) "shilei" 3) "huangchao" 4) "wuxiaolong" 127.0.0.1:6379> zrevrange user 0 -1 withscores 1) "sukailiang" 2) "10" 3) "shilei" 4) "8" 5) "huangchao" 6) "6" 7) "wuxiaolong" 8) "1" 127.0.0.1:6379> 127.0.0.1:6379> 127.0.0.1:6379> # 11. Delete data according to the coordinate and fraction range. zremrangebyscore，zremrangebyrank 127.0.0.1:6379> zrevrangebyscore user 8 1 1) "shilei" 2) "huangchao" 3) "wuxiaolong" 127.0.0.1:6379> 127.0.0.1:6379> 127.0.0.1:6379> # 12.zset also has the operation of finding the intersection and union of two sets. zunionzstore，zinterstore
4, JAVA testing
1.controller
package message.queue.engine.controller;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.ZSetOperations;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.Calendar;
import java.util.Date;
import java.util.concurrent.TimeUnit;
@RestController
@RequestMapping(value = "/v1")
public class HelloController {
@Autowired
private ZSetOperations<String, Object> zSetOperations;
@RequestMapping("/add")
public Boolean add(@RequestParam("orderId") String orderId){
// Time of performance
Date deliveryDate = new Date();
// Performance date
String ymd = Utils.getYMD(deliveryDate);
// One key a day
String key = Const.STAT_PREFIX+ ymd;
// If the order number is repeated, orderId will be overwritten
Boolean result = zSetOperations.add(key, orderId, deliveryDate.getTime());
// Store for one week
if(result){
Boolean expire = zSetOperations.getOperations().expire(key, Const.RDS_EXPIRE_DAY, TimeUnit.DAYS);
}
return result;
}
@RequestMapping("/del")
public Long del(@RequestParam("orderId") String orderId){
// Time of performance
Date deliveryDate = new Date();
// Performance date
String ymd = Utils.getYMD(deliveryDate);
// One key a day
String key = Const.STAT_PREFIX+ ymd;
// delete
Long remove = zSetOperations.remove(key, orderId);
return remove;
}
@RequestMapping("/stat")
public Long stat(){
// current time
Date now = new Date();
// The length of the time period is divided into 30 minutes
int segmentType = 30;
// Performance date
String ymd = Utils.getYMD(now);
// One key a day
String key = Const.STAT_PREFIX+ ymd;
// Starting Today
Calendar calendar = Calendar.getInstance();
calendar.setTime(now);
calendar.set(Calendar.HOUR_OF_DAY, 0);
calendar.set(Calendar.MINUTE, 0);
calendar.set(Calendar.SECOND, 0);
Date todayStart = calendar.getTime();
// Relative time period
Long relativeTime = now.getTime() - todayStart.getTime();
long[] segmentArr;
if(segmentType ==30){
segmentArr = Const.TIME_SEGMENT_30;
}else {
segmentArr = Const.TIME_SEGMENT_60;
}
Long start = null;
Long end = null;
for (int i=0; i< segmentArr.length-1; i++){
if(relativeTime >= segmentArr[i] && relativeTime <= segmentArr[i+1]){
start = segmentArr[i] + todayStart.getTime();
end = segmentArr[i+1] + todayStart.getTime();
}
}
// todo start=null
// Count order quantity according to time period
Long count = zSetOperations.count(key, start, end);
System.out.println(start +" " + end +" " + count);
return count;
}
}
2.Constant
package or.message.queue.engine.constant;
public class Const {
public static final String STAT_PREFIX = "eta:order:stat:zset:";
public static final String ORDER_PREFIX = "eta:order:";
public static final Integer RDS_EXPIRE_DAY = 7;
public static final long[] TIME_SEGMENT_30 = {
0,
1800000, 3600000,
5400000, 7200000,
9000000, 10800000,
12600000, 14400000,
16200000, 18000000, // 6 hours
19800000, 21600000,
23400000, 25200000,
27000000, 28800000,
30600000, 32400000,
34200000, 36000000, // 10 hours
37800000, 39600000,
41400000, 43200000, // 12
45000000, 46800000,
48600000, 50400000,
52200000, 54000000,
55800000, 57600000,
59400000, 61200000,
63000000, 64800000, // 18
66600000, 68400000,
70200000, 72000000, // 20 hours
73800000, 75600000,
77400000, 79200000,
81000000, 82800000,
84600000, 86400000, // 24 hours
};
public static final long[] TIME_SEGMENT_60 = {
0,
3600000,
7200000,
10800000,
14400000,
18000000, // 6 hours
21600000,
25200000,
28800000,
32400000,
36000000, // 10 hours
39600000,
43200000, // 12
46800000,
50400000,
54000000,
57600000,
61200000,
64800000, // 18
68400000,
72000000, // 20 hours
75600000,
79200000,
82800000,
86400000, // 24 hours
};
}
5, Extension of zset
-
zset as delay queue
zset will sort by score, if score represents the timestamp of the desired execution time. When it is inserted into the zset set at a certain time, it will be sorted according to the timestamp size, that is, before and after the execution time.
Start an endless loop thread and continuously take the first key value. If the current timestamp is greater than or equal to the score of the key value, it will be taken out for consumption deletion, which can achieve the purpose of delaying execution
-
Ranking List
It is not easy to distinguish between real-time statistics if the list of "the most popular in one hour" is recorded in the database. We take the timestamp of the current hour as the key of zset, the post ID as the member, the number of hits, comments, etc. as the score, and update the score when the score changes. Use ZREVRANGE or ZRANGE to find the corresponding number of records.