1, Background
Technology stack: Redis+Lua
Double eleven is a festival that belongs to e-commerce rather than operators. However, goose has no choice but to be on duty. After zero, the flow of the payment system rises for half an hour, and then it runs stably. With a serious and responsible attitude, I'd better stick to it all night. I don't know what to do. In order to keep myself from falling asleep, I think of the most easy way to lose sleep, I wanted to realize the business, so I thought about the needs of the students of the next day's products: increase the transaction quota limit of the merchant dimension, pick up the transaction quota code of the channel dimension that has been realized by the system, and read it... I didn't finish reading it, but I really couldn't see it anymore. The curve saves the country, and the circle is a little big. I don't know how fierce it is, so I wrote it myself.
2, Analysis and Implementation
1. Analysis
Limits are divided into many dimensions, including channel level single limit / cumulative limit, daily cumulative limit / number, monthly cumulative limit / number, transaction type level, etc. Limits are generally a rule condition of the payment routing system. These dimensions are generally adjusted and configured according to the connected payment channel limits. Of course, you can also make limits for your platform merchants. The product demand is to make limits for your platform merchants, so we know what the product wants, If you do it next, it will be a question of how to do it.
The payment system depends on middleware and other systems during operation, including the fewer interactions with the database, the better. Based on this principle, first consider where the data should be stored and what data structure it will be stored in. The quota is only a small logic of payment. First, it must be processed quickly, and its requirements for accuracy are not very high, Almost acceptable. The most important thing is not to affect the main process of the transaction! First of all, if you think about caching, you can't put the database. This is only temporary data, and you don't need to drop the database. If you drop the database, the gain is not worth the loss. Do you use in-process cache or distributed cache? Personally, if the configuration information with small amount of data can put process cache, put process cache. Don't connect everything to redis (you should also put it to the data structure). This will lead to the system's strong dependence on redis. Redis will fail and the whole system will collapse.
However, Redis can only be used in this business, because the data will be calculated and changed with the system transaction at run time, and now the application uses multi node deployment to share this data, so we have determined to use Redis. What is the data structure of Redis? String？ List？ Hash？ Many people only know about string. No matter what type of storage is converted to string, performance is not considered. There are no problems when the system traffic is small. When the traffic is large, the system supports concurrency, long processing time, slow response and insufficient bandwidth.
2. Realization
Next, we will determine which data structure to use is more appropriate. We take Alipay's withholding transactions to support all kinds of bank daily accumulative trading limit, for example, if the type of deducting transactions is different for different bank days, the total number of transactions is limited as follows: 1000 strokes of Bank of China Merchants bank, 500 of Xingye Bank, and 2000 of Construction Bank. This information belongs to the bank information supported by a transaction type under the payment channel. The same level of configuration and bank working hours are not discussed here. What we want to talk about is the storage of the used amount, so how should we store it? Which data structure does Redis choose? First, let's analyze the structure of the data we want to save:
ChanelCode Channel coding(unchanged)+TransType Transaction type code(unchanged)+DeptCode Trading institution/Bank code(change): TransCount Number of transactions(change) Ali+Deduct+cmb: 10 Ali+Deduct+cib: 20 Ali+Deduct+ccb: 15 ......
As we can see, our data is in the form of key:value structure, which is the cumulative number of transactions per bank in the Alipay channel. You can reorganize the following:
2021-11-11 Alipay has withheld trading type support banks have used volume data. 20211111: Ali+Deduct+cmb: 10 20211111: Ali+Deduct+cib: 20 20211111: Ali+Deduct+ccb: 15 ...... 2021-11-12 Alipay has withheld trading type support banks have used volume data. 20211112: Ali+Deduct+cmb: 15 20211112: Ali+Deduct+cib: 25 20211112: Ali+Deduct+ccb: 20 ...... 2021-11-13 Alipay has withheld trading type support banks have used volume data. 20211113: Ali+Deduct+cmb: 5 20211113: Ali+Deduct+cib: 20 20211113: Ali+Deduct+ccb: 40 ......
As mentioned above, the Redis data structure we want to use emerges. If you don't know, you should give up. It should be noted that the above data is temporary data, which should be cleaned up after use, cleaned up after use, and cleaned up after use! Therefore, you must set the timeout for each key. In order to make multiple operations atomic, we use Lua script to implement. One script completes operations such as accumulation and timeout setting.
The following lua script is defined: key refers to the transaction completion time, field refers to channel code + transaction type code + trading institution code, and the combination of increment is 1. Because we do cumulative transactions, it is 1.
private final static String SCRIPT_CHANEL_TRANS_TYPE_DEPT_DAY_TRANS_LIMIT_NUM =
"local key = KEYS[1]\n" +
"local field = ARGV[1]\n" +
"local increment = tonumber(ARGV[2])\n" +
"local oneDayMilliSecond = 86400000\n" +
"\n" +
"local result = redis.call('HINCRBY', key, field, increment)\n" +
"\n" +
"local key_pttl = tonumber(redis.call('PTTL', key))\n" +
"if(key_pttl == -1) then\n" +
" redis.call('PEXPIRE', key, oneDayMilliSecond)\n" +
"end\n" +
"\n" +
"return result";When we do other dimension limits, we just change the values of key, field and increment. The monthly cumulative key is changed to YYYY-MM. For transaction type level limits, change the field to channel code + transaction type code. For amount limits, change the increment to amount. A script solves the basic dimension limits, Therefore, using appropriate technical means in specific business scenarios can make your code concise and efficient.
Just call the incrChanelTransTypeDeptDayLimitNum(String key, String field) method directly. If you encounter similar problems in your work, you can use this solution. Of course, you can also have a better way. Welcome to share.
Implement the source code without posting the git address
Post the main code classes:
/**
* @author Kkk
* @version 1.0
* @date 2021/11/11
* @Describe: Implementation of transaction limit interface
*/
@Service
public class TransLimitServiceImpl implements InitializingBean, TransLimitService {
/** lua Script sha*/
private String chanelTransTypeDeptDayLimitScriptSha = null;
@Resource
private RedisService redisService;
private final static String SCRIPT_CHANEL_TRANS_TYPE_DEPT_DAY_TRANS_LIMIT_NUM =
"local key = KEYS[1]\n" +
"local field = ARGV[1]\n" +
"local increment = tonumber(ARGV[2])\n" +
"local oneDayMilliSecond = 86400000\n" +
"\n" +
"local result = redis.call('HINCRBY', key, field, increment)\n" +
"\n" +
"local key_pttl = tonumber(redis.call('PTTL', key))\n" +
"if(key_pttl == -1) then\n" +
" redis.call('PEXPIRE', key, oneDayMilliSecond)\n" +
"end\n" +
"\n" +
"return result";
@Override
public long incrChanelTransTypeDeptDayLimitNum(String key, String field) {
String hasUsed = String.valueOf(redisService.evalScriptCheck(chanelTransTypeDeptDayLimitScriptSha, SCRIPT_CHANEL_TRANS_TYPE_DEPT_DAY_TRANS_LIMIT_NUM, key, field, "1"));
return Long.parseLong(hasUsed );
}
@Override
public void afterPropertiesSet(){
this.chanelTransTypeDeptDayLimitScriptSha = redisService.loadScript(SCRIPT_CHANEL_TRANS_TYPE_DEPT_DAY_TRANS_LIMIT_NUM);
}
}In order to improve efficiency and reduce bandwidth consumption, cache the script to the server through the SCRIPT LOAD command, and then use the SHA1 of the script Operation through EVALSHA command.
/**
* @author Kkk
* @version 1.0
* @date 2021/11/11
* @Describe:
*/
@Service
public class RedisServiceImpl implements RedisService {
final String KEY_START = "trans:limit:";
@Resource
private JedisPool jedisPool;
@Override
public String generateKey(String key) {
return KEY_START + key;
}
@Override
public String loadScript(String script) {
String sha=jedisPool.getResource().scriptLoad(script);
return sha;
}
@Override
public Object evalSha(String scriptSha, String key, String... args) {
key = generateKey(key);
if (args == null || args.length == 0) {
return jedisPool.getResource().evalsha(scriptSha, 1, key);
} else {
String[] redis_args = new String[1 + args.length];
redis_args[0] = key;
System.arraycopy(args, 0, redis_args, 1, args.length);
return jedisPool.getResource().evalsha(scriptSha, 1, redis_args);
}
}
@Override
public Object evalScriptCheck(String scriptSha, String script, String key, String... args) {
String newSha = scriptSha;
if (!jedisPool.getResource().scriptExists(scriptSha)) {
newSha = loadScript(script);
}
return evalSha(newSha, key, args);
}
}After the code runs, log in to Redis to query the effect:
So far, we have designed and completed our requirements using the hash structure of Redis and Lua script.
3. Summary
Omitted~~~