Second kill system based on SpringBoot, Redis and RabbitMQ

Keywords: RabbitMQ Redis Spring Boot

Second kill system based on SpringBoot, Redis and RabbitMQ

The main problems solved by seckill system: concurrent reading and concurrent writing. The core optimization concept of concurrent reading is to minimize the number of users "reading" data from the server, or let them read less data; The processing principle of concurrent write is the same. It requires us to separate a database at the database level for special processing.

Login function summary

1, Twice MD5 encryption

  1. Front end: password = MD5 (clear text + fixed Salt)
  2. Back end: password = MD5 (user input + random Salt)

The client MD5 encryption is to prevent the user password from being transmitted in plaintext in the network, and the server MD5 encryption is to improve the password security and double insurance.

  1. Introducing pom.xml

    <!-- md5 rely on -->
    <dependency>
        <groupId>commons-codec</groupId>
        <artifactId>commons-codec</artifactId>
    </dependency>
    
  2. Write MD5 tool class

    public class MD5Util {
    
        public static String md5(String src) {
            return DigestUtils.md5Hex(src);
        }
    
        private static final String salt = "1a2b3c4d";
    
        public static String inputPassToFormPass(String inputPass) {
            String str = ""+salt.charAt(0)+salt.charAt(2) + inputPass +salt.charAt(5) + salt.charAt(4);
            return md5(str);
        }
    
        public static String formPassToDBPass(String formPass, String salt) {
            String str = ""+salt.charAt(0)+salt.charAt(2) + formPass +salt.charAt(5) + salt.charAt(4);
            return md5(str);
        }
    
        public static String inputPassToDbPass(String inputPass, String saltDB) {
            String formPass = inputPassToFormPass(inputPass);
            String dbPass = formPassToDBPass(formPass, saltDB);
            return dbPass;
        }
    
        public static void main(String[] args) {
            System.out.println(inputPassToFormPass("123456"));//d3b1294a61a07da9b49b6e22b2cbd7f9
            System.out.println(formPassToDBPass(inputPassToFormPass("123456"), "1a2b3c4d"));
            System.out.println(inputPassToDbPass("123456", "1a2b3c4d"));
        }
    }
    

2, Parameter verification

Each class is too cumbersome to write a lot of robustness judgments. We can use validation to simplify our code

  1. pom.xml

    <!-- validation assembly -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-validation</artifactId>
    </dependency>
    
  2. Custom mobile phone number verification rules

    /**
     * Mobile phone number verification rules
     */
    public class IsMobileValidator implements ConstraintValidator<IsMobile,String> {
    
        private boolean required = false;
    
        @Override
        public void initialize(IsMobile constraintAnnotation) {
            required = constraintAnnotation.required();
        }
    
        @Override
        public boolean isValid(String value, ConstraintValidatorContext context) {
            if (required){
                return ValidatorUtil.isMobile(value);
            }else {
                if (StringUtils.isEmpty(value)){
                    return true;
                }else {
                    return ValidatorUtil.isMobile(value);
                }
            }
        }
    }
    
  3. Custom annotation IsMobile

    /**
     * Custom annotation verification mobile number
     */
    @Target({ METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER, TYPE_USE })
    @Retention(RUNTIME)
    @Documented
    @Constraint(validatedBy = {IsMobileValidator.class})
    public @interface IsMobile {
        boolean required() default true;
        String message() default "Mobile phone number format error";
        Class<?>[] groups() default {};
        Class<? extends Payload>[] payload() default {};
    }
    
  4. Verify mobile phone number tool class

    /**
     * Verify mobile phone number tool class
     */
    public class ValidatorUtil {
    
        private static final Pattern mobile_pattern = Pattern.compile("[1]([3-9])[0-9]{9}$");
        public static boolean isMobile(String mobile){
            if (StringUtils.isEmpty(mobile)) {
                return false;
            }
            Matcher matcher = mobile_pattern.matcher(mobile);
            return matcher.matches();
        }
    }
    
  5. Usage: add it directly to the field to be verified

    @Data
    @NoArgsConstructor
    @AllArgsConstructor
    public class LoginVo {
    
        @NotNull
        @IsMobile
        private String mobile;
    
        @NotNull
        @Length(min = 32)
        private String password;
    
    }
    
  6. Add @ Valid to the input parameter of login method

    /**
    * Sign in
    * @return
    */
    @RequestMapping("/doLogin")
    @ResponseBody
    public RespBean doLogin(@Valid LoginVo loginVo) {
       log.info(loginVo.toString());
       return userService.login(loginVo);
    }
    

3, Record user information

The most common is to use cookie+session to record user information

  1. Cookie tool class

    /**
     * Cookie Tool class
     */
    public class CookieUtil {
    
        /**
         * Get the value of the Cookie without encoding
         *
         * @param request
         * @param cookieName
         * @return
         */
        public static String getCookieValue(HttpServletRequest request, String cookieName) {
            return getCookieValue(request, cookieName, false);
        }
    
        /**
         * Get the value of the Cookie,
         *
         * @param request
         * @param cookieName
         * @return
         */
        public static String getCookieValue(HttpServletRequest request, String cookieName, boolean isDecoder) {
            Cookie[] cookieList = request.getCookies();
            if (cookieList == null || cookieName == null) {
                return null;
            }
            String retValue = null;
            try {
                for (int i = 0; i < cookieList.length; i++) {
                    if (cookieList[i].getName().equals(cookieName)) {
                        if (isDecoder) {
                            retValue = URLDecoder.decode(cookieList[i].getValue(),
                                    "UTF-8");
                        } else {
                            retValue = cookieList[i].getValue();
                        }
                        break;
                    }
                }
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }
            return retValue;
        }
    
        /**
         * Get the value of the Cookie,
         *
         * @param request
         * @param cookieName
         * @return
         */
        public static String getCookieValue(HttpServletRequest request, String
                cookieName, String encodeString) {
            Cookie[] cookieList = request.getCookies();
            if (cookieList == null || cookieName == null) {
                return null;
            }
            String retValue = null;
            try {
                for (int i = 0; i < cookieList.length; i++) {
                    if (cookieList[i].getName().equals(cookieName)) {
                        retValue = URLDecoder.decode(cookieList[i].getValue(),
                                encodeString);
                        break;
                    }
                }
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
            }
            return retValue;
        }
    
        /**
         * Setting the value of the Cookie does not set the effective time. The default browser will expire when it is closed and will not be encoded
         */
        public static void setCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue) {
            setCookie(request, response, cookieName, cookieValue, -1);
        }
    
        /**
         * Set the value of the Cookie to take effect within the specified time, but do not encode
         */
        public static void setCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue, int cookieMaxage) {
            setCookie(request, response, cookieName, cookieValue, cookieMaxage, false);
        }
    
        /**
         * Setting the value of the Cookie does not set the effective time, but the encoding
         */
        public static void setCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue, boolean isEncode) {
            setCookie(request, response, cookieName, cookieValue, -1, isEncode);
        }
    
        /**
         * Set the value of the Cookie to take effect within the specified time, encoding parameters
         */
        public static void setCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue, int cookieMaxage, boolean isEncode) {
            doSetCookie(request, response, cookieName, cookieValue, cookieMaxage, isEncode);
        }
    
        /**
         * Set the value of the Cookie to take effect within the specified time, encoding parameter (specify encoding)
         */
        public static void setCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue, int cookieMaxage, String encodeString) {
            doSetCookie(request, response, cookieName, cookieValue, cookieMaxage, encodeString);
        }
    
        /**
         * Delete cookie domain name with cookie
         */
        public static void deleteCookie(HttpServletRequest request, HttpServletResponse response, String cookieName) {
            doSetCookie(request, response, cookieName, "", -1, false);
        }
    
        /**
         * Set the value of the Cookie and make it effective within the specified time
         *
         * @param cookieMaxage cookie Maximum seconds in effect
         */
        private static final void doSetCookie(HttpServletRequest request, HttpServletResponse response, String cookieName, String cookieValue, int cookieMaxage, boolean isEncode) {
            try {
                if (cookieValue == null) {
                    cookieValue = "";
                } else if (isEncode) {
                    cookieValue = URLEncoder.encode(cookieValue, "utf-8");
                }
                Cookie cookie = new Cookie(cookieName, cookieValue);
                if (cookieMaxage > 0)
                    cookie.setMaxAge(cookieMaxage);
                if (null != request) {// Set cookie s for domain names
                    String domainName = getDomainName(request);
                    System.out.println(domainName);
                    if (!"localhost".equals(domainName)) {
                        cookie.setDomain(domainName);
                    }
                }
                cookie.setPath("/");
                response.addCookie(cookie);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        /**
         * Set the value of the Cookie and make it effective within the specified time
         *
         * @param cookieMaxage cookie Maximum seconds in effect
         */
        private static final void doSetCookie(HttpServletRequest request,
                                              HttpServletResponse response,
                                              String cookieName, String cookieValue,
                                              int cookieMaxage, String encodeString) {
            try {
                if (cookieValue == null) {
                    cookieValue = "";
                } else {
                    cookieValue = URLEncoder.encode(cookieValue, encodeString);
                }
                Cookie cookie = new Cookie(cookieName, cookieValue);
                if (cookieMaxage > 0) {
                    cookie.setMaxAge(cookieMaxage);
                }
                if (null != request) {// Set cookie s for domain names
                    String domainName = getDomainName(request);
                    System.out.println(domainName);
                    if (!"localhost".equals(domainName)) {
                        cookie.setDomain(domainName);
                    }
                }
                cookie.setPath("/");
                response.addCookie(cookie);
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    
        /**
         * Get the domain name of the cookie
         */
        private static final String getDomainName(HttpServletRequest request) {
            String domainName = null;
            // Get the url address of the access through the request object
            String serverName = request.getRequestURL().toString();
            if (serverName == null || serverName.equals("")) {
                domainName = "";
            } else {
                // Convert url to lowercase
                serverName = serverName.toLowerCase();
                // If the url address starts with http: / /, intercept http: / /
                if (serverName.startsWith("http://")) {
                    serverName = serverName.substring(7);
                }
                int end = serverName.length();
                // Determine whether the url address contains "/"
                if (serverName.contains("/")) {
                    //Get the position where the first "/" appears
                    end = serverName.indexOf("/");
                }
                // intercept
                serverName = serverName.substring(0, end);
                // Split according to "."
                final String[] domains = serverName.split("\\.");
                int len = domains.length;
                if (len > 3) {
                    // www.xxx.com.cn
                    domainName = domains[len - 3] + "." + domains[len - 2] + "." +
                            domains[len - 1];
                } else if (len <= 3 && len > 1) {
                    // xxx.com or xxx.cn
                    domainName = domains[len - 2] + "." + domains[len - 1];
                } else {
                    domainName = serverName;
                }
            }
            if (domainName != null && domainName.indexOf(":") > 0) {
                String[] ary = domainName.split("\\:");
                domainName = ary[0];
            }
            return domainName;
        }
    }
    
  2. UUID utility class

    /**
     * UUID Tool class
     */
    public class UUIDUtil {
        public static String uuid() {
            return UUID.randomUUID().toString().replace("-", "");
        }
    }
    
  3. Add the following code to the login service

    //Generate cookies
    String ticket = UUIDUtil.uuid();
    request.getSession().setAttribute(ticket,user);
    CookieUtil.setCookie(request,response,"userTicket",ticket);
    

4, Distributed Session

In the previous code, if all operations are on a Tomcat, there is no problem. When we deploy multiple systems and cooperate with Nginx, the problem of user login will occur. Because Nginx uses the default load balancing policy (polling), the requests will be distributed to the back-end applications one by one in chronological order. In other words, at the beginning, after we log in to Tomcat1, the user information is placed in the Session of Tomcat1. After a while, the request was distributed to Tomcat2 by Nginx. At this time, there was no user information in the Session on Tomcat2, so I had to log in again.

4.1. Spring Session cooperates with Redis to realize distributed Session

  1. pom.xml

    <!-- spring data redis rely on -->
    <dependency>
    	<groupId>org.springframework.boot</groupId>
    	<artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <!-- commons-pool2 Object pool dependency -->
    <dependency>
    	<groupId>org.apache.commons</groupId>
    	<artifactId>commons-pool2</artifactId>
    </dependency>
    <!-- spring-session rely on -->
    <dependency>
    	<groupId>org.springframework.session</groupId>
    	<artifactId>spring-session-data-redis</artifactId>
    </dependency>
    
  2. Configure redis and modify application.yml

    spring:
     redis:
       #Timeout
       timeout: 10000ms
       #server address
       host: xxx.xxx.xxx.xxx
       #Server port
       port: 6379
       #database
       database: 0
       #password
       password: root
       lettuce:
         pool:
           #Maximum connections, 8 by default
           max-active: 1024
           #Maximum connection blocking waiting time, default - 1
           max-wait: 10000ms
           #Maximum idle connections
           max-idle: 200
           #Minimum idle connection
           min-idle: 5
    
  3. After completing the above operations, user login will generate data related to user information in redis

4.2. Directly store user information in Redis

  1. Redis configuration class RedisConfig.java

    /**
     * Redis Configuration class
     */
    @Configuration
    public class RedisConfig {
    
        @Bean
        public RedisTemplate<String,Object> redisTemplate(RedisConnectionFactory connectionFactory){
            RedisTemplate<String,Object> redisTemplate = new RedisTemplate<>();
            //key sequencer
            redisTemplate.setKeySerializer(new StringRedisSerializer());
            //value sequencer
            redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());
            //Hash type key sequencer
            redisTemplate.setHashKeySerializer(new StringRedisSerializer());
            //Hash type value sequencer
            redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
            redisTemplate.setConnectionFactory(connectionFactory);
            return redisTemplate;
        }
    }
    
  2. Modify the code that added the cookie before

    //Generate cookies
    String ticket = UUIDUtil.uuid();
    redisTemplate.opsForValue().set("user:" + ticket, user);
    // request.getSession().setAttribute(ticket,user);
    CookieUtil.setCookie(request,response,"userTicket",ticket);
    

There is a disadvantage at this time: after logging in, the user information is directly stored in Redis without setting the expiration time. After the cookie expires, re logging in will generate a new cookie and add it to Redis.

5, Optimize login function

Imagine that after a user logs in and jumps to other pages at will. Do you have to pass in a cookie to determine whether the user exists? In this way, repeat trivial operations. You can use the MVC configuration class to verify before passing in parameters. If this layer is not optimized, the order created in the subsequent second kill function will have no user id because the user information passed in earlier cannot be obtained.

  1. UserArgumentResolver.java

    @Component
    public class UserArgumentResolver implements HandlerMethodArgumentResolver {
    
        @Autowired
        private IUserService userService;
    
        @Override
        public boolean supportsParameter(MethodParameter parameter) {
            Class<?> clazz = parameter.getParameterType();
            return clazz == User.class;
        }
    
        @Override
        public Object resolveArgument(MethodParameter parameter,
                                      ModelAndViewContainer mavContainer,
                                      NativeWebRequest webRequest,
                                      WebDataBinderFactory binderFactory) throws Exception {
            HttpServletRequest request = webRequest.getNativeRequest(HttpServletRequest.class);
            HttpServletResponse response = webRequest.getNativeResponse(HttpServletResponse.class);
            String ticket = CookieUtil.getCookieValue(request, "userTicket");
            if (StringUtils.isEmpty(ticket)) {
                return null;
            }
            return userService.getUserByCookie(ticket, request, response);
        }
    }
    
  2. MVC configuration class WebConfig.java

    @Configuration
    @EnableWebMvc
    public class WebConfig implements WebMvcConfigurer {
    
        @Autowired
        private UserArgumentResolver userArgumentResolver;
    
        @Override
        public void addArgumentResolvers(List<HandlerMethodArgumentResolver> resolvers) {
            resolvers.add(userArgumentResolver);
        }
        
    }
    

Posted by swissmissflash on Wed, 10 Nov 2021 10:26:18 -0800