Analysis of redis serialization in Springboot

Keywords: SpringBoot JSON Redis Spring

Reasons for the study:

When spring boot uses redis to get data and convert it to objects,
Always prompt: "com.alibaba.fastjson.JSONException: syntax error, expect {, actual [, pos 0"."
Finally, it is found that this is due to the serialization of redis. So let's analyze the way redis is serialized:

1. In spring boot, redis storage data serialization methods are commonly used as follows:

  • String RedisSerializer: Serialize data into strings
  • Jackson 2 json RedisSerializer: Serialize data into json
  • Generic Jackson 2 json RedisSerializer: Serialize data into json
  • JdkSerialization RedisSerializer: Serialize data objects

2. Roughly study the advantages and disadvantages of each serialization method:

  • For String RedisSerializer, it's the most basic. If all data are strings, this serialization method is preferred. The String RedisTemplate provided by spring boot uses this serialization method by default. Note that this serialization only saves strings, not objects
  • Jackson 2Json RedisSerializer can save objects and serialize them into json, but when initializing, you need to specify the class to serialize, you can use Object.class.
  • Generic Jackson 2Json RedisSerializer has the same functionality as Jackson 2Json RedisSerializer, except for performance differences, and Generic Jackson 2Json RedisSerializer does not specify the class to be generalized at initialization.
  • JdkSerialization RedisSerializer, which serializes data into objects, but to save data objects, Serializable must be implemented, otherwise no serialization will be prompted. This serialization method is stored in binary mode in redis, so the readability is very poor. RedisTemplate provided in spring boot defaults to this serialization approach. If you want good readability, you need to change its serialization

3. The performance of each serialization method is good or bad:

Many people on the Internet say that Generic Jackson 2Json RedisSerializer is preferred because Jackson 2Json RedisSerializer needs to specify classes to be generalized. Here's a test of the serialization and deserialization capabilities of Jackson 2Json RedisSerializer, Generic Jackson 2Json RedisSerializer, and JdkSerialization Redisserializer in three serialization modes:

@Slf4j
public class main {

    public static void main(String[] args) {
        JdkSerializationRedisSerializer jdkSeri = new JdkSerializationRedisSerializer();
        GenericJackson2JsonRedisSerializer genJsonSeri = new GenericJackson2JsonRedisSerializer();
        Jackson2JsonRedisSerializer jack2Seri = new Jackson2JsonRedisSerializer(Object.class);

        User user = new User("1","tom",20,"boy");
        List<Object> list = new ArrayList<>();
        for(int i = 0; i < 1000; i++){
            list.add(user);
        }

        log.info("jdkSeri Start of serialization");
        long jdkSeri_Start = System.currentTimeMillis();
        byte[] serialize = jdkSeri.serialize(list);
        log.info("jdkSeri Serialization ends, time-consuming:{}ms,The length after serialization is:{}==============",(System.currentTimeMillis()-jdkSeri_Start),serialize.length);
        log.info("jdkSeri Begin deserialization");
        long jdkSeri_Start_again = System.currentTimeMillis();
        jdkSeri.deserialize(serialize);
        log.info("jdkSeri Deserialization takes time:{}ms===========",(System.currentTimeMillis()-jdkSeri_Start_again));


        log.info("genJsonSeri Start of serialization");
        long genJsonSeri_Start = System.currentTimeMillis();
        byte[] serialize1 = genJsonSeri.serialize(list);
        log.info("genJsonSeri Serialization ends, time-consuming:{}ms,The length after serialization is:{}==============",(System.currentTimeMillis()-genJsonSeri_Start),serialize1.length);
        log.info("genJsonSeri Begin deserialization");
        long genJsonSeri_Start_again = System.currentTimeMillis();
        genJsonSeri.deserialize(serialize1);
        log.info("genJsonSeri Deserialization takes time:{}ms===========",(System.currentTimeMillis()-genJsonSeri_Start_again));


        log.info("jack2Seri Start of serialization");
        long jack2Seri_Start = System.currentTimeMillis();
        byte[] serialize2 = jack2Seri.serialize(list);
        log.info("jack2Seri Serialization ends, time-consuming:{}ms,The length after serialization is:{}===============",(System.currentTimeMillis()-jack2Seri_Start),serialize2.length);
        log.info("jack2Seri Begin deserialization");
        long jack2Seri_Start_again = System.currentTimeMillis();
        jack2Seri.deserialize(serialize2);
        log.info("jack2Seri Deserialization takes time:{}ms=============",(System.currentTimeMillis()-jack2Seri_Start_again));
    }
}
16:29:32.710 [main] INFO com.whj.springboot.config.main-jdkSeri serialization begins
 16:29:32.790 [main] INFO com.whj.springboot.config.main-jdkSeri serialization ends, time consuming: 68ms, and the length after serialization is: 5175==============
16:29:32.798 [main] INFO com.whj.springboot.config.main-jdkSeri began deserialization
 16:29:32.977 [main] INFO com.whj.springboot.config.main-jdkSeri deserialization time-consuming: 179 ms===========
16:29:32.977 [main] INFO com.whj.springboot.config.main-genJsonSeri serialization begins
 16:29:33.237 [main] INFO com.whj.springboot.config.main-genJsonSeri serialization ends, takes 260 ms, and the length after serialization is 87025==============
16:29:33.237 [main] INFO com.whj.springboot.config.main-genJsonSeri began deserialization
 16:29:33.428 [main] INFO com.whj.springboot.config.main-genJsonSeri deserialization time-consuming: 191ms===========
16:29:33.428 [main] INFO com.whj.springboot.config.main-jack2Seri serialization begins
 16:29:33.436 [main] INFO com.whj.springboot.config.main-jack2Seri serialization ends, takes 7 ms, and the length after serialization is 45001===============
16:29:33.436 [main] INFO com.whj.springboot.config.main-jack2Seri began deserialization
 16:29:33.451 [main] INFO com.whj.springboot.config.main-jack2Seri deserialization time-consuming: 15ms=============

Many people on the Internet analyse that it is recommended to use Generic Jackson 2Json RedisSerializer, but through the above tests, we can conclude that:
1. In terms of serialization and deserialization performance, Jackson 2Json RedisSerializer is the most efficient and readable way in redis.
2. In terms of occupying storage space, the length of JdkSerialization RedisSerializer is the smallest after serialization (because it is stored in binary mode)
On the contrary, the Generic Jackson 2Json RedisSerializer approach does not find any bright features (or maybe I know a little too, welcome to discuss)

4. Question Answer

Back to my initial question, because my ValueSerializer uses String RedisSerializer, which converts objects into string savers when they are saved, so that in redis they are treated as ordinary strings, not objects. When the data is fetched and converted into objects using fastJson, the above problem will be reported, because it is not object data, but a string.

5. JSON Conversion Object

1. Transform json data into objects, using com.alibaba.fastjson.JSON.parseObject(str,User.class);
Note: fastJson is powerful, and even if the User object contains object attributes, it can be converted successfully.
2. Convert generic data into objects, using fastjson as well
JSON.parseObject(s,new TypeReference<User>(){})

6. Modify RedisTemplate serialization

@Configuration
public class RedisConfig {

    @Bean
    public RedisTemplate<String,Object> redisTemplate(RedisConnectionFactory redisConnectionFactory){
        /**
         * Configure your redisTemplate
         * StringRedisTemplate Use String RedisSerializer to serialize by default
         * RedisTemplate By default, use JdkSerialization RedisSerializer to serialize
         */
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(redisConnectionFactory);

        //Turn on the default type
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        //Set date format
        objectMapper.setDateFormat(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss"));
        Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
        jackson2JsonRedisSerializer.setObjectMapper(objectMapper);

        redisTemplate.setKeySerializer(new StringRedisSerializer());
        redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }
}

Posted by aperales10 on Tue, 23 Apr 2019 13:45:34 -0700