Spring Kafka - KafkaListener starts and stops regularly

Keywords: PHP kafka Database

I. application scenario of timing start

For example, in the stand-alone environment, we need to use Kafka for data persistence. Since the active time of users is from 10 a.m. to 12 p.m., a large amount of data persistence in this time period may affect the performance of the database and lead to the decrease of user experience. We can choose to do the persistence operation in the period of low user activity, that is, 12 p.m. After o'clock and before 10 o'clock in the morning of the second item.
 

II. Implementation ideas

  1. Disable KafkaListener self startup (AutoStartup)
  2. Write two scheduled tasks, 12:00 p.m. and 10:00 a.m.
  3. Start KafkaListener at 12:00 and close KafkaListener at 10:00

Here, we need to pay attention to the method of starting the listening container. When the project is started, the listening container is not started, while resume means recovery, not start. So we need to judge whether the container is running. If it is running, call the resume method, otherwise call the start method.

 

III. implementation code

@Component
@EnableScheduling
public class TaskListener{

    private static final Logger log= LoggerFactory.getLogger(TaskListener.class);

    @Autowired
    private KafkaListenerEndpointRegistry registry;

    @Autowired
    private ConsumerFactory consumerFactory;

    @Bean
    public ConcurrentKafkaListenerContainerFactory delayContainerFactory() {
        ConcurrentKafkaListenerContainerFactory container = new ConcurrentKafkaListenerContainerFactory();
        container.setConsumerFactory(consumerFactory);
        //Disable auto start
        container.setAutoStartup(false);
        return container;
    }

    @KafkaListener(id = "durable", topics = "topic.quick.durable",containerFactory = "delayContainerFactory")
    public void durableListener(String data) {
        //Do data persistence here
        log.info("topic.quick.durable receive : " + data);
    }


    //Timer, start monitoring at 0 a.m. every day
    @Scheduled(cron = "0 0 0 * * ?")
    public void startListener() {
        log.info("Turn on monitoring");
        //Judge whether the monitoring container is started. If it is not started, start it.
        if (!registry.getListenerContainer("durable").isRunning()) {
            registry.getListenerContainer("durable").start();
        }
        registry.getListenerContainer("durable").resume();
    }

    //Timer, turn off monitoring at 10:00 every morning
    @Scheduled(cron = "0 0 10 * * ?")
    public void shutDownListener() {
        log.info("lsnrctl stop");
        registry.getListenerContainer("durable").pause();
    }



}

 

 

Reference resources:

  https://www.jianshu.com/p/2447592ca5a9

Posted by strago on Mon, 28 Oct 2019 14:15:34 -0700