Video directions 👉 Station B dark horse micro service Super recommended!

MQ implements data synchronization between elasticsearch and mysql

The hotel data in elasticsearch comes from MySQL database. Therefore, when MySQL data changes, elasticsearch must also change. This is the data synchronization between elasticsearch and mysql.

1. Train of thought analysis

There are three common data synchronization schemes:

• Synchronous call
• Asynchronous notification
• Listen to binlog

1.1 synchronous call

Scheme 1: synchronous call

The basic steps are as follows:

• The hotel demo provides an external interface to modify the data in elastic search
• After completing the database operation, the hotel management service directly calls the interface provided by the hotel demo

1.2 asynchronous notification

Scheme 2: asynchronous notification

The process is as follows:

• After adding, deleting and modifying the mysql database data, hotel admin sends an MQ message
• The hotel demo listens to MQ and completes the elasticsearch data modification after receiving the message

1.3 monitoring binlog

Scheme 3: monitoring binlog

The process is as follows:

• Enable binlog function for mysql
• The addition, deletion and modification operations completed by mysql will be recorded in binlog
• Hotel demo monitors binlog changes based on canal and updates the contents of elastic search in real time

1.4 selection

Method 1: synchronous call

• Advantages: simple implementation, rough
• Disadvantages: high business coupling

Mode 2: asynchronous notification

• Advantages: low coupling and general difficulty in implementation
• Disadvantages: reliability dependent on mq

Method 3: listen to binlog

• Advantage: completely decouple services
• Disadvantages: opening binlog increases the burden on the database and the implementation complexity is high

2.MQ data synchronization

2.1 ideas

The hotel admin project provided by the pre class materials is used as the micro service of hotel management. When the hotel data is added, deleted or modified, the same operation is required for the data in elasticsearch.

Steps:

• Import the hotel admin project provided in the pre class materials, start and test the CRUD of hotel data

• Declare exchange, queue, RoutingKey

• Complete message sending in the add, delete and modify business in hotel admin

• Complete message listening in hotel demo and update the data in elastic search

• Start and test the data synchronization function

2.2 declaring switches and queues

The MQ structure is shown in the figure below:

(1) Introduce dependency

Introduce the dependency of rabbitmq in hotel admin and hotel Demo:

<!--amqp-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

(2) Declare queue switch name

Create a new class MqConstants under cn.itcast.hotel.constantnts package in hotel admin and hotel Demo:

package cn.itcast.hotel.constatnts;

    public class MqConstants {
    /**
     * Switch
     */
    public final static String HOTEL_EXCHANGE = "hotel.topic";
    /**
     * Listen to new and modified queues
     */
    public final static String HOTEL_INSERT_QUEUE = "hotel.insert.queue";
    /**
     * Listening to deleted queues
     */
    public final static String HOTEL_DELETE_QUEUE = "hotel.delete.queue";
    /**
     * Added or modified RoutingKey
     */
    public final static String HOTEL_INSERT_KEY = "hotel.insert";
    /**
     * Deleted RoutingKey
     */
    public final static String HOTEL_DELETE_KEY = "hotel.delete";
}

(3) Claim queue switch

In the hotel demo, define the configuration class MqConfig to declare the queue, switch, binding queue and switch:

@Configuration
public class MqConfig {
    /**
     * Define switch
     * @return
     */
    @Bean
    public TopicExchange topicExchange() {
        return new TopicExchange(MqConstants.HOTEL_EXCHANGE, true, false);
    }

    /**
     * Define new or modified queues
     * @return
     */
    @Bean
    public Queue insertQueue() {
        return new Queue(MqConstants.HOTEL_INSERT_QUEUE, true);
    }

    /**
     * Define deleted queues
     * @return
     */
    @Bean
    public Queue deleteQueue() {
        return new Queue(MqConstants.HOTEL_DELETE_QUEUE, true);
    }

    /**
     * Bind modify queue and switch
     * @return
     */
    @Bean
    public Binding insertQueueBinding() {
        return BindingBuilder.bind(insertQueue())
                .to(topicExchange())
                .with(MqConstants.HOTEL_INSERT_KEY);
    }

    /**
     * Bind delete queue and switch
     * @return
     */
    @Bean
    public Binding deleteQueueBinding() {
        return BindingBuilder.bind(deleteQueue())
                .to(topicExchange())
                .with(MqConstants.HOTEL_DELETE_KEY);
    }
}

2.4 sending MQ messages

When adding, deleting and modifying services occur in hotel admin, MQ messages are sent respectively:

The id of the hotel to be passed in to the hotel demo for modification

@PostMapping
public void saveHotel(@RequestBody Hotel hotel){
    // New hotels
    hotelService.save(hotel);
    // Send MQ message
    rabbitTemplate.convertAndSend(HotelMqConstants.EXCHANGE_NAME, HotelMqConstants.INSERT_KEY, hotel.getId());
}

@PutMapping()
public void updateById(@RequestBody Hotel hotel){
    if (hotel.getId() == null) {
        throw new InvalidParameterException("id Cannot be empty");
    }
    hotelService.updateById(hotel);
    // Send MQ message
    rabbitTemplate.convertAndSend(HotelMqConstants.EXCHANGE_NAME, HotelMqConstants.INSERT_KEY, hotel.getId());
}

@DeleteMapping("/{id}")
public void deleteById(@PathVariable("id") Long id) {
    hotelService.removeById(id);
    // Send MQ message
    rabbitTemplate.convertAndSend(HotelMqConstants.EXCHANGE_NAME, HotelMqConstants.DELETE_KEY, id);
}

2.5 accept MQ messages

When the hotel demo receives an MQ message, it needs to do the following:

• Add message: query the hotel information according to the passed Hotel id, and then add a piece of data to the index library
• Delete message: delete a piece of data in the index library according to the id of the passed hotel

(1) First, add and delete services in IHotelService under cn.itcast.hotel.service package of hotel demo

void deleteById(Long id);
void insertById(Long id);

(2) Implement business in HotelService under cn.itcast.hotel.service.impl package in hotel Demo:

@Override
public void deleteById(Long id) {
    try {
        // 1. Prepare Request
        DeleteRequest request = new DeleteRequest("hotel", id.toString());
        // 2. Send request
        client.delete(request, RequestOptions.DEFAULT);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

@Override
public void insertById(Long id) {
    try {
        // 0. Query hotel data according to id
        Hotel hotel = getById(id);
        // Convert to document type
        HotelDoc hotelDoc = new HotelDoc(hotel);

        // 1. Prepare the Request object
        IndexRequest request = new IndexRequest("hotel").id(hotel.getId().toString());
        // 2. Prepare Json documents
        request.source(JSON.toJSONString(hotelDoc), XContentType.JSON);
        // 3. Send request
        client.index(request, RequestOptions.DEFAULT);
    } catch (IOException e) {
        throw new RuntimeException(e);
    }
}

(3) Write listener

Add a new class in cn.itcast.hotel.mq package in hotel Demo:

@Component
public class Hotelistener {
    @Autowired
    private IHotelService iHotelServicel;
    /**
     * Monitor the newly added or modified business of the hotel
     * @param id Hotel id
     */
    @RabbitListener(queues = MqConstants.HOTEL_INSERT_QUEUE)
    public void listenHotelInsertOrupdate(Long id){
        iHotelServicel.saveById(id);
    }
    /**
     * Monitor the business deleted by the hotel
     * @param id Hotel id
     */
    @RabbitListener(queues = MqConstants.HOTEL_DELETE_QUEUE)
    public void listenHotelDelete(Long id){
        iHotelServicel.deleteById(id);
    }
}

Last favorite little partner, remember the third company! 😏🍭😘