person one is in love with
Recently, I met such a demand in the company's development: how to realize regular order? And you can interrupt, restart and delete this task at any time. I thought of the following technologies at that time
- @Scheduling annotation + Cron expression. Hahahahahaha, maybe many small partners will do the same, but later think about it carefully. This method will only execute tasks regularly, but will not interrupt, not to mention the requirements of interrupting, restarting and deleting specified tasks. Pass directly
- Then I thought it could be realized with the help of message queue. The user creates a new scheduled task, first receives the task, and then asynchronously sends the task message (sets the expiration time) to the Mq dead letter queue. After the message in the dead letter queue times out, it will be automatically forwarded to other queues. The Mq listener listening to other queues can consume this message and execute our business logic. In this way, the regular order task is realized. It's also easy to delete scheduled tasks. When you want to delete a task, you can directly kill the message in Mq, so that the listener can't consume this message. But the interrupt restart task is a little hard to implement with Mq. Pass directly
- With the help of Redis, you can refer to this article I wrote earlier redis implements delay queue
- Finally, I compromised and used a task-specific framework Quatz to achieve the above requirements
text
When I used Rabbitmq to implement the requirements in the opening article and wrote the test Demo, I thought about the following problems, which is why I wrote this article. If these three problems have corresponding processing mechanisms in our project, the project is still very robust
- What if the message fails to be sent?
- What if a message fails to be consumed?
- What if the Rabbitmq service hangs up?
MQ message confirmation mechanism
First, let's discuss the first problem. If we send another message to MQ and the message is not received by MQ for various reasons, it is obvious that this process is abnormal. So how can we ensure that our messages will reach the message queue? In fact, Rabbitmq provides a message confirmation mechanism for the client through ConfirmCallback and ReturnCallback
The interface implements real-time monitoring of the message sending process. If the message successfully arrives at MQ, MQ will mark the message: ACK = true and return it to us (meaning that the message is sent successfully). Through this mark, we can do subsequent processing operations on whether the message is successfully sent.
That's all for the theory. Now let's practice it. First, add the dependence of RabbitMq
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
Then open the support for related callback functions
spring:
rabbitmq:
host: 127.0.0.1
username: guest
password: guest
port: 5672
listener:
simple:
#Enable message manual ACK
acknowledge-mode: manual
#The callback method will be triggered if the open message arrives at the switch and does not arrive at the queue
publisher-returns: true
#The message to the switch triggers a callback method
publisher-confirm-type: correlated
Write relevant test code. Readers can also implement the ConfirmCallback and ReturnCallback interfaces into a class by themselves. They like to write them in a class. It is worth noting that the following code has the function that if sending a message fails, it will retry sending the message three times, and the reason for retrying sending the message will be recorded in the log. If the message is always sent to MQ fails, the status of the scheduled task in the modification library will be invalid. Readers can replace it according to their own business logic
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.AmqpException;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.core.MessageDeliveryMode;
import org.springframework.amqp.core.MessagePostProcessor;
import org.springframework.amqp.core.MessageProperties;
import org.springframework.amqp.rabbit.connection.CorrelationData;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.support.converter.Jackson2JsonMessageConverter;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.concurrent.atomic.AtomicInteger;
@Slf4j
@Service
public class NewsImpl implements RabbitTemplate.ConfirmCallback, NewsService, RabbitTemplate.ReturnCallback {
private AtomicInteger i = new AtomicInteger(0);
@Autowired
private RabbitTemplate rabbitTemplate;
public void senMsgToRabbitMq(Order data) {
String CorrelationData = JSONObject.toJSON(data).toString();
rabbitTemplate.setExchange("cdTopicExchange");
rabbitTemplate.setRoutingKey("user.dead");
rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
//When the open message does not reach the queue, the ReturnCallback is triggered
rabbitTemplate.setMandatory(true);
//Specifies the callback function that will be triggered when the message arrives at the switch
rabbitTemplate.setConfirmCallback(this);
//Specifies the callback function when the message reaches the switch but does not reach the queue
rabbitTemplate.setReturnCallback(this);
rabbitTemplate.convertAndSend(data, new MessagePostProcessor() {
@Override
public Message postProcessMessage(Message message) throws AmqpException {
MessageProperties msgProperties = message.getMessageProperties();
msgProperties.setCorrelationId("2");
//Set expiration time: 4s
msgProperties.setExpiration("4000");
//Set persistence
msgProperties.setDeliveryMode(MessageDeliveryMode.PERSISTENT);
return message;
}
}, new CorrelationData(CorrelationData));
}
@SneakyThrows
@Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.out.println();
if (ack) i.compareAndSet(i.get(), 0);
else {
if (i.get() < 3) {
Thread.sleep(1000);
Order data = JSONObject.parseObject(correlationData.getId(), Order.class);
System.err.println(i.get()+"Resend order message to MQ: " + data);
this.senMsgToRabbitMq(data);
i.compareAndSet(i.get(), i.get() + 1);
System.err.println(cause);
} else {
System.err.println("rabbitmq If the message is always not received, send the error log to the administrator");
System.err.println("Modify the scheduled task status as invalid");
}
}
}
@Override
public void returnedMessage(Message message, int replyCode, String replyText, String exchange, String routingKey) {
System.out.println("Message body message : " + message);
System.out.println("Message body message : " + replyCode);
System.out.println("Description:" + replyText);
System.out.println("Switch used by message exchange : " + exchange);
System.out.println("The routing key used by the message routing : " + routingKey);
}
}
When there are no switches in MQ, the following test triggers the ConfirmCallback callback function, which will retry sending messages three times
When MQ has a corresponding switch but the corresponding queue does not exist, the test results are as follows, and the ReturnCallback function will be triggered
MQ message consumption confirmation
Let's talk about the second problem. As for the failure of message consumption, MQ comes with a message consumption ACK mechanism. If there is no additional configuration for MQ in our project, it defaults to automatic ACK. The so-called ACK is the signing in of messages. The following describes the commonly used ACK mechanisms
API introduction.
channel.basicNack(deliveryTag, multiple, requeue);
Parameter 1: a tag of the current message, type Long (increment)
Parameter 2: batch processing data (< = data marked by current message)
Parameter 3: whether to put the queue back
for instance:
channel.basicNack(6, false, false); It means discarding 6 messages
channel.basicNack(6, true, false); It means discarding 6 and messages before 6
channel.basicNack(6, false, true); This means putting 6 messages back on the queue
channel.basicNack(6, true, true); It means to put the messages before 6 and 6 back on the queue
//You can select whether to sign in the current message or sign in all messages marked with < = current message channel.basicAck(deliveryTag, multiple); //Castrated batch, other and basic NACK usage are consistent channel.basicReject(deliveryTag, requeue);
Practical writing: if the order fails, the order will be retried three times. Remember to process the message finally, otherwise the problem of MQ message stacking will occur
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.Exchange;
import org.springframework.amqp.rabbit.annotation.Queue;
import org.springframework.amqp.rabbit.annotation.QueueBinding;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.util.HashMap;
@Data
@Slf4j
@Component
public class TopicCustomer {
/**
* Monitor the delay task NewsImpl.senMsgToRabbitMq(order.setPrice(100),200L);
*/
@RabbitListener(bindings = {
@QueueBinding(
value = @Queue(value = "cTopicQueue"),
exchange = @Exchange(value = "cTopicExchange", type = "topic"),
key = "user.live")
})
public void receivel(String msg, Channel channel, Message message) throws IOException {
Boolean flag = false;
try {
for (int i = 0; i < 3; i++) {
try {
System.err.println("The first: " + i + "Next order:" + msg);
flag = false;
if (flag) {
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
System.err.println("The first: " + i + "Next order:" + msg + "success");
break;
}
System.err.println("The first: " + i + "Next order:" + msg + "fail");
} catch (Exception e) {
System.err.println("An exception occurred during the order placing process. Start retrying the order three times" + e.getCause());
continue;
}
}
} catch (Exception e) {
System.err.println("Scheduled task execution failed");
} finally {
if (!flag) channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, false);
System.err.println("Discard message");
}
}
MQ message persistence
Let's start with the third question. What if the Rabbitmq service hangs up? In fact, when the service hangs up, the biggest remedy we can take is to quickly restart the service and then restore production data. It's easy to understand this. Set persistence when initializing our switches, queues and messages.
@Configuration
public class rabbitmqConfig {
@Bean
public TopicExchange cdTopicExchange() {
TopicExchange topicExchange = new TopicExchange("cdTopicExchange", true, false, null);
return topicExchange;
}
/**
* Dead letter queue
* The message timeout is sent to cTopicExchange, which is equivalent to forwarding. Specify routing key, and switch
*/
@Bean
public Queue cdTopicQueue() {
HashMap<String, Object> params = new HashMap<>();
params.put("x-dead-letter-exchange", "cTopicExchange");
params.put("x-dead-letter-routing-key", "user.live");
Queue queue = new Queue("cdTopicQueue", true, false, false, params);
return queue;
}
@Bean
public Binding cdBind() {
Binding binding = BindingBuilder.bind(cdTopicQueue()).to(cdTopicExchange()).with("user.dead");
return binding;
}
@Bean
public TopicExchange cTopicExchange() {
TopicExchange topicExchange = new TopicExchange("cTopicExchange", true, false, null);
return topicExchange;
}
@Bean
public Queue cTopicQueue() {
Queue queue = new Queue("cTopicQueue", true, false, false);
return queue;
}
@Bean
public Binding cBind() {
Binding binding = BindingBuilder.bind(cTopicQueue()).to(cTopicExchange()).with("user.live");
return binding;
}
}
I am now a senior student. Hahaha, if there is anything poorly written in the article, you are welcome to correct it,