1, Release confirmation principle
The producer sets the channel to confirm mode. Once the channel enters the confirm mode, all messages published on the channel will be assigned a unique ID (starting from 1). Once the message is delivered to all matching queues, the broker will send a confirmation to the producer (including the unique ID of the message), This enables the producer to know that the message has correctly arrived at the destination queue. If the message and queue are persistent, the confirmation message will be sent after writing the message to the disk. The delivery tag field of the confirmation message returned by the broker to the producer contains the sequence number of the confirmation message. In addition, the broker can also set the multiple field of basic.ack, Indicates that all messages before this serial number have been processed.
The biggest advantage of confirm mode is that it is asynchronous. Once a message is published, the producer application can continue to send the next message while waiting for the channel to return the confirmation. After the message is finally confirmed, the producer application can handle the confirmation message through the callback method. If RabbitMQ loses the message due to its own internal error, A nack message is sent, and the producer application can also process the nack message in the callback method.
1.1. Release confirmation strategy
1.1.1. Method of opening release confirmation
Publishing confirmation is not enabled by default. If you want to enable it, you need to call the method confirmSelect. Whenever you want to use publishing confirmation, you need to call this method in channel
1.1.2 single confirmation release
This is a simple confirmation method. It is a synchronous confirmation publishing method, that is, after publishing a message, only it is confirmed to be published, and subsequent messages can continue to be published. The waitForConfirmOrDie(long) method returns only when the message is confirmed. If the message is not confirmed within the specified time range, it will throw an exception.
The biggest disadvantage of this confirmation method is that the publishing speed is particularly slow, because if the published messages are not confirmed, the publishing of all subsequent messages will be blocked. This method provides a throughput of no more than hundreds of published messages per second. Of course, this may be sufficient for some applications.
package com.xbmu.rabbitmq.four;
import com.rabbitmq.client.Channel;
import com.xbmu.rabbitmq.utils.RabbitMqUtils;
import java.util.UUID;
public class Task03 {
private static final int MESSAGE_COUNT = 1000;
public static void main(String[] args) throws Exception {
// Single confirmation release
// Issue 1000 individual confirmation messages, taking 554ms
publishMessageIndividually();
}
public static void publishMessageIndividually() throws Exception {
Channel channel = RabbitMqUtils.getChannel();
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation
channel.confirmSelect();
long begin = System.currentTimeMillis();
for (int i = 0; i < MESSAGE_COUNT; i++) {
String message = i + "";
channel.basicPublish("",queueName,null,message.getBytes());
// If the server returns false or does not return within the timeout, the producer can resend the message
boolean flag = channel.waitForConfirms();
if (flag){
System.out.println("Message sent successfully");
}
}
long end = System.currentTimeMillis();
System.out.println("release"+MESSAGE_COUNT+"Separate confirmation messages, time consuming"+(end-begin)+"ms");
}
}
1.1.3 batch confirmation release
The above method is very slow. Compared with a single message waiting for confirmation, publishing a batch of messages first and then confirming together can greatly improve the throughput. Of course, the disadvantage of this method is that when a failure causes a problem in publishing, we don't know which message has a problem. We must save the whole batch in memory, To record important information and then republish the message. Of course, this scheme is still synchronous and blocks the release of messages.
package com.xbmu.rabbitmq.four;
import com.rabbitmq.client.Channel;
import com.xbmu.rabbitmq.utils.RabbitMqUtils;
import java.util.UUID;
public class Task03 {
private static final int MESSAGE_COUNT = 1000;
public static void main(String[] args) throws Exception {
// Batch confirmation release
// It takes 126ms to publish 1000 batch confirmation messages
publishMessageBatch();
}
public static void publishMessageBatch() throws Exception {
Channel channel = RabbitMqUtils.getChannel();
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName,false,false,false,null);
// Open release confirmation
channel.confirmSelect();
// Bulk confirmation message size
int batchSize = 100;
// Number of unacknowledged messages
int outstandingMessageCount = 0;
long begin = System.currentTimeMillis();
for (int i = 0; i < MESSAGE_COUNT; i++) {
String message = i + "";
channel.basicPublish("",queueName,null,message.getBytes());
outstandingMessageCount ++ ;
if (outstandingMessageCount == batchSize){
channel.waitForConfirms();
outstandingMessageCount = 0;
}
}
// To ensure that there are still unconfirmed messages left, confirm again
if(outstandingMessageCount > 0){
channel.waitForConfirms();
}
long end = System.currentTimeMillis();
System.out.println("release"+MESSAGE_COUNT+"Batch confirmation messages, time consuming"+(end-begin)+"ms");
}
}
1.1.4 asynchronous confirmation Publishing
Although the programming logic of asynchronous confirmation is more complex than the above two, it has the highest cost performance. It can not be said whether it is reliable or efficient. It uses callback function to achieve reliable message delivery. This middleware also ensures whether the delivery is successful through function callback.
package com.xbmu.rabbitmq.four;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.ConfirmCallback;
import com.xbmu.rabbitmq.utils.RabbitMqUtils;
import java.util.UUID;
import java.util.concurrent.ConcurrentNavigableMap;
import java.util.concurrent.ConcurrentSkipListMap;
public class Task03 {
private static final int MESSAGE_COUNT = 1000;
public static void main(String[] args) throws Exception {
// Asynchronous confirmation Publishing
// It takes 31ms to publish 1000 asynchronous confirmation messages
publishMessageAsync();
}
public static void publishMessageAsync() throws Exception {
try (Channel channel = RabbitMqUtils.getChannel()) {
String queueName = UUID.randomUUID().toString();
channel.queueDeclare(queueName, false, false, false, null);
// Open release confirmation
channel.confirmSelect();
/**
* A thread safe and orderly hash table, which is suitable for high concurrency
* 1. Easily associate sequence numbers with messages
* 2. Easily delete entries in batches as long as the serial number is given
* 3. Support concurrent access
*/
ConcurrentSkipListMap<Long, String> outstandingConfirms = new
ConcurrentSkipListMap<>();
/**
* A callback to acknowledge receipt of a message
* 1. Message sequence number
* 2.true Messages less than or equal to the current serial number can be confirmed
* false Confirm current serial number message
*/
ConfirmCallback ackCallback = (sequenceNumber, multiple) -> {
if (multiple) {
// The returned unconfirmed message is less than or equal to the current serial number. It is a map
ConcurrentNavigableMap<Long, String> confirmed =
outstandingConfirms.headMap(sequenceNumber, true);
// Clear this part of the unacknowledged message
confirmed.clear();
}else{
// Only messages with the current serial number are cleared
outstandingConfirms.remove(sequenceNumber);
}
};
ConfirmCallback nackCallback = (sequenceNumber, multiple) -> {
String message = outstandingConfirms.get(sequenceNumber);
System.out.println(" Published messages"+message+" Unconfirmed, serial number"+sequenceNumber);
};
/**
* Add a listener for asynchronous acknowledgement
* 1. Callback to acknowledge receipt of message
* 2. Callback for message not received
*/
channel.addConfirmListener(ackCallback, nackCallback);
long begin = System.currentTimeMillis();
for (int i = 0; i < MESSAGE_COUNT; i++) {
String message = " news" + i;
/**
* channel.getNextPublishSeqNo() Gets the sequence number of the next message
* An association is made with the message body through the serial number
* All are unacknowledged message bodies
*/
outstandingConfirms.put(channel.getNextPublishSeqNo(), message);
channel.basicPublish("", queueName, null, message.getBytes());
}
long end = System.currentTimeMillis();
System.out.println(" release" + MESSAGE_COUNT + " Asynchronous acknowledgement message, time consuming" + (end - begin) +
"ms");
}
}
}
1.1.5. How to handle asynchronous unacknowledged messages
The best solution is to put unconfirmed messages into a memory based queue that can be accessed by the publishing thread. For example, use the ConcurrentLinkedQueue queue to transfer messages between confirm callbacks and the publishing thread.
1.1.6 comparison of release confirmation speed in the above 3
- Publish message separately
Synchronization waiting for confirmation is simple, but the throughput is very limited. - Batch publish message
Batch synchronization waits for confirmation. It is simple and reasonable throughput. Once there is a problem, it is difficult to infer that the message has a problem. - Asynchronous processing
Optimal performance and resource usage can be well controlled in case of errors, but it is slightly difficult to implement.