Producer principle
Producer overview
The party sending the message is called the producer. His role in the entire RocketMQ production and consumption system is shown in the figure:
Producer group: a logical concept. When using producer instances, you need to specify a group name. A producer group can produce multiple topic messages.
Producer instance: a producer group deploys multiple processes, and each process can become a producer instance.
Topic: topic name. A topic consists of several queues.
The producer in the RocketMQ client has two independent implementation classes
org.apache.rocketmq.client.producer.DefaultMQProducer org.apache.rocketmq.client.producer.TransactionMQProducer
The former is used to produce general messages, sequential messages, one-way messages, batch messages and delayed messages, while the latter is mainly used to produce transaction messages.
Message structure and message type
The core fields of the message class are defined as follows:
public class Message implements Serializable {
private static final long serialVersionUID = 8445773977080406428L;
private String topic;
private int flag;
private Map<String, String> properties;
private byte[] body;
private String transactionId;
public Message() {
}
public Message(String topic, byte[] body) {
this(topic, "", "", 0, body, true);
}
public Message(String topic, String tags, String keys, int flag, byte[] body, boolean waitStoreMsgOK) {
this.topic = topic;
this.flag = flag;
this.body = body;
if (tags != null && tags.length() > 0) {
this.setTags(tags);
}
if (keys != null && keys.length() > 0) {
this.setKeys(keys);
}
this.setWaitStoreMsgOK(waitStoreMsgOK);
}
public Message(String topic, String tags, byte[] body) {
this(topic, tags, "", 0, body, true);
}
public Message(String topic, String tags, String keys, byte[] body) {
this(topic, tags, keys, 0, body, true);
}
public void setKeys(String keys) {
this.putProperty(MessageConst.PROPERTY_KEYS, keys);
}
void putProperty(final String name, final String value) {
if (null == this.properties) {
this.properties = new HashMap<String, String>();
}
this.properties.put(name, value);
}
void clearProperty(final String name) {
if (null != this.properties) {
this.properties.remove(name);
}
}
public void putUserProperty(final String name, final String value) {
if (MessageConst.STRING_HASH_SET.contains(name)) {
throw new RuntimeException(String.format(
"The Property<%s> is used by system, input another please", name));
}
if (value == null || value.trim().isEmpty()
|| name == null || name.trim().isEmpty()) {
throw new IllegalArgumentException(
"The name or value of property can not be null or blank string!"
);
}
this.putProperty(name, value);
}
public String getUserProperty(final String name) {
return this.getProperty(name);
}
public String getProperty(final String name) {
if (null == this.properties) {
this.properties = new HashMap<String, String>();
}
return this.properties.get(name);
}
public String getTopic() {
return topic;
}
public void setTopic(String topic) {
this.topic = topic;
}
public String getTags() {
return this.getProperty(MessageConst.PROPERTY_TAGS);
}
public void setTags(String tags) {
this.putProperty(MessageConst.PROPERTY_TAGS, tags);
}
public String getKeys() {
return this.getProperty(MessageConst.PROPERTY_KEYS);
}
public void setKeys(Collection<String> keys) {
StringBuilder sb = new StringBuilder();
for (String k : keys) {
sb.append(k);
sb.append(MessageConst.KEY_SEPARATOR);
}
this.setKeys(sb.toString().trim());
}
public int getDelayTimeLevel() {
String t = this.getProperty(MessageConst.PROPERTY_DELAY_TIME_LEVEL);
if (t != null) {
return Integer.parseInt(t);
}
return 0;
}
public void setDelayTimeLevel(int level) {
this.putProperty(MessageConst.PROPERTY_DELAY_TIME_LEVEL, String.valueOf(level));
}
public boolean isWaitStoreMsgOK() {
String result = this.getProperty(MessageConst.PROPERTY_WAIT_STORE_MSG_OK);
if (null == result) {
return true;
}
return Boolean.parseBoolean(result);
}
public void setWaitStoreMsgOK(boolean waitStoreMsgOK) {
this.putProperty(MessageConst.PROPERTY_WAIT_STORE_MSG_OK, Boolean.toString(waitStoreMsgOK));
}
public void setInstanceId(String instanceId) {
this.putProperty(MessageConst.PROPERTY_INSTANCE_ID, instanceId);
}
public int getFlag() {
return flag;
}
public void setFlag(int flag) {
this.flag = flag;
}
public byte[] getBody() {
return body;
}
public void setBody(byte[] body) {
this.body = body;
}
public Map<String, String> getProperties() {
return properties;
}
void setProperties(Map<String, String> properties) {
this.properties = properties;
}
public String getBuyerId() {
return getProperty(MessageConst.PROPERTY_BUYER_ID);
}
public void setBuyerId(String buyerId) {
putProperty(MessageConst.PROPERTY_BUYER_ID, buyerId);
}
public String getTransactionId() {
return transactionId;
}
public void setTransactionId(String transactionId) {
this.transactionId = transactionId;
}
}Topic: topic name, which can be created through RocketMQ Console
Flag: not currently
Propertise: message extension information. Tag, keys and delay level are saved here
Body: message body, byte array. Attention should be paid to the codes used by producers, and consumers must also use the same codes for decoding, otherwise garbled codes will be generated.
setKeys(): set the key of the message. Messageconst.key can be used for multiple keys_ Separator or use another overloaded method directly. If messageIndexEnable=true in the Broker, the hash index of the message will be created according to the key to help users query quickly.
setTags(): tags for message filtering. Users can subscribe to some tags of a Topic, so that the Broker will only send messages subscribed to Topic tags to consumers.
setDelayTimeLevel(): set the delay level, how long the delay is, and consumers can consume.
putUserProperty(): if there are other extension information, it can be stored here. There is a Map inside. Repeated calls will overwrite the old value.
RocketMQ supports normal messages, partition ordered messages, global ordered messages, delayed messages and transaction messages
Ordinary messages: ordinary messages are also called concurrent messages. Compared with traditional queues, concurrent messages have no order, but production and consumption are carried out in parallel, and the single machine performance can reach 100000 TPS.
Partition ordered message: similar to the partition in kafka, a Topic message is divided into multiple partitions for storage and consumption. The messages in a partition are traditional queues and follow the principle of FIFO.
Global ordered messages: if the number of partitions of a topic is set to 1, the messages in the topic are single partitions, and all messages follow the FIFO principle.
Delayed message: after the message is sent, the consumer can consume only after a certain time or a specified time point. When there are no delayed messages, the basic approach is to schedule tasks based on scheduled plans and send messages regularly. In RocketMQ, you only need to set the delay level when sending messages.
Transaction message: it mainly involves distributed transactions, that is, consumers can consume messages only when multiple operations succeed or fail at the same time. RocketMQ gracefully implements distributed transactions by sending half messages, processing local transactions, submitting messages, or rolling back messages.
Producer high availability
Generally, we hope that no matter what happens to the Broker and Namesrv, there will be no unknown state or message loss when sending messages. In the process of message sending, various failures such as server damage and power failure may occur in the client, Broker and Namesrv. When these failures occur, how does RockerMQ handle them?
1. Client assurance mechanism
The first guarantee mechanism: retry mechanism. RocketMQ supports synchronous and asynchronous sending. Either way, you can retry after the configuration fails. If a single Broker fails, you will select another Broker to ensure the normal sending of messages.
The configuration item retryTimesWhenSendFailed indicates the number of synchronization retries. By default, it is 2 times. Plus 1 normal sending, there are 3 opportunities in total.
Please refer to the retry code sent synchronously
org.apache.rocketmq.client.impl.producer.DefaultMQProducerImpl
After each transmission failure, the retry code will be executed unless the transmission is interrupted. The synchronous sending retry code is as follows:
private SendResult sendDefaultImpl(
Message msg,
final CommunicationMode communicationMode,
final SendCallback sendCallback,
final long timeout
) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
this.makeSureStateOK();
Validators.checkMessage(msg, this.defaultMQProducer);
final long invokeID = random.nextLong();
long beginTimestampFirst = System.currentTimeMillis();
long beginTimestampPrev = beginTimestampFirst;
long endTimestamp = beginTimestampFirst;
TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());
if (topicPublishInfo != null && topicPublishInfo.ok()) {
boolean callTimeout = false;
MessageQueue mq = null;
Exception exception = null;
SendResult sendResult = null;
int timesTotal = communicationMode == CommunicationMode.SYNC ? 1 + this.defaultMQProducer.getRetryTimesWhenSendFailed() : 1;
int times = 0;
String[] brokersSent = new String[timesTotal];
for (; times < timesTotal; times++) {
String lastBrokerName = null == mq ? null : mq.getBrokerName();
MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName);
if (mqSelected != null) {
mq = mqSelected;
brokersSent[times] = mq.getBrokerName();
try {
beginTimestampPrev = System.currentTimeMillis();
if (times > 0) {
//Reset topic with namespace during resend.
msg.setTopic(this.defaultMQProducer.withNamespace(msg.getTopic()));
}
long costTime = beginTimestampPrev - beginTimestampFirst;
if (timeout < costTime) {
callTimeout = true;
break;
}
sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout - costTime);
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);
switch (communicationMode) {
case ASYNC:
return null;
case ONEWAY:
return null;
case SYNC:
if (sendResult.getSendStatus() != SendStatus.SEND_OK) {
if (this.defaultMQProducer.isRetryAnotherBrokerWhenNotStoreOK()) {
continue;
}
}
return sendResult;
default:
break;
}
} catch (RemotingException e) {
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);
log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);
log.warn(msg.toString());
exception = e;
continue;
} catch (MQClientException e) {
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);
log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);
log.warn(msg.toString());
exception = e;
continue;
} catch (MQBrokerException e) {
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, true);
log.warn(String.format("sendKernelImpl exception, resend at once, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);
log.warn(msg.toString());
exception = e;
if (this.defaultMQProducer.getRetryResponseCodes().contains(e.getResponseCode())) {
continue;
} else {
if (sendResult != null) {
return sendResult;
}
throw e;
}
} catch (InterruptedException e) {
endTimestamp = System.currentTimeMillis();
this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);
log.warn(String.format("sendKernelImpl exception, throw exception, InvokeID: %s, RT: %sms, Broker: %s", invokeID, endTimestamp - beginTimestampPrev, mq), e);
log.warn(msg.toString());
log.warn("sendKernelImpl exception", e);
log.warn(msg.toString());
throw e;
}
} else {
break;
}
}
if (sendResult != null) {
return sendResult;
}
String info = String.format("Send [%d] times, still failed, cost [%d]ms, Topic: %s, BrokersSent: %s",
times,
System.currentTimeMillis() - beginTimestampFirst,
msg.getTopic(),
Arrays.toString(brokersSent));
info += FAQUrl.suggestTodo(FAQUrl.SEND_MSG_FAILED);
MQClientException mqClientException = new MQClientException(info, exception);
if (callTimeout) {
throw new RemotingTooMuchRequestException("sendDefaultImpl call timeout");
}
if (exception instanceof MQBrokerException) {
mqClientException.setResponseCode(((MQBrokerException) exception).getResponseCode());
} else if (exception instanceof RemotingConnectException) {
mqClientException.setResponseCode(ClientErrorCode.CONNECT_BROKER_EXCEPTION);
} else if (exception instanceof RemotingTimeoutException) {
mqClientException.setResponseCode(ClientErrorCode.ACCESS_BROKER_TIMEOUT);
} else if (exception instanceof MQClientException) {
mqClientException.setResponseCode(ClientErrorCode.BROKER_NOT_EXIST_EXCEPTION);
}
throw mqClientException;
}
validateNameServerSetting();
throw new MQClientException("No route info of this topic: " + msg.getTopic() + FAQUrl.suggestTodo(FAQUrl.NO_TOPIC_ROUTE_INFO),
null).setResponseCode(ClientErrorCode.NOT_FOUND_TOPIC_EXCEPTION);
}
Asynchronous sending retry code can be referenced
org.apache.rocketmq.client.impl.MQClientAPIImpl
The code is as follows:
private void sendMessageAsync(
final String addr,
final String brokerName,
final Message msg,
final long timeoutMillis,
final RemotingCommand request,
final SendCallback sendCallback,
final TopicPublishInfo topicPublishInfo,
final MQClientInstance instance,
final int retryTimesWhenSendFailed,
final AtomicInteger times,
final SendMessageContext context,
final DefaultMQProducerImpl producer
) throws InterruptedException, RemotingException {
final long beginStartTime = System.currentTimeMillis();
this.remotingClient.invokeAsync(addr, request, timeoutMillis, new InvokeCallback() {
@Override
public void operationComplete(ResponseFuture responseFuture) {
long cost = System.currentTimeMillis() - beginStartTime;
RemotingCommand response = responseFuture.getResponseCommand();
if (null == sendCallback && response != null) {
try {
SendResult sendResult = MQClientAPIImpl.this.processSendResponse(brokerName, msg, response, addr);
if (context != null && sendResult != null) {
context.setSendResult(sendResult);
context.getProducer().executeSendMessageHookAfter(context);
}
} catch (Throwable e) {
}
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), false);
return;
}
if (response != null) {
try {
SendResult sendResult = MQClientAPIImpl.this.processSendResponse(brokerName, msg, response, addr);
assert sendResult != null;
if (context != null) {
context.setSendResult(sendResult);
context.getProducer().executeSendMessageHookAfter(context);
}
try {
sendCallback.onSuccess(sendResult);
} catch (Throwable e) {
}
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), false);
} catch (Exception e) {
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), true);
onExceptionImpl(brokerName, msg, timeoutMillis - cost, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, e, context, false, producer);
}
} else {
producer.updateFaultItem(brokerName, System.currentTimeMillis() - responseFuture.getBeginTimestamp(), true);
if (!responseFuture.isSendRequestOK()) {
MQClientException ex = new MQClientException("send request failed", responseFuture.getCause());
onExceptionImpl(brokerName, msg, timeoutMillis - cost, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
} else if (responseFuture.isTimeout()) {
MQClientException ex = new MQClientException("wait response timeout " + responseFuture.getTimeoutMillis() + "ms",
responseFuture.getCause());
onExceptionImpl(brokerName, msg, timeoutMillis - cost, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
} else {
MQClientException ex = new MQClientException("unknow reseaon", responseFuture.getCause());
onExceptionImpl(brokerName, msg, timeoutMillis - cost, request, sendCallback, topicPublishInfo, instance,
retryTimesWhenSendFailed, times, ex, context, true, producer);
}
}
}
});
}
The retry is completed asynchronously in the communication layer. When the response value returned by the operationComplete() method is null, the retry code will be executed again. The return value response is null, usually because the client receives a TCP request, unpacking fails, or no matching request is found.
The producer configuration item retryTimesWhenSendAsyncFailed indicates the number of asynchronous retries. By default, it is 2 times. Plus 1 time of normal sending, there are 3 sending opportunities in total.
The second guarantee mechanism: client fault tolerance. RocketMQ Client will maintain a "Broker send delay" relationship. According to this relationship, select a Broker with a lower send delay level to send messages. This can maximize the ability of the Broker, eliminate the brokers that are down, unavailable or with a higher send delay level, and try to ensure the normal sending of messages.
This mechanism is mainly reflected in how to select a Queue when sending a message. The source code is in the selectOneMessageQueue() method called by sendDefautImpl() method
public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) {
if (this.sendLatencyFaultEnable) {
try {
// First step
int index = tpInfo.getSendWhichQueue().incrementAndGet();
for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) {
int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size();
if (pos < 0)
pos = 0;
MessageQueue mq = tpInfo.getMessageQueueList().get(pos);
if (latencyFaultTolerance.isAvailable(mq.getBrokerName()))
return mq;
}
// Step 2
final String notBestBroker = latencyFaultTolerance.pickOneAtLeast();
int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker);
if (writeQueueNums > 0) {
final MessageQueue mq = tpInfo.selectOneMessageQueue();
if (notBestBroker != null) {
mq.setBrokerName(notBestBroker);
mq.setQueueId(tpInfo.getSendWhichQueue().incrementAndGet() % writeQueueNums);
}
return mq;
} else {
latencyFaultTolerance.remove(notBestBroker);
}
} catch (Exception e) {
log.error("Error occurred when selecting message queue", e);
}
// Step 3
return tpInfo.selectOneMessageQueue();
}
return tpInfo.selectOneMessageQueue(lastBrokerName);
}sendLatencyFaultEnable: the send delay fault tolerance switch is off by default. If the switch is on, the send delay fault tolerance mechanism will be triggered to select the send Queue.
How to select when sending a Queue?
Step 1: get a Broker that is acceptable in latency and the same as the last one. First, a self incrementing sequence number index is obtained, and the position subscript pos of the Queue is obtained by modulus. If the delay time of the Broker corresponding to the pos is acceptable, and it is sent for the first time, or it is the same as the Broker sent last time, put the Queue back.
Step 2: if a Broker is not selected in step 1, select a Broker with lower latency.
Step 3: if no Broker is selected in step 1 and step 2, select a Broker randomly.
The second code mainly includes a random selection method
tpInfo.selectOneMessageQueue(lastBrokerName);
The function of this method is to randomly select a Broker. The specific implementation is as follows:
public MessageQueue selectOneMessageQueue(final String lastBrokerName) {
if (lastBrokerName == null) {
// First step
return selectOneMessageQueue();
} else {
// Step 2
for (int i = 0; i < this.messageQueueList.size(); i++) {
int index = this.sendWhichQueue.incrementAndGet();
int pos = Math.abs(index) % this.messageQueueList.size();
if (pos < 0)
pos = 0;
MessageQueue mq = this.messageQueueList.get(pos);
if (!mq.getBrokerName().equals(lastBrokerName)) {
return mq;
}
}
// Step 3
return selectOneMessageQueue();
}
}Step 1: if there is no last used Broker as a reference, select a Broker at random.
Step 2: if there is a last used Broker, select a non last used Broker to evenly disperse the Broker pressure.
Step 3: if no Broker is selected in the first and second steps, the bottom scheme is adopted and a Broker is selected randomly.
When executing the above code, you need the Broker and send delayed data as a judgment sentence. How do these data come from?
After sending a message, the client will call the updatefaultemitem() method to update the delay of the Broker currently receiving the message. These main logics are implemented in the MQFaultStrategy class. The delay policy has a standard interface LatencyFaultTolerance, through which we can implement a custom delay policy.
2. Broker side guarantee
The data synchronization mode ensures synchronous replication and asynchronous replication. Synchronous replication means that the message is sent to the Master Broker and then synchronized to the Slave Broker; Asynchronous replication means that the message is sent to the Master Broker, that is, it is sent successfully. In the production environment, it is recommended to deploy at least two masters and one Slave, which are described in detail below:
(1) One Slave loses power. During Broker synchronous replication, the production fails to be sent for the first time. It succeeds after retrying to another group of brokers. During Broker asynchronous replication, normal production is not affected
(2) Two Slave power down. During Broker synchronous replication, production fails. During Broker asynchronous replication, normal production is not affected.
(3) One Master is powered off. During Broker synchronous replication, production fails for the first time, and it succeeds after retrying to another group of brokers. The practice of Broker asynchronous replication is the same as that of synchronous replication.
(4) Two masters are powered off. All production failed.
(5) The same group of Master and Slave power down. During broker synchronous replication, the production fails to send for the first time. It succeeds after retrying to another group of brokers. Broker asynchronous replication is not affected.
(6) Both sets of machines are powered off. All production failed.
To sum up, if you want to achieve absolute high reliability, you can copy the master-slave configured by the Broker synchronously. As long as the producer receives the feedback that the message is saved successfully, the message will not be lost. In most scenarios, master-slave asynchronous replication can be configured, which is very efficient.