Premise review of RocketMQ
RocketMQ is a distributed, queue model message oriented middleware with the following characteristics:
- It can ensure strict message order
- Provide rich message pull mode
- Efficient subscriber level scalability
- Real time message subscription mechanism
- 100 million message accumulation capacity
Why use RocketMQ
- It is emphasized that the cluster has no single point and is scalable. Any point is highly available and horizontally scalable
- Massive message stacking capability, low write delay after message stacking
- Support tens of thousands of queues
- Message failure retry mechanism
- Message can be queried
- Active open source community
- Maturity has passed the test of Taobao's double 11
Development and changes of RocketMQ
RocketMQ's open source uses files as a persistence tool. The performance of Alibaba's non open source is higher. oceanBase is used as a persistence tool. In RocketMQ 1. X and 2.x, zookeeper is used to manage clusters. In 3.x, nameserver is used to replace zk, which is lighter. In addition, RocketMQ clients have two operation modes: DefaultMQPushConsumer and DefaultMQPullConsumer.
Maven configuration for DefaultMQPushConsumer
<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-client</artifactId> <version>4.3.0</version> </dependency>
DefaultMQPushConsumer usage example
- CONSUME_FROM_LAST_OFFSET: start the consumption from the last position of the queue for the first time, and then start the subsequent consumption, and then start the consumption according to the progress of the last consumption
- CONSUME_FROM_FIRST_OFFSET: start consumption from the initial position of the queue for the first time, then start it again, and then start consumption according to the progress of the last consumption
- CONSUME_FROM_TIMESTAMP: start consumption from the specified time point and location for the first time, then start it again, and then start consumption according to the progress of last consumption
The first start mentioned above refers to a consumer who has never consumed. If the consumer has consumed, the consumer's consumption location will be recorded on the broker. If the consumer hangs up and starts again, it will automatically start from the progress of the last consumption
public class MQPushConsumer {
public static void main(String[] args) throws MQClientException {
String groupName = "rocketMqGroup1";
// It is used to organize multiple consumers together to improve the concurrent processing capability
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer(groupName);
// Set the nameServer address, with multiple addresses; separate
consumer.setNamesrvAddr("name-serverl-ip:9876;name-server2-ip:9876"); consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
consumer.setMessageModel(MessageModel.BROADCASTING);
// To subscribe to topic, you can filter the specified messages, for example: "topictest", "tagL | tag2 | tag3", * or null indicates all messages of topic
consumer.subscribe("order-topic", "*");
consumer.registerMessageListener(new MessageListenerConcurrently() {
@Override
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> mgs,
ConsumeConcurrentlyContext consumeconcurrentlycontext) {
System.out.println(Thread.currentThread().getName()+"Receive New Messages:"+mgs);
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
consumer.start();
}
}
- CLUSTERING: the default mode. Each consumer in the same consumer group (with the same groupname) consumes only part of the subscribed message. All consumer messages in the same consumer group add up to the subscribed message
- Subscribe to the topic as a whole, so as to achieve the purpose of load balancing
- BROADCASTING: each consumer in the same consumer group consumes all the subscribed topic messages, that is, a consumption will be distributed multiple times and consumed by multiple consumers.
ConsumeConcurrentlyStatus.RECONSUME_LATER boker will initiate retries according to the set messageDelayLevel. The default is 16 times.
The main functions of each object in DefaultMQPushConsumerImpl are as follows:
Rebalance pushimpl: mainly responsible for determining which queues the current consumer should consume messages from;
- 1) PullAPIWrapper: long connection, which is responsible for pulling messages from the broker, and then using consummessageservice to call back the user's Listener to execute message consumption logic;
- 2) ConsumeMessageService: realize the so-called "Push passive" consumption mechanism; After the message pulled from the Broker is encapsulated into a ConsumeRequest and submitted to ConsumeMessageService, this service is responsible for callback of the user's Listener consumption message;
- 3) OffsetStore: maintain the consumption record (offset) of the current consumer; There are two implementations: local and Rmote. Local is stored on the local disk, which is suitable for BROADCASTING broadcast consumption mode; Remote stores the consumption progress on the Broker, which is applicable to the cluster consumption mode;
- 4) MQClientFactory: responsible for managing clients (consumer s, producer s) and providing multi-functional interfaces for various services (Rebalance, PullMessage, etc.); Most of the logic is done in this class;
consumer.registerMessageListener execution process:
/**
* Register a callback to execute on message arrival for concurrent consuming.
* @param messageListener message handling callback.
*/
@Override
public void registerMessageListener(MessageListenerConcurrently messageListener) {
this.messageListener = messageListener; this.defaultMQPushConsumerImpl.registerMessageListener(messageListener);
}
Through source code, we can see that the main implementation process calls synchronous start method of DefaultMQPushConsumerImpl in DefaultMQPushConsumerImpl class after consumer.start.
public synchronized void start() throws MQClientException {
switch (this.serviceState) {
case CREATE_JUST:
log.info("the consumer [{}] start beginning. messageModel={}, isUnitMode={}", this.defaultMQPushConsumer.getConsumerGroup(),
this.defaultMQPushConsumer.getMessageModel(), this.defaultMQPushConsumer.isUnitMode());
this.serviceState = ServiceState.START_FAILED;
this.checkConfig();
this.copySubscription();
if (this.defaultMQPushConsumer.getMessageModel() == MessageModel.CLUSTERING) {
this.defaultMQPushConsumer.changeInstanceNameToPID();
}
this.mQClientFactory = MQClientManager.getInstance().getAndCreateMQClientInstance(this.defaultMQPushConsumer, this.rpcHook);
this.rebalanceImpl.setConsumerGroup(this.defaultMQPushConsumer.getConsumerGroup());
this.rebalanceImpl.setMessageModel(this.defaultMQPushConsumer.getMessageModel());
this.rebalanceImpl.setAllocateMessageQueueStrategy(this.defaultMQPushConsumer.getAllocateMessageQueueStrategy());
this.rebalanceImpl.setmQClientFactory(this.mQClientFactory);
this.pullAPIWrapper = new PullAPIWrapper(
mQClientFactory,
this.defaultMQPushConsumer.getConsumerGroup(), isUnitMode());
this.pullAPIWrapper.registerFilterMessageHook(filterMessageHookList);
if (this.defaultMQPushConsumer.getOffsetStore() != null) {
this.offsetStore = this.defaultMQPushConsumer.getOffsetStore();
} else {
switch (this.defaultMQPushConsumer.getMessageModel()) {
case BROADCASTING:
this.offsetStore = new LocalFileOffsetStore(this.mQClientFactory, this.defaultMQPushConsumer.getConsumerGroup());
break;
case CLUSTERING:
this.offsetStore = new RemoteBrokerOffsetStore(this.mQClientFactory, this.defaultMQPushConsumer.getConsumerGroup());
break;
default:
break;
}
this.defaultMQPushConsumer.setOffsetStore(this.offsetStore);
}
this.offsetStore.load();
if (this.getMessageListenerInner() instanceof MessageListenerOrderly) {
this.consumeOrderly = true;
this.consumeMessageService =
new ConsumeMessageOrderlyService(this, (MessageListenerOrderly) this.getMessageListenerInner());
} else if (this.getMessageListenerInner() instanceof MessageListenerConcurrently) {
this.consumeOrderly = false;
this.consumeMessageService =
new ConsumeMessageConcurrentlyService(this, (MessageListenerConcurrently) this.getMessageListenerInner());
}
this.consumeMessageService.start();
boolean registerOK = mQClientFactory.registerConsumer(this.defaultMQPushConsumer.getConsumerGroup(), this);
if (!registerOK) {
this.serviceState = ServiceState.CREATE_JUST;
this.consumeMessageService.shutdown();
throw new MQClientException("The consumer group[" + this.defaultMQPushConsumer.getConsumerGroup()
+ "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
null);
}
mQClientFactory.start();
log.info("the consumer [{}] start OK.", this.defaultMQPushConsumer.getConsumerGroup());
this.serviceState = ServiceState.RUNNING;
break;
case RUNNING:
case START_FAILED:
case SHUTDOWN_ALREADY:
throw new MQClientException("The PushConsumer service state not OK, maybe started once, "
+ this.serviceState
+ FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
null);
default:
break;
}
this.updateTopicSubscribeInfoWhenSubscriptionChanged();
this.mQClientFactory.checkClientInBroker();
this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
this.mQClientFactory.rebalanceImmediately();
}
Through mQClientFactory.start(); We found him calling
public void start() throws MQClientException {
synchronized (this) {
switch (this.serviceState) {
case CREATE_JUST:
this.serviceState = ServiceState.START_FAILED;
// If not specified,looking address from name server
if (null == this.clientConfig.getNamesrvAddr()) {
this.mQClientAPIImpl.fetchNameServerAddr();
}
// Start request-response channel
this.mQClientAPIImpl.start();
// Start various schedule tasks
this.startScheduledTask();
// Start pull service
this.pullMessageService.start();
// Start rebalance service
this.rebalanceService.start();
// Start push service
this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
log.info("the client factory [{}] start OK", this.clientId);
this.serviceState = ServiceState.RUNNING;
break;
case RUNNING:
break;
case SHUTDOWN_ALREADY:
break;
case START_FAILED:
throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
default:
break;
}
}
}
There are multiple starts in this method. We mainly look at pullMessageService.start(); Here we find that the Push mode underlying of RocketMQ is actually implemented through pull. Let's look at the logic handled by the pullMessageService:
private void pullMessage(final PullRequest pullRequest) {
final MQConsumerInner consumer = this.mQClientFactory.selectConsumer(pullRequest.getConsumerGroup());
if (consumer != null) {
DefaultMQPushConsumerImpl impl = (DefaultMQPushConsumerImpl) consumer;
impl.pullMessage(pullRequest);
} else {
log.warn("No matched consumer for the PullRequest {}, drop it", pullRequest);
}
}
We found that in fact, it uses the pullMessage method of DefaultMQPushConsumerImpl class to process messages logically
pullRequest pull method
Pullrequest is explained here. We mentioned above that the push mode of rocketmq is actually implemented through the pull mode encapsulation. Pullrequest achieves the push effect through long polling.
The long polling mode has both the advantages of pull and the real-time performance of push mode.
-
The push mode is that after receiving the message, the server side actively pushes the message to the client side, with high real-time performance. The disadvantage is that the workload of the server side is large, which affects the performance. Secondly, the processing capacity of the client side is different, and the state of the client side is not controlled by the server side. If the client side cannot process messages in time, it is easy to cause message accumulation, which has affected the normal business, etc.
-
The pull mode is that the client circulates to pull messages from the server side. The initiative is on the client side. It processes one message and then pulls the next. The disadvantage is that the cycle time is not easy to set, the time is too short and it is easy to be busy, which wastes CPU resources. If the time interval is too long, the client's processing capacity will decline, and sometimes some messages will not be processed in time.
Long polling can combine the advantages of both
- Check whether the dropped of the ProcessQueue object in the PullRequest object is true (maintain the corresponding ProcessQueue object when creating a pull message request for the MessageQueue under the topic in the RebalanceService thread. If the Consumer no longer subscribes to the topic, it will set the dropped of the object to true); If yes, it is considered that the request has been cancelled, and the method is directly jumped out;
- Update the timestamp (ProcessQueue.lastPullTimestamp) of the ProcessQueue object in the PullRequest object to the current timestamp;
- Check whether the Consumer is RUNNING, that is, whether DefaultMQPushConsumerImpl.serviceState is RUNNING; If it is not RUNNING or suspended (DefaultMQPushConsumerImpl.pause=true), call the pullmessageservice.executepullrequestlater (PullRequest, PullRequest, long timedelay) method to delay and then pull the message, where timeDelay=3000; The purpose of this method is to put the PullRequest object into the PullMessageService. pullRequestQueue queue again after 3 seconds; And jump out of the method;
- Flow control. If the msgCount of the ProcessQueue object is greater than the flow control threshold of the consumer (DefaultMQPushConsumer.pullThresholdForQueue, the default value is 1000), call the PullMessageService.executePullRequestLater method and put the PullRequest request request into the PullMessageService.pullRequestQueue queue again after 50 milliseconds; And jump out of the method;
- If it is not sequential consumption (i.e. DefaultMQPushConsumerImpl.consumeOrderly equals false), check the difference between the first key value and the last key value of the msgtreemap: treemap < long, messageext > variable of the ProcessQueue object. The key value represents the queue offset of the query, queueoffset; If the difference is greater than the threshold (specified by DefaultMQPushConsumer. consumeConcurrentlyMaxSpan, the default is 2000), call the PullMessageService.executePullRequestLater method and put the PullRequest request into the PullMessageService.pullRequestQueue queue again after 50 milliseconds; And jump out of the method;
- Take the topic value of the PullRequest.messageQueue object as the parameter to obtain the corresponding SubscriptionData object from rebalanceimpl. Subscriptioninner: concurrenthashmap, SubscriptionData >. If the object is null, considering the concurrency relationship, call the executePullRequestLater method and try again later; And jump out of the method;
- If the message model is in cluster mode (RebalanceImpl.messageModel is equal to CLUSTERING), take the MessageQueue variable value of PullRequest object and type =READ_FROM_MEMORY (get consumption progress offset value from memory) calls the readOffset (MessageQueue MQ, readoffsettype) method of the DefaultMQPushConsumerImpl. offsetStore object (initialized as a remotebrakerofsetstore object) as a parameter to get the consumption progress offset value from local memory. If the offset value is greater than 0, set the temporary variable commitOffsetEnable equal to true; otherwise, it is false; This offset value is used as the commitOffset parameter in the pullKernelImpl method. After the Broker pulls the message, it determines whether to update the message progress with this offset according to the commitOffsetEnable parameter value. The logic of the readOffset method is: obtain the consumption progress offset from the remotebrakerofsetstore.offsettable: concurrenthashmap < MessageQueue, atomiclong > variable with the input parameter MessageQueue object; If the offset is not null, the value is returned; otherwise, - 1 is returned;
- When the subscription relationship needs to be updated after each message pull (represented by the DefaultMQPushConsumer. postSubscriptionWhenPull parameter, which is false by default) and the classFilterMode of the SubscriptionData object obtained from RebalanceImpl.subscriptionInner with the topic value parameter is equal to false (false by default), set the third byte of the sysFlag tag to 1, otherwise the byte is set to 0;
- The first byte of the sysFlag tag is set to the value of commitOffsetEnable; The second byte (suspend flag) is set to 1; The fourth byte is set to the value of classFilterMode;
- Initialize the anonymous inner class PullCallback and implement the onSucess/onException method; This method will call back only in the case of asynchronous request;
- Call the underlying pull message API interface:
PullAPIWrapper.pullKernelImpl
Pullapiwrapper.pullkernelimpl (messagequeue MQ, string subexpression, long subversion, long offset, int maxnums, int sysflag, long commitoffset, long brokersuspendmaxtimemillis, long timeoutmillis, communicationmode, pullcallback pullcallback) method to pull messages.
Pass the callback class PullCallback into this method. When the message is pulled asynchronously, the method of this callback class will be called back after receiving the response.
public void pullMessage(final PullRequest pullRequest) {
final ProcessQueue processQueue = pullRequest.getProcessQueue();
if (processQueue.isDropped()) {
log.info("the pull request[{}] is dropped.", pullRequest.toString());
return;
}
pullRequest.getProcessQueue().setLastPullTimestamp(System.currentTimeMillis());
try {
this.makeSureStateOK();
} catch (MQClientException e) {
log.warn("pullMessage exception, consumer state not ok", e);
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_EXCEPTION);
return;
}
if (this.isPause()) {
log.warn("consumer was paused, execute pull request later. instanceName={}, group={}", this.defaultMQPushConsumer.getInstanceName(), this.defaultMQPushConsumer.getConsumerGroup());
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_SUSPEND);
return;
}
long cachedMessageCount = processQueue.getMsgCount().get();
long cachedMessageSizeInMiB = processQueue.getMsgSize().get() / (1024 * 1024);
if (cachedMessageCount > this.defaultMQPushConsumer.getPullThresholdForQueue()) {
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);
if ((queueFlowControlTimes++ % 1000) == 0) {
log.warn(
"the cached message count exceeds the threshold {}, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}",
this.defaultMQPushConsumer.getPullThresholdForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes);
}
return;
}
if (cachedMessageSizeInMiB > this.defaultMQPushConsumer.getPullThresholdSizeForQueue()) {
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);
if ((queueFlowControlTimes++ % 1000) == 0) {
log.warn(
"the cached message size exceeds the threshold {} MiB, so do flow control, minOffset={}, maxOffset={}, count={}, size={} MiB, pullRequest={}, flowControlTimes={}",
this.defaultMQPushConsumer.getPullThresholdSizeForQueue(), processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), cachedMessageCount, cachedMessageSizeInMiB, pullRequest, queueFlowControlTimes);
}
return;
}
if (!this.consumeOrderly) {
if (processQueue.getMaxSpan() > this.defaultMQPushConsumer.getConsumeConcurrentlyMaxSpan()) {
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_FLOW_CONTROL);
if ((queueMaxSpanFlowControlTimes++ % 1000) == 0) {
log.warn(
"the queue's messages, span too long, so do flow control, minOffset={}, maxOffset={}, maxSpan={}, pullRequest={}, flowControlTimes={}",
processQueue.getMsgTreeMap().firstKey(), processQueue.getMsgTreeMap().lastKey(), processQueue.getMaxSpan(),
pullRequest, queueMaxSpanFlowControlTimes);
}
return;
}
} else {
if (processQueue.isLocked()) {
if (!pullRequest.isLockedFirst()) {
final long offset = this.rebalanceImpl.computePullFromWhere(pullRequest.getMessageQueue());
boolean brokerBusy = offset < pullRequest.getNextOffset();
log.info("the first time to pull message, so fix offset from broker. pullRequest: {} NewOffset: {} brokerBusy: {}",
pullRequest, offset, brokerBusy);
if (brokerBusy) {
log.info("[NOTIFYME]the first time to pull message, but pull request offset larger than broker consume offset. pullRequest: {} NewOffset: {}",
pullRequest, offset);
}
pullRequest.setLockedFirst(true);
pullRequest.setNextOffset(offset);
}
} else {
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_EXCEPTION);
log.info("pull message later because not locked in broker, {}", pullRequest);
return;
}
}
final SubscriptionData subscriptionData = this.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());
if (null == subscriptionData) {
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_EXCEPTION);
log.warn("find the consumer's subscription failed, {}", pullRequest);
return;
}
final long beginTimestamp = System.currentTimeMillis();
PullCallback pullCallback = new PullCallback() {
@Override
public void onSuccess(PullResult pullResult) {
if (pullResult != null) {
pullResult = DefaultMQPushConsumerImpl.this.pullAPIWrapper.processPullResult(pullRequest.getMessageQueue(), pullResult,
subscriptionData);
switch (pullResult.getPullStatus()) {
case FOUND:
long prevRequestOffset = pullRequest.getNextOffset();
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
long pullRT = System.currentTimeMillis() - beginTimestamp;
DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullRT(pullRequest.getConsumerGroup(),
pullRequest.getMessageQueue().getTopic(), pullRT);
long firstMsgOffset = Long.MAX_VALUE;
if (pullResult.getMsgFoundList() == null || pullResult.getMsgFoundList().isEmpty()) {
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
} else {
firstMsgOffset = pullResult.getMsgFoundList().get(0).getQueueOffset();
DefaultMQPushConsumerImpl.this.getConsumerStatsManager().incPullTPS(pullRequest.getConsumerGroup(),
pullRequest.getMessageQueue().getTopic(), pullResult.getMsgFoundList().size());
boolean dispatchToConsume = processQueue.putMessage(pullResult.getMsgFoundList());
DefaultMQPushConsumerImpl.this.consumeMessageService.submitConsumeRequest(
pullResult.getMsgFoundList(),
processQueue,
pullRequest.getMessageQueue(),
dispatchToConsume);
if (DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval() > 0) {
DefaultMQPushConsumerImpl.this.executePullRequestLater(pullRequest,
DefaultMQPushConsumerImpl.this.defaultMQPushConsumer.getPullInterval());
} else {
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
}
}
if (pullResult.getNextBeginOffset() < prevRequestOffset
|| firstMsgOffset < prevRequestOffset) {
log.warn(
"[BUG] pull message result maybe data wrong, nextBeginOffset: {} firstMsgOffset: {} prevRequestOffset: {}",
pullResult.getNextBeginOffset(),
firstMsgOffset,
prevRequestOffset);
}
break;
case NO_NEW_MSG:
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
DefaultMQPushConsumerImpl.this.correctTagsOffset(pullRequest);
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
break;
case NO_MATCHED_MSG:
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
DefaultMQPushConsumerImpl.this.correctTagsOffset(pullRequest);
DefaultMQPushConsumerImpl.this.executePullRequestImmediately(pullRequest);
break;
case OFFSET_ILLEGAL:
log.warn("the pull request offset illegal, {} {}",
pullRequest.toString(), pullResult.toString());
pullRequest.setNextOffset(pullResult.getNextBeginOffset());
pullRequest.getProcessQueue().setDropped(true);
DefaultMQPushConsumerImpl.this.executeTaskLater(new Runnable() {
@Override
public void run() {
try {
DefaultMQPushConsumerImpl.this.offsetStore.updateOffset(pullRequest.getMessageQueue(),
pullRequest.getNextOffset(), false);
DefaultMQPushConsumerImpl.this.offsetStore.persist(pullRequest.getMessageQueue());
DefaultMQPushConsumerImpl.this.rebalanceImpl.removeProcessQueue(pullRequest.getMessageQueue());
log.warn("fix the pull request offset, {}", pullRequest);
} catch (Throwable e) {
log.error("executeTaskLater Exception", e);
}
}
}, 10000);
break;
default:
break;
}
}
}
@Override
public void onException(Throwable e) {
if (!pullRequest.getMessageQueue().getTopic().startsWith(MixAll.RETRY_GROUP_TOPIC_PREFIX)) {
log.warn("execute the pull request exception", e);
}
DefaultMQPushConsumerImpl.this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_EXCEPTION);
}
};
boolean commitOffsetEnable = false;
long commitOffsetValue = 0L;
if (MessageModel.CLUSTERING == this.defaultMQPushConsumer.getMessageModel()) {
commitOffsetValue = this.offsetStore.readOffset(pullRequest.getMessageQueue(), ReadOffsetType.READ_FROM_MEMORY);
if (commitOffsetValue > 0) {
commitOffsetEnable = true;
}
}
String subExpression = null;
boolean classFilter = false;
SubscriptionData sd = this.rebalanceImpl.getSubscriptionInner().get(pullRequest.getMessageQueue().getTopic());
if (sd != null) {
if (this.defaultMQPushConsumer.isPostSubscriptionWhenPull() && !sd.isClassFilterMode()) {
subExpression = sd.getSubString();
}
classFilter = sd.isClassFilterMode();
}
int sysFlag = PullSysFlag.buildSysFlag(
commitOffsetEnable, // commitOffset
true, // suspend
subExpression != null, // subscription
classFilter // class filter
);
try {
// Let's continue to follow up this method, which is how the client pulls messages
this.pullAPIWrapper.pullKernelImpl(
pullRequest.getMessageQueue(),
subExpression,
subscriptionData.getExpressionType(),
subscriptionData.getSubVersion(),
pullRequest.getNextOffset(),
this.defaultMQPushConsumer.getPullBatchSize(),
sysFlag,
commitOffsetValue,
BROKER_SUSPEND_MAX_TIME_MILLIS,
CONSUMER_TIMEOUT_MILLIS_WHEN_SUSPEND,
// The communication mode of messages is asynchronous
CommunicationMode.ASYNC,
pullCallback
);
} catch (Exception e) {
log.error("pullKernelImpl exception", e);
this.executePullRequestLater(pullRequest, PULL_TIME_DELAY_MILLS_WHEN_EXCEPTION);
}
}
Send remote request pull message
In the MQClientAPIImpl.pullMessage method, there are two methods: asynchronous pull and synchronous pull according to the value of the input parameter communicationMode.
Whether asynchronous or synchronous pull, a ResponseFuture object will be constructed before sending the pull request. The serial number of the request message is the key value and stored in the nettyremotengabstract.responsetable: concurrenthashmap, ResponseFuture > variable. There are several situations for this variable:
- Delete the corresponding record in the responseTable variable directly after sending fails;
- After receiving the response message, it will find the ResponseFuture object from the responseTable with the serial number in the response message (returned by the server as it is according to the serial number of the request message) and set the responseCommand variable of the object. If it is sent synchronously, it will wake up the ResponseFuture.waitResponse method waiting for response; If it is sent asynchronously, the ResponseFuture.executeInvokeCallback() method will be called to complete the callback logic processing;
- When NettyRemotingClient.start() starts, it will also initialize the scheduled task. The scheduled task will scan the responseTable list regularly every 1 second, traverse the ResponseFuture object in the list, and check whether the waiting for response times out. If it times out, call the ResponseFuture. executeInvokeCallback() method, and delete the object from the responseTable list;
public PullResult pullMessage(
final String addr,
final PullMessageRequestHeader requestHeader,
final long timeoutMillis,
final CommunicationMode communicationMode,
final PullCallback pullCallback
) throws RemotingException, MQBrokerException, InterruptedException {
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.PULL_MESSAGE, requestHeader);
switch (communicationMode) {
case ONEWAY:
assert false;
return null;
case ASYNC:
this.pullMessageAsync(addr, request, timeoutMillis, pullCallback);
return null;
case SYNC:
return this.pullMessageSync(addr, request, timeoutMillis);
default:
assert false;
break;
}
return null;
}
Synchronous pulling
For the synchronous sending method, call the mqclientapimpl.pullmessagesync (string addr, remotingcommand request, long timeoutmillis) method. The general steps are as follows:
- Call the RemotingClient.invokeSync(String addr, RemotingCommand request, long timeoutMillis) method:
- Get the Channel information of the broker address. Obtain the ChannelWrapper object from the remotingclient.channelTables: concurrenthashmap, ChannelWrapper > variable according to the broker address and return the Channel variable of the object; If there is no ChannelWrapper object, establish a new connection with the broker address and store the connection information in the channelTables variable for next use;
- If the NettyRemotingClient.rpcHook:RPCHook variable is not empty (the variable initializes DefaultMQPushConsumer in the application layer or the DefaultMQPullConsumer object passes in the value), call the rpchook.dobeforequest (string remoteaddr, remotingcommand request) method;
- Call nettyremotengabstract.invokesyncimpl (channel, remotingcommand request, long timeoutmillis) method. The logic of this method is as follows:
- A) Initialize the ResponseFuture object with the requested sequence number (opaue) and timeout; And save the ResponseFuture object into the NettyRemotingAbstract.responseTable: ConcurrentHashMap variable;
- B) Call the Channel.writeAndFlush(Object msg) method to send the request object RemotingCommand to the Broker; Then the addListener (GenericFutureListener<? Extends Future<? Super Void>> listener) method is called to add the internal anonymous class: the internal anonymous class implements the operationComplete method of the ChannelFutureListener interface, and after the completion of the sending, the operationComplete method of the listening class is callback. In this method, the ChannelFuture. ChannelFuture. () is first called. Method to check whether the sending is successful. If successful, set sendRequestOK of the ResponseFuture object equal to true, exit the callback method and wait for the response result; If it fails, set sendRequestOK of the ResponseFuture object equal to false, then delete the record of the request serial number (opaue) from nettyremotengabstract.responsetable, set the responseCommand of the ResponseFuture object equal to null, and wake up the waiting of the ResponseFuture.waitResponse(long timeoutMillis) method;
- C) Call the ResponseFuture.waitResponse(long timeoutMillis) method to wait for the response result; In case of sending failure, receiving response message (see section 5.10.3 for details) or timeout, the method will wake up and return the value of ResponseFuture.responseCommand variable;
- D) If the responseCommand value returned in the previous step is null, an exception will be thrown: if ResponseFuture.sendRequestOK is true, a RemotingTimeoutException exception will be thrown; otherwise, a RemotingSendRequestException exception will be thrown;
- E) If the responseCommand value returned in the previous step is not null, the responseCommand variable value is returned;
- If the NettyRemotingClient.rpcHook: RPCHook variable is not empty, call the RPCHook.doAfterResponse(String remoteAddr, RemotingCommand request) method;
- The return value of the above step is the RemotingCommand object. Call the mqclientapimpl. Processpullresponse (RemotingCommand response) method as a parameter, parse and encapsulate the return object into a PullResultExt object, and then return it to the caller. The result state of the response message is transformed as follows:
- If the Code of the RemotingCommand object is equal to SUCCESS, PullResultExt.pullStatus=FOUND;
- If the Code of the RemotingCommand object is equal to PULL_NOT_FOUND, then PullResultExt.pullStatus= NO_NEW_MSG；
- If the Code of the RemotingCommand object is equal to PULL_RETRY_IMMEDIATELY, then PullResultExt.pullStatus= NO_MATCHED_MSG；
- If the Code of the RemotingCommand object is equal to PULL_OFFSET_MOVED, then PullResultExt.pullStatus= OFFSET_ILLEGAL；
@Override
public RemotingCommand invokeSync(String addr, final RemotingCommand request, long timeoutMillis)
throws InterruptedException, RemotingConnectException, RemotingSendRequestException, RemotingTimeoutException {
long beginStartTime = System.currentTimeMillis();
final Channel channel = this.getAndCreateChannel(addr);
if (channel != null && channel.isActive()) {
try {
if (this.rpcHook != null) {
this.rpcHook.doBeforeRequest(addr, request);
}
long costTime = System.currentTimeMillis() - beginStartTime;
if (timeoutMillis < costTime) {
throw new RemotingTimeoutException("invokeSync call timeout");
}
RemotingCommand response = this.invokeSyncImpl(channel, request, timeoutMillis - costTime);
if (this.rpcHook != null) {
this.rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(channel), request, response);
}
return response;
} catch (RemotingSendRequestException e) {
log.warn("invokeSync: send request exception, so close the channel[{}]", addr);
this.closeChannel(addr, channel);
throw e;
} catch (RemotingTimeoutException e) {
if (nettyClientConfig.isClientCloseSocketIfTimeout()) {
this.closeChannel(addr, channel);
log.warn("invokeSync: close socket because of timeout, {}ms, {}", timeoutMillis, addr);
}
log.warn("invokeSync: wait response timeout exception, the channel[{}]", addr);
throw e;
}
} else {
this.closeChannel(addr, channel);
throw new RemotingConnectException(addr);
}
}
getMQClientAPIImpl().pullMessage is finally written to and refreshed in the queue through channel. Then, the general processing logic at the message server is that after the server receives a new message request, if there is no message in the queue and is not in a hurry to return, it passes through a cycle state, and each time waitForRunning for a period of time, the default is 5 seconds, and then check. If the broker has no new message, the third check time will return empty when the time exceeds SuspendMaxTimeMills, If a new message is received during the waiting process, directly call the notifyMessageArriving function to return the request result. The core of "long polling" is that the broker holds the request from the client for a short period of time. If a new message arrives within this time, it will immediately return the message to the consumer using the existing connection. The initiative of long polling rests with the consumer. Even if the broker has a large number of messages, it will not actively push them to the consumer.