Transaction messages are a very important feature provided by RocketMQ. They are open source after version 4.x and can be used to easily implement distributed transactions.This article gives a detailed description of the transaction messages for RocketMQ and gives a code example.

I. Related Concepts

On the basis of its message definition, RocketMQ extends two related concepts to transactional messages:

1. Half(Prepare) Message - Semi-Message (Preprocessed Message)

Semi-messages are a special type of messages that cannot be consumed by consumers for the time being.When a transaction message is successfully delivered to the Broker, but the Broker does not receive a second confirmation from the Producer, the transaction message is in a "temporary non-consumable" state, which is called a semi-message.

1. Message Status Check - Message Status Check

Due to network jitter and restart of Producer, the second acknowledgement message sent by Producer to Broker may not be delivered successfully.If Broker detects that a transaction message has been in a semi-message state for a long time, it actively initiates a lookup operation to the Producer side to query the transaction state (Commit or Rollback) of the transaction message on the Producer side.As you can see, Message Status Check is primarily used to resolve timeouts in distributed transactions.

2. Execution process

The above is the transaction message execution flowchart provided by the official network, and the following is an analysis of the specific process:

1. Step1:Producer sends Half Message to the Broker side;
2. Step2:Broker ACK, Half Message sent successfully;
3. Step3:Producer performs local transactions;
4. Step4: When the local transaction is complete, Producer sends a second confirmation message to the Broker to confirm the Commit or Rollback status of the Half Message, depending on the state of the transaction.When Broker receives a second confirmation message, it sends the Commit status directly to the Consumer side to execute the consumption logic, while it marks the Rollback as a failure and clears it after a period of time without sending it to the Consumer.Normally, when this distributed transaction is complete, the only thing left to deal with is the timeout problem, where Broker still does not receive a second confirmation message from Producer after some time.
5. Step5: For timeout status, Broker actively initiates a message lookup to Producer;
6. Step6:Producer processes the lookup message and returns the execution result of the corresponding local transaction;
7. Step7:Broker performs a Commit or Rollback operation on the result of a message lookup, the same as Step4.

3. Code examples

This section simulates the transaction messages of RocketMQ through a simple scenario: there are two micro-services, order service and commodity service.The order service processes the order and sends a message to the merchandise service to reduce the inventory of the successfully placed merchandise.

First, order services:

/**
 * @Auther: ZhangShenao
 * @Date: 2019/3/27 16:44
 * @Description:Use RocketMQ transaction message - Order service sends transaction message, then place local order and notify merchandise service to reduce inventory
 */
public class OrderService {
  public static void main(String[] args) throws Exception {
    TransactionMQProducer producer = new TransactionMQProducer();
    producer.setNamesrvAddr(RocketMQConstants.NAMESRV_ADDR);
    producer.setProducerGroup(RocketMQConstants.TRANSACTION_PRODUCER_GROUP);

    //Customize the thread pool to perform transactional operations
    ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 50, 10L, TimeUnit.SECONDS, new ArrayBlockingQueue<>(20), (Runnable r) -> new Thread("Order Transaction Massage Thread"));
    producer.setExecutorService(executor);

    //Set Transaction Message Listener
    producer.setTransactionListener(new OrderTransactionListener());

    producer.start();

    System.err.println("OrderService Start");

    for (int i = 0;i < 10;i++){
      String orderId = UUID.randomUUID().toString();
      String payload = "Place an order,orderId: " + orderId;
      String tags = "Tag";
      Message message = new Message(RocketMQConstants.TRANSACTION_TOPIC_NAME, tags, orderId, payload.getBytes(RemotingHelper.DEFAULT_CHARSET));

      //Send Transaction Message
      TransactionSendResult result = producer.sendMessageInTransaction(message, orderId);
      System.err.println("Send Transaction Message,Send results: " + result);
    }
  }
}

Transaction messages require a TransactionListener, which mainly performs execution and transaction review of local transactions with the following code:

/**
 * @Auther: ZhangShenao
 * @Date: 2019/3/27 16:50
 * @Description:Order Transaction Message Listener
 */
public class OrderTransactionListener implements TransactionListener {
  private static final Map<String, Boolean> results = new ConcurrentHashMap<>();

  @Override
  public LocalTransactionState executeLocalTransaction(Message msg, Object arg) {
    String orderId = (String) arg;

    //Log local transaction execution results
    boolean success = persistTransactionResult(orderId);
    System.err.println("Order service performs local transaction order placing,orderId: " + orderId + ", result: " + success);
    return success ? LocalTransactionState.COMMIT_MESSAGE : LocalTransactionState.ROLLBACK_MESSAGE;
  }

  @Override
  public LocalTransactionState checkLocalTransaction(MessageExt msg) {
    String orderId = msg.getKeys();
    System.err.println("Execute Transaction Message Review,orderId: " + orderId);
    return Boolean.TRUE.equals(results.get(orderId)) ? LocalTransactionState.COMMIT_MESSAGE : LocalTransactionState.ROLLBACK_MESSAGE;
  }

  private boolean persistTransactionResult(String orderId) {
    boolean success = Math.abs(Objects.hash(orderId)) % 2 == 0;
    results.put(orderId, success);
    return success;
  }
}

Here are commodity services and monitors:

/**
 * @Auther: ZhangShenao
 * @Date: 2019/3/27 17:09
 * @Description:Use RocketMQ Transaction Message - Merchandise Service receives transaction message for placing an order and reduces inventory locally if message commit succeeds
 */
public class ProductService {
  public static void main(String[] args) throws Exception {
    DefaultMQPushConsumer consumer = new DefaultMQPushConsumer();
    consumer.setNamesrvAddr(RocketMQConstants.NAMESRV_ADDR);
    consumer.setConsumerGroup(RocketMQConstants.TRANSACTION_CONSUMER_GROUP);
    consumer.subscribe(RocketMQConstants.TRANSACTION_TOPIC_NAME, "*");
    consumer.registerMessageListener(new ProductListener());
    consumer.start();
    System.err.println("ProductService Start");
  }
}

/**
 * @Auther: ZhangShenao
 * @Date: 2019/3/27 17:14
 * @Description:
 */
public class ProductListener implements MessageListenerConcurrently {
  @Override
  public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
    Optional.ofNullable(msgs).orElse(Collections.emptyList()).forEach(m -> {
      String orderId = m.getKeys();
      System.err.println("Monitor order message,orderId: " + orderId + ", Goods Services Decrease Inventory");
    });
    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
  }
}

Running OrderService and Product Service, respectively, shows that merchandise services will be notified of inventory reduction only if the transaction executes successfully.

Hear the order message, orderId: f25a7127-307e-45ce-8f83-6e0a922ebb94, Commodity Services inventory reduction
 Hear the next order message, orderId: d960171d-97c0-4e13-aa4a-c2b96102de4b, goods service inventory reduction
 Monitor next order message, orderId: 63aedaa2-ce74-4cb7-bf58-fb6a73082a73, Goods Services inventory reduction
 Hear the next order message, orderId: 25764461-70b2-44db-8296-960211179e6e, Goods Services inventory reduction
 Order Id: fb319fe7-c8be-4edf-ae4e-6108898068ca, Commodity Services inventory reduction
 Hear the next order message, orderId: 4f61a61a-7254-458a-bc10-9d4006a9f581, Goods Services inventory reduction