kafka supports transactions from version 0.2.11. This document gives a brief description of kafka transactions, java code examples, some simple descriptions of the code, and relevant precautions. I hope it can be helpful to friends who need to use kafka transactions.

On June 28, 2017, Kafka officially released version 0.11.0.0. From this version, Kafka supports transactions. So what is a transaction in Kafka?

kafka transaction support producers can send a set of messages as a single transaction, which either succeeds or fails atomically. For example, after paying an order, the user needs to notify the inventory module to reduce the inventory and the coupon model to deduct the coupon. You need to make both messages succeed or fail. The kafka transaction can be used at this time.

Let's look at the code first.
Step 1: introduce dependency and add kafka client dependency in pom.xml

 <dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>2.7.0</version>
</dependency>

Step 2: send message related codes

Properties props = new Properties();
props.put("bootstrap.servers", "10.20.200.166:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("transactional.id", "my-transactional-id");
KafkaProducer<String, String> producer = new KafkaProducer<>(props);

producer.initTransactions();

int c =0;
while (true) {
    if(c++>=3){
        break;
    }
    try {
        producer.beginTransaction();

        ProducerRecord<String, String> record = new ProducerRecord<>("TEST1", "message a " + c);
        producer.send(record);
        System.out.println("send TEST1 " + c);

        ProducerRecord<String, String> record2 = new ProducerRecord<>("TEST2", "message b " + c);
        producer.send(record2);
        System.out.println("send TEST2 " + c);

        producer.commitTransaction();
    }catch (RuntimeException e){
        System.out.println(e.getMessage());
        producer.abortTransaction();
    }
}

1. The first six lines are to build a KafkaProducer instance. In the spring boot project, this total score is usually encapsulated into a method, and then the object injection management is completed through @ Bean. bootstrap.servers corresponds to the address of your kafka server, which needs to be modified according to your local actual situation. This kind of address should also be placed in the configuration file.
2. Line 8 producer.initTransactions(); It means to initialize the kafka transaction. This method can be called once for each producer.
3. Next is a loop control. This is to express that the producer object can be reused to send multiple transaction messages. For each transaction, producer. Begntransaction() indicates the beginning of a transaction, producer.send(record) indicates the message to be sent in this transaction. In each transaction, the send method can be called multiple times, depending on the business requirements, and then submit the transaction through producer.commitTransaction(). If an exception occurs, Then cancel the transaction through producer.abortTransaction().

matters needing attention:

1. The value of transactional.id cannot be repeated. If there is only a single node in your environment, this value can be directly used as a fixed string. But if your program needs to support horizontal expansion, such as two or more servers running your code at the same time, there will be problems at this time. For the same transactional.id, after a new KafkaProducer calls initTransactions, the original process will report an error. Only the latest processes work properly. Therefore, at this time, you need to ensure that the values of transactional.id are different when different nodes run. You can use UUID.randomUUID().toString() to generate a random ID that is guaranteed not to be repeated, or directly configure different transactional. IDS in different instance servers. For multiple transactions running on the same node, the same KafkaProducer can be used, and the same transactional.id can be used.

Reference documents:
https://www.confluent.io/blog...