Programmer Group

catalogue 1, Disadvantages of traditional log collection 2, ELK collection system process 3, Build ELK system 4, Code 5, Verification effect 1, Disadvantages of traditional log collection         We know that most of us use logs to judge where the program reports errors, so that we can take a powerful medicine aga ...

1, TiDB Binlog In the previous article, we introduced the use of TiDB Binlog to synchronize data to downstream Mysql. In this article, we learn to use TiDB Binlog tool to synchronize data to custom business logic in Kafka. For example, TIDB can synchronize data with ES, MongoDB or Redis. This function is not different from the binlog function ...

What is kafka? Apache Kafka is a distributed message queue based on publish / subscribe mode. It is an open source message system written by Scala. It is mainly used in the field of big data real-time processing and used to process streaming data. It is a very classic message engine and is famous for its high performance and high availability. ...

1, Use of ConsumerThe source code analysis of Consumer mainly focuses on KafkaConsumer, which is the implementation class of the Consumer interface. KafkaConsumer provides a well encapsulated API. Developers can easily pull messages from Kafka server based on this API. In this way, developers do not care about the underlying operations such as ...

ConsumeQueue for source code analysis When sending a message, the data is displayed in the ConsumeQueue Five messages are sent continuously. The messages are of variable length. First, all information is put into the Commitlog. Each message needs to be locked when it is put into the Commitlog to ensure sequential writing. ​ After the Comm ...

kafka supports transactions from version 0.2.11. This document gives a brief description of kafka transactions, java code examples, some simple descriptions of the code, and relevant precautions. I hope it can be helpful to friends who need to use kafka transactions.On June 28, 2017, Kafka officially released version 0.11.0.0. From this version ...

When a node in the cluster suddenly goes down and goes offline, if the partitions on the node are single replica, these partitions will become unavailable, and the corresponding data will be lost before the node is restored; If the partition on a node is multi replica, the role of the leader replica on this node will be transferred to other fol ...

1. Obtain Kafka download Latest version, unzip $ tar -xzf kafka_2.13-3.0.0.tgz $ cd kafka_2.13-3.0.0 2. Start Kafka environment Run the following command in order to start the services correctly in order # Start the ZooKeeper service # Note: Soon, ZooKeeper will no longer be required by Apache Kafka. $ bin/zookeeper-server-start.sh co ...

1. Introduction Elasticsearch introduces the Cross Cluster Search (CCS Cross Cluster Search) function in version 5.3 to replace the cube node to be discarded. Similar to triple node, Cross Cluster Search is used to realize cross cluster data search. Cross Cluster Search enables you to run a single search request against one or more remote clus ...

Java Memory Model Thread synchronization Thread synchronization mechanism is a set of data access mechanism suitable for coordinating threads, which can ensure thread safety The thread synchronization mechanism provided by the java platform includes lock, volatile keyword, final keyword, static keyword, and related API s such as obje ...