Programmer Group

YDB is deployed in Mini Centos environment based on haoop, zookeeper and kafka YDB introduction YDB full name Yan Yun YDB It is a real-time, multi-dimensional, interactive query, statistics and analysis engine based on Hadoop distributed architecture. It has second-level performance under trillion data scale and stable and reliable perform ...

Originally, I was going to try to play with kafka server by myself, but I didn't expect to hit nails when I came up, so I couldn't toss around for a long time. Environmental Science: kafka 2.9.2 - 0.0.2.2 Phenomenon: Create themes and produce data within the server No problem receiving with the same server Call kafka with java on another ...

I. Realizing Functions Flume monitors a log file and sends the data to kafka, and then another flume gets the data from kafka. II. Implementation steps 1. environment flume1.6.0 kafka_2.10-0.8.2.1 zookeeper3.4.5 2.flume monitors logs and sends them to kafka's configuration file test1_1.6.conf [reference: http://archive. ...

There are many ways to install kafka: 1. Single-machine mode (divided into windows mode; linux mode); 2. Pseudo-distributed mode; 3. Distributed mode; Specific building methods for reference: https://blog.csdn.net/xlgen157387/article/details/77312569?utm_source=blogxgwz0 The following is the stand-alone mode under Linux: 1. In ...

Cause, when using mirror-maker to test cluster copy data from production cluster top, the error is as follows: [2018-10-23 10:21:47,821] FATAL [mirrormaker-thread-2] Mirror maker thread failure due to  (kafka.tools.MirrorMaker$MirrorMakerThread) java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutEx ...

Reference article: https://www.bbsmax.com/A/n2d9bqDvzD/   The corresponding Kafka version: kafka_2.12-2.0.0.jar, namely scala 2.12, kafka 2.0.0 version.   Note the kafka version and recommend 1.0.0 + version. We usually use the following way to query the group.id of all consumption:   kafka-consumer-groups.sh --bootstra ...

1. Modify the storage time of kafka data in the subject by default of 7 days ------------------------------------------------- [kafka/conf/server.properties] log.retention.hours=1 2. aggregate query using hive ---------------------------------------------------- 1.hive command line query // Query all calle ...

The article is reproduced only for self-collection. See the original article. https://www.jianshu.com/p/0802cf22a135 Preface Any collection agent needs performance testing and resource limitation function testing before it can be fully promoted in the company to ensure that: Know clearly where the performance bottleneck is. Understand clear ...