YDB is deployed in Mini Centos environment based on haoop, zookeeper and kafka

YDB is deployed in Mini Centos environment based on haoop, zookeeper and kafka YDB introduction YDB full name Yan Yun YDB It is a real-time, multi-dimensional, interactive query, statistics and analysis engine based on Hadoop distributed architecture. It has second-level performance under trillion data scale and stable and reliable perform ...

Posted by narked on Wed, 13 Feb 2019 13:12:18 -0800

kafka Error Failed to send messages after 3 tries Solution

Originally, I was going to try to play with kafka server by myself, but I didn't expect to hit nails when I came up, so I couldn't toss around for a long time. Environmental Science: kafka 2.9.2 - 0.0.2.2 Phenomenon: Create themes and produce data within the server No problem receiving with the same server Call kafka with java on another ...

Posted by utahcon on Sat, 09 Feb 2019 18:09:22 -0800

Kafka (12): integration of Kafka and flume

I. Realizing Functions Flume monitors a log file and sends the data to kafka, and then another flume gets the data from kafka. II. Implementation steps 1. environment flume1.6.0 kafka_2.10-0.8.2.1 zookeeper3.4.5 2.flume monitors logs and sends them to kafka's configuration file test1_1.6.conf [reference: http://archive. ...

Posted by cutups on Tue, 29 Jan 2019 11:15:15 -0800

Simple Application Example of java Kafka

There are many ways to install kafka: 1. Single-machine mode (divided into windows mode; linux mode); 2. Pseudo-distributed mode; 3. Distributed mode; Specific building methods for reference: https://blog.csdn.net/xlgen157387/article/details/77312569?utm_source=blogxgwz0 The following is the stand-alone mode under Linux: 1. In ...

Posted by phpusers on Mon, 28 Jan 2019 00:33:17 -0800

How to avoid registering hostname of Kafka broker machine into zookeeper

Cause, when using mirror-maker to test cluster copy data from production cluster top, the error is as follows: [2018-10-23 10:21:47,821] FATAL [mirrormaker-thread-2] Mirror maker thread failure due to  (kafka.tools.MirrorMaker$MirrorMakerThread) java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutEx ...

Posted by rish1103 on Sat, 26 Jan 2019 14:00:16 -0800

Kafka-Kafka-Java extension query specifies all consumer-group s of topic

Reference article: https://www.bbsmax.com/A/n2d9bqDvzD/   The corresponding Kafka version: kafka_2.12-2.0.0.jar, namely scala 2.12, kafka 2.0.0 version.   Note the kafka version and recommend 1.0.0 + version. We usually use the following way to query the group.id of all consumption:   kafka-consumer-groups.sh --bootstra ...

Posted by speckledapple on Fri, 25 Jan 2019 15:21:13 -0800

callLog Case Analysis of Big Data Telephone Log (4)

1. Modify the storage time of kafka data in the subject by default of 7 days ------------------------------------------------- [kafka/conf/server.properties] log.retention.hours=1 2. aggregate query using hive ---------------------------------------------------- 1.hive command line query // Query all calle ...

Posted by shortcircuit on Sun, 20 Jan 2019 20:24:12 -0800

Centralized Log Analysis Platform - ELK Stack - Filebeat Pressure Measurement

The article is reproduced only for self-collection. See the original article. https://www.jianshu.com/p/0802cf22a135 Preface Any collection agent needs performance testing and resource limitation function testing before it can be fully promoted in the company to ensure that: Know clearly where the performance bottleneck is. Understand clear ...

Posted by sarika on Thu, 10 Jan 2019 22:21:10 -0800