Programmer Group

In version 2.3 of Kafka, the Kafka connector has been greatly improved. First, when adding and deleting connectors, the way Kafka connectors handle tasks is modified. Previously, this action brought the whole system to a standstill, which has been criticized by the development and maintenance personnel. In addition, other problems frequently me ...

1.flume configuration Create a file that flume listens to instead of a kafka producer to transfer content to consumers. The configuration is as follows ###############to sinnsource,channel give a name###############a kafka_agent.sources = kafka_source kafka_agent.sinks = kafka_sink kafka_agent.channels = kafka_channel ##############To ...

1. @KafkaListener message listening springboot Kafka uses the @KafkaListener annotation to listen for consumer messages. Message listening can be divided into two types: single data consumption and batch consumption. The difference is the number of ...

Welcome to pay attention to personal Wechat Public Number: Xiaoha learns Java, pushes dry articles in the Java field every day, and pays attention to free and no routine with 100G mass learning and interview resources yo!! Personal website: https:// ...

Original address: http://blog.csdn.net/shangboerds/article/details/38944743 -- Start 1. Start ZooKeeper Start ZooKeeper first. set KAFKA_HOME=C:/dev/kafka_2.10-0.8.2.2   cd %KAFKA_HOME%/bin/windows   zookeeper-server-start.bat %KAFKA_HOME%/config/zookeeper.properties   2. Start Kafka borker First, copy ...

1. The problem of too much self-log After running kafka for a period of time, it will find that its host disk usage is slowly increasing, check the amount of data log holdings or the threshold set before. At this time, it is kafka's own log print booster disk. The default ~/kafka_2.11-0.9.0.0/config/log4j.properties are as follows: log ...

I. server.properties in kafka configuration file #broker's global unique number, not duplicated broker.id=0 #The port used to listen for links, where producer or consumer will establish a connection port=9092 #Number of threads processing network requests num.network.threads=3 #Off-the-shelf quantities used to process disk IO nu ...

Create kafka topic View all topic lists View the specified topic information Console to topic Production Data Console consumes topic data View the maximum (minimum) offset of a partition in topic Increase the number of topic partitions Delete topic, use cautiously, only delete metadata in zookeeper, message file must be deleted m ...

A Brief Introduction to flume When you read this article, you should have a general understanding of flume, but in order to take care of the beginners, you will still say flume. When you first use flume, you don't need to understand too much about it. Just understand the following figure, you can use flume to transfer log data into kafka. ...

Last time, when the device sends data, we need to see what the data sent by the device is on the management page, so now we can complete the device data monitoring module. iot-device-data Create iot-device-data module, because this module is also a subscription board service, so it also consumes Mapping data in kakfa and introduces the correspo ...