Programmer Group

Install nginx Kafka plug-in nginx can directly write data into kafka. 1. install git yum install -y git 2. Switch to the directory / usr/local/src, and then clone the c client source code of kafka to local cd /usr/local/src git clone https://github.com/edenhill/librdkafka 3. Enter librdkafka and compile ...

new function Allow consumers to get from the most recent copy Increase support for incremental cooperative rebalancing for consumer rebalancing protocol New MirrorMaker 2.0 (MM2), new multi cluster cross data center replication engine Introducing a new Java authorized program interface Supports non key connections in KTable Adminis ...

At present, there are several commonly used Golang Kafka clients, each of which has its own advantages and disadvantages Client name Advantages and disadvantages sarama The number of users is relatively large, but it is relatively difficult to use, with better performance confluent-kafka-go The encapsulation of Kafka in C language has st ...

Foreword: Recent project of combining kafka and zookeeper containerization with rancher, I consulted related websites and books and found that there are many reasons why I chose to customize it if it is relatively strong to standardize the company. Let me briefly talk about the reasons why I chose to customize it finally: (Because k8s+kakfa te ...

1. Download Kafka? 2.12-2.0.0.tgz and upload it to / usr/local/kafka / Extract and remove the installation package 2. configuration Modify the config/server.properties of each server cd /usr/local/kafka/kafka_2.12-2.0.0/config vim server.properties Modify several places:broker.id : unique, fill in the number, 10,11,12,13,1 ...

Environmental preparation Kafka Version: kafka_2.10-0.10.2.0 (2.10 for Scala version 0.10.2.0 for Kafka version) Official download address: http://kafka.apache.org/downloads Zookeeper Version: zookeeper-3.4.13 Official download address: http://zookeeper.apache.org/releases.html JDK Version: jdk1.8.0_ Official download address: h ...

Main references https://www.cnblogs.com/00986014w/p/9561901.html This blog post, but he's not using the official image of zookeeper I use the Zookeeper cluster with three nodes, which can be modified by reference. ##Build Zookeeper cluster zookeeper-svc.yaml of cluster Service apiVersion: v1 kind: Service metadata: labels: app: zookeeper ...

If you need to collect IIS logs in your work, you can use the filebeat component; Log format resolution of iis: Log example: #Software: Microsoft Internet Information Services 7.5 #Version: 1.0 #Date: 2019-03-14 00:00:00 #Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc- ...

Huge buildings are always made of one wood and one stone. Why don't we make them? I often do some odd things, that's why. That's right, but I didn't say that! ——Lu Xun brew is the installation package tool of mac system. Similar to yum or apt get. What is the package path of brew installation. The following is an exam ...

Flume collects Log4j logs and sends them to Kafka for storage Environmental preparation Download Flume: http://flume.apache.org/ Install: extract the download package to the custom path Configure agent # Function: filter json information storage kafka agent.sources = s1 agent.channels = c1 agent.sinks = k1 agent.sources.s1.type = avro agen ...