Programmer Group

1. Introduction to Kafka   1.1. Main functions According to the introduction of the official website, ApacheKafka ® is a distributed streaming media platform, which has three main functions: 1: It lets you publish and subscribe to streams of records 2: it lets you store streams of records in a fault tolerant way 3: It lets you process ...

This paper uses Spring cloud as 2.1.8 RELEASE, version=Greenwich.SR3 This paper is based on the implementation of the first two articles, eureka-server, eureka-client, eureka-ribbon and spring-gateway. Reference resources eureka-server eureka-client eureka-ribbon spring-cloud-stream Summary Spring Cloud Stream is a framework for building ...

Note: the number of partitions num.partitions is specified in the configuration file server.properties. This refers to the sum of the number of partitions of multiple single topic s. If there are multiple brokers, the partition may be distributed on different nodes, then the total number of all partitions of multiple brokers is ...

Kafka consumer Reliability assurance As a consumer, consumption data needs to consider: 1. No repeated consumption 2. No lack of consumption information Partition allocation policy There are multiple consumers in a consumer group and multiple partitions in a topic, so it is inevitable to involve the allocation of partitions, that is, to determi ...

Last article I talked about how to integrate kafka with spring cloud stream and run a demo. If I publicize the article of spring cloud stream this time, I can do it here. But in fact, engineering is never a simple problem of technology or not. In the actual development, we will encounter a lot of details (referred to as pit). In this article, w ...

In fact, the best way to learn is to communicate with each other. Recently, I also discussed some issues about RocketMQ message pulling and balancing with netizens. Let me write down my summary here. On the problem of message loop pull in push mode A previous article on rebalancing: Kafka rebalancing mechanism ", it is said that the Rocket ...

E-commerce platform source code Please add Penguin: 103874626. kafka message platform uses spring+kafka integration scheme, details are as follows: Use the maximum version 2.1.0.RELEASE integration jar package: Spring integration Kafka Zookeeper and Kafka distributed clusters use init.properties configuration schem ...

I. application scenario of timing start For example, in the stand-alone environment, we need to use Kafka for data persistence. Since the active time of users is from 10 a.m. to 12 p.m., a large amount of data persistence in this time period may affect the performance of the database and lead to the decrease of user experience. We can choo ...

Preface In Learn Flink from 0 to 1: introduction to Data Source In this article, I will introduce Flink Data Source and user-defined Data Source briefly. In this article, I will introduce it in more detail and write a demo for you to understand. Flink Kafka source Preparation Let's take a look at the demo for Flink to get data from Kafka ...

Explain:1. Only two topic s are synchronized this time, and subsequent optimization will continue to update...2. Self built cluster CDH5.8, kafka2.1.0; Alibaba cloud cluster Standard Version kafka0.10.xTrample:1. Add the CMM of kafka role instance in cdh, which should not support SSL connection.2. VPC network access. I don't know that the purc ...