Version information

Kafka: 0.10.2,Scala: 2.10.6,Java: 1.8.0_221,IntelliJ IDEA: 2019.2,Zookeeper: 3.4.13,Gradle: 5.6.2,Git: 2.22.0

2. Building Kafka Source Environment

From Git GitHub Upper Clone has a Kafka repository and switches to version 0.10.2.

# Create a new Kafka directory on the D: drive and bring the Kafka project Clone here
git clone https://github.com/apache/kafka.git
# Switch to version 0.10.2
git checkout origin/0.10.2

There are a few points to confirm before building a Kafka project with Gradle:

1. Make sure the scalaVersion in the gradle.properties configuration file is consistent with the Scala version in Windows, and the 0.10.2 version of Kafka is the 2.10.6 version of Cala.

2. Modify two places in the build.gradle file:

   2.1. Modify the version number of org.scoverage:gradle-scoverage:

   dependencies {
       // For Apache Rat plugin to ignore non-Git files
       classpath "org.ajoberstar:grgit:1.7.0"
       classpath 'com.github.ben-manes:gradle-versions-plugin:0.13.0'
       classpath 'org.scoverage:gradle-scoverage:2.5.0' // This was originally 2.1.0 modified to 2.5.0
     }  
   

   Correct previous errors:

   D:\kafka>gradle idea
   Starting a Gradle Daemon (subsequent builds will be faster)
   
   > Configure project :
   Building project 'core' with Scala version 2.10.6
   
   FAILURE: Build failed with an exception.
   
   * Where:
   Build file 'D:\work\Apache-Projects\kafka\build.gradle' line: 388
   
   * What went wrong:
   A problem occurred evaluating root project 'kafka'.
   > Failed to apply plugin [id 'org.scoverage']
      > Could not create an instance of type org.scoverage.ScoverageExtension.
         > You can't map a property that does not exist: propertyName=testClassesDir
   
   

   2.2. Configure Gradle's warehouse as Ali's mirror warehouse

   // Configure Gradle's warehouse as Ali's mirror warehouse
   repositories {
       maven {
         url 'http://maven.aliyun.com/nexus/content/groups/public/'
       }
       mavenCentral()
     }
   

   Correct previous errors:

   D:\kafka>gradle idea
   
   > Configure project :
   Building project 'core' with Scala version 2.10.6
   
   FAILURE: Build failed with an exception.
   
   * What went wrong:
   A problem occurred configuring project ':core'.
   > Could not resolve all files for configuration ':core:scoverage'.
      > Could not resolve org.scoverage:scalac-scoverage-plugin_2.10:1.3.0.
        Required by:
            project :core
         > Could not resolve org.scoverage:scalac-scoverage-plugin_2.10:1.3.0.
            > Could not get resource 'https://repo.maven.apache.org/maven2/org/scoverage/scalac-scoverage-plugin_2.10/1.3.0/scalac-scoverage-plugin_2.10-1.3.0.pom'.
               > Could not GET 'https://repo.maven.apache.org/maven2/org/scoverage/scalac-scoverage-plugin_2.10/1.3.0/scalac-scoverage-plugin_2.10-1.3.0.pom'.
                  > Connect to repo.maven.apache.org:443 [repo.maven.apache.org/151.101.40.215] failed: Connection timed out: connect
      > Could not resolve org.scoverage:scalac-scoverage-runtime_2.10:1.3.0.
        Required by:
            project :core
         > Could not resolve org.scoverage:scalac-scoverage-runtime_2.10:1.3.0.
            > Could not get resource 'https://repo.maven.apache.org/maven2/org/scoverage/scalac-scoverage-runtime_2.10/1.3.0/scalac-scoverage-runtime_2.10-1.3.0.pom'.
               > Could not GET 'https://repo.maven.apache.org/maven2/org/scoverage/scalac-scoverage-runtime_2.10/1.3.0/scalac-scoverage-runtime_2.10-1.3.0.pom'.
                  > Connect to repo.maven.apache.org:443 [repo.maven.apache.org/151.101.40.215] failed: Connection timed out: connect
   
   

After modifying the above file, you can execute the gradle idea directly in the kafka root directory and open it with IDEA after success.

3. Configuring Kafka Source Environment

1. IDEA installs the Scala plug-in, searches for Scala in File -> Settings -> Plugin, and installs it.

   

2. Configure the Scala SDK in File -> Settings -> ProjectStructure

   

3. Copy the log4j.properties file from the config directory to the core/src/main/scala directory, which allows Kafka to output log information at runtime.
   

4. Configure server.properties file

   # It runs in a Windows environment, so you need to modify this configuration, notice the double backslash here
# This exists in the profile, modify the value
   log.dirs=D:\\kafka\\tmp\\kafka-logs 
# Turn on this feature to allow Kafka to delete unwanted temporary topic s after running for a while
   # New Configuration Item
delete.topic.enable=true
   # Disable automatic topic creation
   # New Configuration Item
   auto.create.topics.enable=false
   # If there is no zookeeper service locally, you can refer to the installation, configuration, and operation of zookeeper in the next section
   # This configuration item exists in the file and is modified as appropriate
   zookeeper.connect=localhost:2181/kafka
   

Build a bin package

gradlew clean releaseTarGz

This typically builds the project, followed by the tar package

If you only change what's inside the core project, just execute gradlew: jar_core_2_10 to package the core.Then replace the jar package of Kafka in lib in the Kafka directory of the cluster.