About jetcd

• jetcd is the official java client tool of etcd v3. Java projects can perform various operations on etcd through this library. The latest release version is 0.5.0
• Jetcd official GitHub: https://github.com/etcd-io/jetcd
• Etcd online API documentation: https://etcd.io/docs/next/learning/api/

About jetcd combat series

"Jetcd actual combat series" is Xinchen's new original series, which aims to learn how to operate etcd with jetcd. In addition to basic addition, deletion, modification and query, it also involves etcd's unique functions such as version, monitoring, lease, etc;

Links to series articles

1. One of jetcd's actual combat: extreme speed experience
2. jetcd Practice II: basic operation
3. jetcd practice 3: advanced operations (transaction, monitoring, lease)

Overview of this article

As the beginning of jetcd actual combat series, it is mainly to prepare for the whole series, including the following contents:

1. Sort out the version information of applications and libraries involved in actual combat;
2. Deploy etcd cluster based on docker compose;
3. Create a new gradle project as the parent project of the whole actual combat series;
4. Write helloworld application to verify that jetcd can normally access etcd cluster;

Source download

• The complete source code in this actual combat can be downloaded from GitHub. The address and link information are shown in the table below( https://github.com/zq2599/blog_demos):

• There are multiple folders in the git project. kubebuilder related applications are under the jetcd tutorials folder, as shown in the red box below:

• There are several sub projects under the jetcd tutorials folder. In this article, helloworld:

Version information

The etcd used in the actual combat series is a cluster composed of three instances deployed in the docker environment:

1. etcd: 3.4.7
2. docker: 20.10.5(Community)
3. docker-compose: 1.28.5
4. jetcd: 0.5.0
5. jdk: 1.8.0_271
6. springboot: 2.4.4
7. IDEA: 2020.2.3 (Ultimate Edition)
8. gradle: 6.8.3
9. Computer operating system: macOS Big Sur 11.2.3

Deploy cluster

• Confirm that docker and docker compose are running normally
• Create a new docker-compose.yml file:

version: '3'
services:
  etcd1:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd1'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd1:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd1:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2379:2379
    volumes:
      - ./store/etcd1/data:/etcd_data
  etcd2:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd2'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd2:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd2:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2380:2379
    volumes:
      - ./store/etcd2/data:/etcd_data
  etcd3:
    image: "quay.io/coreos/etcd:v3.4.7"
    entrypoint: /usr/local/bin/etcd
    command:
      - '--name=etcd3'
      - '--data-dir=/etcd_data'
      - '--initial-advertise-peer-urls=http://etcd3:2380'
      - '--listen-peer-urls=http://0.0.0.0:2380'
      - '--listen-client-urls=http://0.0.0.0:2379'
      - '--advertise-client-urls=http://etcd3:2379'
      - '--initial-cluster-token=etcd-cluster'
      - '--heartbeat-interval=250'
      - '--election-timeout=1250'
      - '--initial-cluster=etcd1=http://etcd1:2380,etcd2=http://etcd2:2380,etcd3=http://etcd3:2380'
      - '--initial-cluster-state=new'
    ports:
      - 2381:2379
    volumes:
      - ./store/etcd3/data:/etcd_data

• It can be seen from the above script that the ports 2379, 2380 and 2381 of the host are used to map the ports 2379 of the three etcd containers;
• Execute the command docker compose up - D, as shown below. It can be seen that the three containers have been created with names of 28_etcd1_1,28_etcd2_1,28_etcd3_1. The container name is related to the current directory name:

zhaoqin@zhaoqindeMacBook-Pro-2 28 % docker-compose up -d
Creating network "28_default" with the default driver
Creating 28_etcd2_1 ... done
Creating 28_etcd3_1 ... done
Creating 28_etcd1_1 ... done

• The etcd cluster has been started successfully. Try whether the basic operation commands are normal. Execute the following command to create a new key value pair. The key is / aaa/foo and the value is 111:

docker exec 28_etcd1_1 /usr/local/bin/etcdctl put /aaa/foo 111

• The view command is as follows (I changed a container here):

docker exec 28_etcd2_1 /usr/local/bin/etcdctl get /aaa/foo -w fields

• Complete results are obtained. In addition to the key values, there are Revision, ModRevision, Version, Lease and other fields:

"ClusterID" : 10316109323310759371
"MemberID" : 15168875803774599630
"Revision" : 2
"RaftTerm" : 2
"Key" : "/aaa/foo"
"CreateRevision" : 2
"ModRevision" : 2
"Version" : 1
"Value" : "111"
"Lease" : 0
"More" : false
"Count" : 1

Create a new gradle project as the parent project of the whole actual combat series

• Next, build a new gradle project, and the whole actual combat series is developed under this parent project;
• The new gradle project is called jetcd tutorials, and its build.gradle content is as follows:

import java.time.OffsetDateTime
import java.time.format.DateTimeFormatter

// Relevant settings used by gradle itself
buildscript {
    // Warehouse
    repositories {
        // local
        mavenLocal()
        // If there is a private server, configure it here. If not, please comment it out
        maven {
            url 'http://192.168.50.43:8081/repository/aliyun-proxy/'
        }
        // Alibaba cloud
        maven {
            url 'http://maven.aliyun.com/nexus/content/groups/public/'
        }
        // Central warehouse
        mavenCentral()
        // Grand plugin
        maven {
            url 'https://plugins.gradle.org/m2/'
        }
    }

    // Variables used by sub modules
    ext {
        springBootVersion = '2.4.4'
    }
}

// plug-in unit
plugins {
    id 'java'
    id 'java-library'
    // With this declaration, the sub module can use the org.springframework.boot plug-in without specifying the version, but apply=false means that the current module does not use this plug-in
    id 'org.springframework.boot' version "${springBootVersion}" apply false
    id 'io.spring.dependency-management' version '1.0.11.RELEASE'
}

// gradle wrapper specifies the version
wrapper {
    gradleVersion = '6.8.3'
}

// Take the current time
def buildTimeAndDate = OffsetDateTime.now()

// Generate string variables based on time
ext {
    projectVersion = project.version
    buildDate = DateTimeFormatter.ISO_LOCAL_DATE.format(buildTimeAndDate)
    buildTime = DateTimeFormatter.ofPattern('HH:mm:ss.SSSZ').format(buildTimeAndDate)
}


// Configuration for all projects, including root projects
allprojects {
    group 'com.bolingcavalry'
    version '1.0-SNAPSHOT'

    apply plugin: 'java'
    apply plugin: 'idea'
    apply plugin: 'io.spring.dependency-management'

    // Compile related parameters
    compileJava {
        sourceCompatibility = JavaVersion.VERSION_1_8
        targetCompatibility = JavaVersion.VERSION_1_8
        options.encoding = 'UTF-8'
        options.compilerArgs =  [
                '-Xlint:all', '-Xlint:-processing'
        ]
    }

    // Copy LICENSE
    tasks.withType(Jar) {
        from(project.rootDir) {
            include 'LICENSE'
            into 'META-INF'
        }
    }

    // When generating the jar file, the contents of MANIFEST.MF are as follows
    jar {
        manifest {
            attributes(
                    'Created-By': "${System.properties['java.version']} (${System.properties['java.vendor']} ${System.properties['java.vm.version']})".toString(),
                    'Built-By': 'travis',
                    'Build-Date': buildDate,
                    'Build-Time': buildTime,
                    'Built-OS': "${System.properties['os.name']}",
                    'Specification-Title': project.name,
                    'Specification-Version': project.version,
                    'Specification-Vendor': 'Will Zhao',
                    'Implementation-Title': project.name,
                    'Implementation-Version': project.version,
                    'Implementation-Vendor': 'Will Zhao'
            )
        }
    }

    // Warehouse
    repositories {
        // local
        mavenLocal()
        // If there is a private server, configure it here. If not, please comment it out
        maven {
            url 'http://192.168.50.43:8081/repository/aliyun-proxy/'
        }
        // Alibaba cloud
        maven {
            url 'http://maven.aliyun.com/nexus/content/groups/public/'
        }
        // Central warehouse
        mavenCentral()
        // Grand plugin
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
}

// Similar to maven's dependency management, the versions of all jar s are specified here, and the sub modules do not need to specify the version when relying
allprojects { project ->
    buildscript {
        dependencyManagement {
            imports {
                mavenBom "org.springframework.boot:spring-boot-starter-parent:${springBootVersion}"
                mavenBom "org.junit:junit-bom:5.7.0"
            }

            dependencies {
                dependency 'org.projectlombok:lombok:1.16.16'
                dependency 'org.apache.commons:commons-lang3:3.11'
                dependency 'commons-collections:commons-collections:3.2.2'
                dependency 'io.etcd:jetcd-core:0.5.0'
                dependency 'org.slf4j:slf4j-log4j12:1.7.30'
            }
        }
    }
}

// Coordinate information
group 'com.bolingcavalry'
version '1.0-SNAPSHOT'

• Now that the root module has been built, the following entire series of codes will be written under the root module;

Write helloworld application

• Next, write a hello world application to verify whether jetcd can operate etcd cluster;
• Create a new gradle sub module named helloworld under the root module, and its build.gradle content is as follows:

plugins {
    id 'java'
}

// Sub module's own dependency
dependencies {
    implementation 'io.etcd:jetcd-core'
    implementation 'org.projectlombok:lombok'
    // The annotation processor will not be passed. Modules that use lombok generated code need to declare the annotation processor themselves
    annotationProcessor 'org.projectlombok:lombok'
    implementation 'org.slf4j:slf4j-log4j12'
    testImplementation('org.junit.jupiter:junit-jupiter')
}

test {
    useJUnitPlatform()
}

group 'com.bolingcavalry'
version '1.0-SNAPSHOT'

1. HelloWorld.java is added. The code is very simple. The getKVClient method is used to generate the client instance. The put method writes the key value pair to etcd, and the get method queries the value of the specified key to etcd:

package com.bolingcavalry;

import io.etcd.jetcd.ByteSequence;
import io.etcd.jetcd.Client;
import io.etcd.jetcd.KV;
import io.etcd.jetcd.kv.GetResponse;
import io.etcd.jetcd.kv.PutResponse;
import lombok.extern.slf4j.Slf4j;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import static com.google.common.base.Charsets.UTF_8;

@Slf4j
public class HelloWorld {

    /**
     * Create a new key value client instance
     * @return
     */
    private KV getKVClient(){
        String endpoints = "http://192.168.50.239:2379,http://192.168.50.239:2380,http://192.168.50.239:2381";
        Client client = Client.builder().endpoints(endpoints.split(",")).build();
        return client.getKVClient();
    }

    /**
     * Convert the string to the ByteSequence instance required by the client
     * @param val
     * @return
     */
    private static ByteSequence bytesOf(String val) {
        return ByteSequence.from(val, UTF_8);
    }

    /**
     * Query the value corresponding to the specified key
     * @param key
     * @return
     * @throws ExecutionException
     * @throws InterruptedException
     */
    public String get(String key) throws ExecutionException, InterruptedException{
        log.info("start get, key [{}]", key);
        GetResponse response = getKVClient().get(bytesOf(key)).get();

        if (response.getKvs().isEmpty()) {
            log.error("empty value of key [{}]", key);
            return null;
        }

        String value = response.getKvs().get(0).getValue().toString(UTF_8);
        log.info("finish get, key [{}], value [{}]", key, value);
        return value;
    }

    /**
     * Create key value pairs
     * @param key
     * @param value
     * @return
     * @throws ExecutionException
     * @throws InterruptedException
     */
    public PutResponse put(String key, String value) throws ExecutionException, InterruptedException {
        log.info("start put, key [{}], value [{}]", key, value);
        return getKVClient().put(bytesOf(key), bytesOf(value)).get();
    }
}

1. Next, let's write a unit test class to verify whether the above code is valid:

package com.bolingcavalry;

import io.etcd.jetcd.kv.PutResponse;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import java.util.concurrent.ExecutionException;
import static org.junit.jupiter.api.Assertions.*;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class HelloWorldTest {
    // Key with test
    private static final String KEY = "/abc/foo-" + System.currentTimeMillis();

    // Values for testing
    private static final String VALUE = "/abc/foo";

    @org.junit.jupiter.api.Test
    @Order(2)
    void get() throws ExecutionException, InterruptedException {
        String getResult = new HelloWorld().get(KEY);
        assertEquals(VALUE, getResult);
    }

    @Test
    @Order(1)
    void put() throws ExecutionException, InterruptedException {
        PutResponse putResponse = new HelloWorld().put(KEY, VALUE);
        assertNotNull(putResponse);
        assertNotNull(putResponse.getHeader());
    }
}

1. After the code is written, execute the unit test according to the red box in the figure below. It can be seen that the jetcd operation etcd is successful:

• So far, the beginning of the jetcd actual combat series has been completed. We have built an etcd cluster and preliminarily experienced the basic functions of jetcd. The next chapter will start with the basic operation and learn jetcd from simple to deep;