Deployment of Kubernetes cluster ETCD

Keywords: SSL JSON Linux network

kubernetes cluster deployment

Three official deployment methods
1.minikube
minikube is a tool that can run a single point of kubernetes quickly locally, only for testing K8S or daily development
Deployment address: https://kubernetes.io/docs/setup/minkube/
2.kubeadm
Kubeadm is also a tool that provides kubeadm init and kubeadm join for rapid deployment of kubernetes clusters
Deployment address: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
3. Binary package
Download the binary package of the distribution from the official, manually deploy each component to form a kubernetes cluster
Address: https://github.com/kubernetes/kubernetes/releases
https://github.com/kubernetes/kubernetes/releases?after=v1.13.1
Here we use binary group installation to build kubernetes
kubernetes deployment ideas

  1. Self signed ssl certificate
  2. etcd database cluster deployment
  3. node install docker
  4. flannel container cluster network deployment
  5. Deploy master components
  6. Deploy node components
  7. Deploy a test example
  8. Deploy web ui (dashboard)
  9. Deploy dns resolution service (coredns) within cluster
    kubernetes single node topology

    1, master, node1, node2 environment deployment
    master: turn off network manager, clear iptabels, turn off core protection, and edit host name
    [root@localhost ~]# hostnamectl set-hostname master
    [root@localhost ~]# su
    [root@master ~]# systemctl stop NetworkManager
    [root@master ~]# systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [root@master ~]# setenforce 0
    [root@master ~]# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
    [root@master1 ~]# iptables -F

    node1:

    [root@localhost ~]# hostnamectl set-hostname master1
    [root@localhost ~]# su
    [root@master1 ~]# systemctl stop NetworkManager
    [root@master1 ~]# systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [root@master1 ~]# setenforce 0
    [root@master1 ~]# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
    [root@node01 ~]# iptables -F

    node2:

    [root@localhost ~]# hostnamectl set-hostname node02
    [root@localhost ~]# su
    [root@node02 ~]# systemctl stop NetworkManager
    [root@node02 ~]# systemctl disable NetworkManager
    Removed symlink/etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink/etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [root@node02 ~]# setenforce 0
    [root@node02 ~]# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
    [root@node02 ~]# iptables -F

    Self signed SSL certificate
    Components and certificates used
    etcd: ca.pem,server.pem,server-key.pem
    flannel: ca.pem,server.pem,server-key.pem
    kube-apiserver: ca.pem,server.pem,server-key.pem
    kubelet: ca.pem,server.pem
    kube-proxy: ca.pem,kube-proxy.pem,kube-proxy-key.pem
    kubectl: ca.pem,admin.pem,admin-key.pem
    1. To generate CA certificate first, the communication between components must have CA certificate

    [root@master1 ~]# mkdir k8s
    [root@master1 ~]# cd k8s
    [root@master1 k8s]# pwd
    /root/k8s
    [root@master1 k8s]# mkdir /abc
    [root@master1 k8s]# mount.cifs //192.168.0.88/linuxs /abc
    Password for root@//192.168.0.88/linuxs:  
    [root@master1 k8s]# cp /abc/k8s/etcd* .
    [root@master1 k8s]# ll
    total 8
    -rwxr-xr-x. 1 root root 1088 Apr 29 00:13 etcd-cert.sh
    -rwxr-xr-x. 1 root root 1764 Apr 29 00:13 etcd.sh

    2.etcd-cert.sh is used to create CA certificate about etcd

    [root@master1 k8s]# cat etcd-cert.sh 
    #ca-config.json is the configuration file of ca certificate
    cat > ca-config.json <<EOF
    {
    "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h", #expiry 10 years
         "usages": [
            "signing",
            "key encipherment", #Use key validation
            "server auth",
            "client auth"
        ]
      }
    }
    }
    }
    EOF
    #ca-csr.json is the signature file of ca certificate
    cat > ca-csr.json <<EOF
    {
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
    }
    EOF
    cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
    #-----------------------
    #server-csr.json is the communication verification between three nodes of specified etcd
    cat > server-csr.json <<EOF
    {
    "CN": "etcd",
    "hosts": [
    "192.168.191.130",
    "192.168.191.131",
    "192.168.191.132"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
    }
    EOF
    cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

    3.etcd.sh is used to create startup scripts and configuration files

    [root@master1 k8s]# cat etcd.sh 
    #!/bin/bash
    # example: ./etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    ETCD_NAME=$1
    ETCD_IP=$2
    ETCD_CLUSTER=$3
    WORK_DIR=/k8s/etcd
    cat <<EOF >$WORK_DIR/cfg/etcd
    #[Member]
    ETCD_NAME="${ETCD_NAME}"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"
    EOF
    cat <<EOF >/usr/lib/systemd/system/etcd.service
    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=notify
    EnvironmentFile=${WORK_DIR}/cfg/etcd
    ExecStart=${WORK_DIR}/bin/etcd \
    --name=\${ETCD_NAME} \
    --data-dir=\${ETCD_DATA_DIR} \
    --listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
    --listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
    --advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
    --initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
    --initial-cluster=\${ETCD_INITIAL_CLUSTER} \
    --initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
    --initial-cluster-state=new \
    --cert-file=${WORK_DIR}/ssl/server.pem \
    --key-file=${WORK_DIR}/ssl/server-key.pem \
    --peer-cert-file=${WORK_DIR}/ssl/server.pem \
    --peer-key-file=${WORK_DIR}/ssl/server-key.pem \
    --trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
    --peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target
    EOF
    systemctl daemon-reload
    systemctl enable etcd
    systemctl restart etcd

    4. Download cfssl official script package
    Note:
    1.cfssl certificate generation tool
    2.cfssljson generates certificate by passing in json file
    3. Cfssl certinfo view certificate information

    [root@master1 k8s]# vim cfssl.sh
    curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
    curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
    curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
    chmod +x /usr/local/bin/cfssl*
    [root@master1 k8s]# bash cfssl.sh 
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100  9.8M  100  9.8M    0     0   106k      0  0:01:35  0:01:35 --:--:-- 98678
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100 2224k  100 2224k    0     0   316k      0  0:00:07  0:00:07 --:--:--  455k
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100 6440k  100 6440k    0     0   531k      0  0:00:12  0:00:12 --:--:--  736k

    5. Verification

    [root@master1 k8s]# ll /usr/local/bin/*
    -rwxr-xr-x. 1 root root 10376657 Apr 29 00:20 /usr/local/bin/cfssl
    -rwxr-xr-x. 1 root root  6595195 Apr 29 00:21 /usr/local/bin/cfssl-certinfo
    -rwxr-xr-x. 1 root root  2277873 Apr 29 00:20 /usr/local/bin/cfssljson
    [root@master1 k8s]# rm -rf cfssl.sh 

    6. Create temporary directory of etcd component certificate and define ca certificate configuration

    [root@master1 k8s]# mkdir etcd-cert
    [root@master1 k8s]# mv etcd-cert.sh etcd-cert
    [root@master1 k8s]# cd etcd-cert/
    [root@master1 etcd-cert]# ls
    etcd-cert.sh
    [root@master1 etcd-cert]# cat > ca-config.json <<EOF
    {
    "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
    }
    }
    EOF
    [root@master1 etcd-cert]# ls
    ca-config.json  etcd-cert.sh

    7.ca certificate signature

    [root@master1 etcd-cert]# cat > ca-csr.json <<EOF
    {
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
    }
    EOF
    [root@master1 etcd-cert]# ls
    ca-config.json  ca-csr.json  etcd-cert.sh

    8. Generate certificate, and generate two certificate files: ca-key.pem ca.pem

    [root@master1 etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
    2020/04/29 00:44:16 [INFO] generating a new CA key and certificate from CSR
    2020/04/29 00:44:16 [INFO] generate received request
    2020/04/29 00:44:16 [INFO] received CSR
    2020/04/29 00:44:16 [INFO] generating key: rsa-2048
    2020/04/29 00:44:16 [INFO] encoded CSR
    2020/04/29 00:44:16 [INFO] signed certificate with serial number 527285287467326079906972398205016440554485642971
    [root@master1 etcd-cert]# ls
    ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  etcd-cert.sh

    9. Specify the communication verification between three nodes of etcd

    [root@master1 etcd-cert]#  cat > server-csr.json <<EOF
    {
    "CN": "etcd",
    "hosts": [
    "192.168.191.130",
    "192.168.191.131",
    "192.168.191.132"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
    }
    EOF
    [root@master1 etcd-cert]# ls
    ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  etcd-cert.sh  server-csr.json

    10. Generate server certificate and key of etcd

    [root@master1 etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
    2020/04/29 00:48:07 [INFO] generate received request
    2020/04/29 00:48:07 [INFO] received CSR
    2020/04/29 00:48:07 [INFO] generating key: rsa-2048
    2020/04/29 00:48:07 [INFO] encoded CSR
    2020/04/29 00:48:07 [INFO] signed certificate with serial number 79028110669307243971733075611743333137367463128
    2020/04/29 00:48:07 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
    websites. For more information see the Baseline Requirements for the Issuance and Management
    of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
    specifically, section 10.2.3 ("Information Requirements").
    [root@master1 etcd-cert]# ls
    ca-config.json  ca-csr.json  ca.pem        server.csr       server-key.pem
    ca.csr          ca-key.pem   etcd-cert.sh  server-csr.json  server.pem

    Three materials:
    -config.json certificate configuration parameters
    -csr.json certificate signature file parameters
    . csr certificate signing file
    Generate results:
    -key.pem key
    . pem certificate
    2, Etcd database cluster deployment
    Binary package download address: https://github.com/etcd-io/etcd/releases
    To view the cluster status:

    /k8s/etcd/bin/etcdctl 
    –ca-file=/k8s/etcd/ssl/ca.pem 
    –cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem 
    –endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" 

    1. Pull the installation package down to the master node

    [root@master1 etcd-cert]# cp /abc/k8s/etcd-v3.3.10-linux-amd64.tar.gz /root/k8s/
    [root@master1 etcd-cert]# cd ..
    [root@master1 k8s]# pwd
    /root/k8s
    [root@master1 k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64.tar.gz
    [root@master1 k8s]# tar xf etcd-v3.3.10-linux-amd64.tar.gz 
    [root@master1 k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [root@master1 k8s]# cd etcd-v3.3.10-linux-amd64/
    [root@master1 etcd-v3.3.10-linux-amd64]# ls
    Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md

    2. Create the working directory of etcd. The following are the directory of configuration file cfg, command bin and certificate ssl

    [root@master1 etcd-v3.3.10-linux-amd64]# mkdir /k8s/etcd/{cfg,bin,ssl} -p
    [root@master1 etcd-v3.3.10-linux-amd64]# cd /k8s
    [root@master1 k8s]# tree .
    .
    └── etcd
    ├── bin
    ├── cfg
    └── ssl

    3. Copy the certificate file and command file to

    [root@master1 k8s]# mv /root/k8s/etcd-v3.3.10-linux-amd64/etcd* /k8s/etcd/bin/
    [root@master1 k8s]# cp /root/k8s/etcd-cert/*.pem /k8s/etcd/ssl/
    [root@master1 k8s]# cd/etcd/bin/etcd/etcdctl/cfg/ssl
    [root@master1 ssl]#ls
    ca-key.pem     ca.pem
    server-key.pem    server.pem

    4. Edit the configuration file and startup script of etcd

    #!/bin/bash
    # example: ./etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    ETCD_NAME=$1
    ETCD_IP=$2
    ETCD_CLUSTER=$3
    WORK_DIR=/k8s/etcd
    cat <<EOF >$WORK_DIR/cfg/etcd
    #[Member]
    ETCD_NAME="${ETCD_NAME}"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"
    EOF
    cat <<EOF >/usr/lib/systemd/system/etcd.service
    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=notify
    EnvironmentFile=${WORK_DIR}/cfg/etcd
    ExecStart=${WORK_DIR}/bin/etcd \
    --name=\${ETCD_NAME} \
    --data-dir=\${ETCD_DATA_DIR} \
    --listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} \
    --listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
    --advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} \
    --initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
    --initial-cluster=\${ETCD_INITIAL_CLUSTER} \
    --initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} \
    --initial-cluster-state=new \
    --cert-file=${WORK_DIR}/ssl/server.pem \
    --key-file=${WORK_DIR}/ssl/server-key.pem \
    --peer-cert-file=${WORK_DIR}/ssl/server.pem \
    --peer-key-file=${WORK_DIR}/ssl/server-key.pem \
    --trusted-ca-file=${WORK_DIR}/ssl/ca.pem \
    --peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target
    EOF
    systemctl daemon-reload
    systemctl enable etcd
    systemctl restart etcd
    [root@master1 k8s]# sh etcd.sh  
    Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
    Job for etcd.service failed because the control process exited with error code. See "systemctl status etcd.service" and "journalctl -xe" for details.
    [root@master1 etcd]# cd /k8s
    [root@master1 k8s]# ll /usr/lib/systemd/system/ | grep etcd
    -rw-r--r--. 1 root root  923 Apr 29 08:11 etcd.service

    5. The first step is to generate the configuration file and start the script generation under systemd
    Port 2379 is provided to the external port, 2380 is the internal cluster communication port, and the maximum port is 65536

    [root@master1 k8s]# cd /root/k8s/
    [root@master1 k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [root@master1 k8s]# pwd
    /root/k8s
    [root@master1 k8s]# bash etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380

    At this time, enter a waiting state, find other etcd cluster nodes, and exit by default after less than 5 minutes
    Let's go to two node s to deploy etcd
    6. Check whether etcd single node is on

    [root@master1 k8s]# netstat -natp | grep etcd
    tcp        0      0 192.168.247.149:2379    0.0.0.0:*               LISTEN      26604/etcd          
    tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      26604/etcd          
    tcp        0      0 192.168.247.149:2380    0.0.0.0:*               LISTEN      26604/etcd    

    7. Copy the certificate to the other two nodes

    [root@master1 k8s]# scp -r /k8s root@192.168.191.131:/k8s
    root@192.168.191.131's password: 
    etcd                                                                                  100%  523   252.4KB/s   00:00    
    etcd                                                                                  100%   18MB  95.0MB/s   00:00    
    etcdctl                                                                               100%   15MB 124.7MB/s   00:00    
    ca-key.pem                                                                            100% 1679   725.6KB/s   00:00    
    ca.pem                                                                                100% 1265   287.0KB/s   00:00    
    server-key.pem                                                                        100% 1675   813.0KB/s   00:00    
    server.pem

    8. Copy the startup script to the other two nodes

    [root@master1 k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.191.131:/usr/lib/systemd/system/
    root@192.168.191.131's password: 
    etcd.service                                                                          100%  923   814.8KB/s   00:00    
    [root@master1 k8s]# scp /usr/lib/systemd/system/etcd.service root@192.168.191.132:/usr/lib/systemd/system/
    root@192.168.191.132's password: 
    etcd.service

    9. The relevant files have been copied. Some parameters in the configuration file / k8s/etcd/cfg/etcd need to be modified
    node1:

    [root@node01 k8s]# vim /k8s/etcd/cfg/etcd
    #[Member]
    ETCD_NAME="etcd02"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://192.168.191.131:2380"
    ETCD_LISTEN_CLIENT_URLS="https://192.168.191.131:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.191.131:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://192.168.191.131:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://192.168.191.130:2380,etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"

    node2:

    [root@node02 k8s]# vim /k8s/etcd/cfg/etcd
    #[Member]
    ETCD_NAME="etcd03"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://192.168.191.132:2380"
    ETCD_LISTEN_CLIENT_URLS="https://192.168.191.132:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.191.132:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://192.168.191.132:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://192.168.191.130:2380,etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"

    10. Enable etcd service

    [root@node02 ~]# systemctl start etcd
    [root@node02 ~]# systemctl status etcd
    ● etcd.service - Etcd Server
    Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2020-04-29 08:42:15 CST; 9s ago
    [root@node01 ~]# systemctl start etcd
    [root@node01 ~]# systemctl status etcd
    ● etcd.service - Etcd Server
    Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2020-04-29 08:42:15 CST; 9s ago

    11. At this time, execute the script command again on the master node master1

    [root@master1 k8s]# cd /root/k8s/
    [root@master1 k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [root@master1 k8s]# bash etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    [root@master1 k8s]# 

    Check cluster status

    /k8s/etcd/bin/etcdctl \
    --ca-file=/k8s/etcd/ssl/ca.pem \
    --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem \
    --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
    cluster-health
    [root@master1 k8s]# cd /k8s/etcd/ssl/
    [root@master1 ssl]# pwd
    /k8s/etcd/ssl
    [root@master1 ssl]# ls
    ca-key.pem  ca.pem  server-key.pem  server.pem
    [root@master1 ssl]# /k8s/etcd/bin/etcdctl \
    > --ca-file=ca.pem \
    > --cert-file=server.pem --key-file=server-key.pem \
    > --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" \
    > cluster-health
    member 8f4e6ce663f0d49a is healthy: got healthy result from https://192.168.191.131:2379
    member b6230d9c6f20feeb is healthy: got healthy result from https://192.168.191.132:2379
    member d618618928dffeba is healthy: got healthy result from https://192.168.191.130:2379
    cluster is healthy

    At this point, the etcd services of our master and two node s are deployed
    Both nodes need to deploy the docker environment. Before deploying etcd, the two nodes have already deployed the docker environment.

Posted by project18726 on Wed, 29 Apr 2020 08:36:10 -0700