Kubeadm is an official tool provided by Kubernetes to quickly install the Kubernetes cluster. With each release of Kubernetes updated synchronously, Kubeadm will adjust some of the cluster configuration practices. By experimenting with kubeadm, you can learn some new Kubernetes official best practices on cluster configuration.
operating system
Ubuntu 16.04+, Debian 9, CentOS 7, RHEL 7, Fedora 25/26 (best-effort), others
Memory 2GB +, 2-core CPU +
Communication between cluster nodes
Unique host name, MAC address and product_uuid for each node
Check MAC address: use ip link or ifconfig-a
Check product_uuid:cat/sys/class/dmi/id/product_uuid
Prohibit swap partitioning.This will enable kubelet to function properly
Get ready
1.1 System Configuration
Host name corresponds to IP:
[root@k8s-master ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.1.201 k8s-master 192.168.1.202 k8s-node1 192.168.1.203 k8s-node2
If each host has a firewall enabled and needs to open the ports required by each component of Kubernetes, you can see the section "Check required ports" in Installing kubeadm.For simplicity, disable firewalls at each node:
systemctl stop firewalld systemctl disable firewalld
Disable SELINUX:
sed -i 's/enforcing/disabled/' /etc/selinux/config <br/>setenforce 0
or
vi /etc/selinux/config SELINUX=disabled
Turn off swap:
swapoff -a # temporary vim /etc/fstab # permanent
Synchronization time:
yum install ntpdate -y ntpdate ntp.api.bz
Create a/etc/sysctl.d/k8s.conf file and add the following:
net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1
Execute the command to make the changes effective:
modprobe br_netfilter sysctl -p /etc/sysctl.d/k8s.conf
1.2 kube-proxy opens preconditions for ipvs
Since ipvs have been added to the backbone of the kernel, the following kernel modules need to be loaded before opening ipvs for kube-proxy, and the following scripts will be executed on all Kubernetes nodes node1 and node2:
cat > /etc/sysconfig/modules/ipvs.modules <<EOF #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
The above script creates the / etc/sysconfig/modules/ipvs.modules file, which ensures that the required modules are automatically loaded after the node restarts.Use the lsmod | grep-e ip_vs-e nf_conntrack_ipv4 command to see if the required kernel module has been loaded correctly.
Next, you need to make sure that the ipset package is installed on each node
yum install ipset
To facilitate viewing the proxy rules for ipvs, it is best to install the management tool ipvsadm
yum install ipvsadm
If the above preconditions are not met, even if the configuration of kube-proxy turns on ipvs mode, it will fall back to iptables mode.
1.3 Install Docker
Kubernetes has used CRI(Container Runtime Interface) container runtime interfaces since 1.6.The default container runtime is still Docker, using the built-in dockershis CRI implementation in kubelet.
Install docker's yum source:
yum install -y yum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
View the latest version of Docker:
yum list docker-ce.x86_64 --showduplicates |sort -r [root@go-docker ~]# yum list docker-ce.x86_64 --showduplicates |sort -r * updates: mirrors.aliyun.com Loading mirror speeds from cached hostfile Loaded plugins: fastestmirror, langpacks * extras: mirrors.aliyun.com docker-ce.x86_64 3:19.03.5-3.el7 docker-ce-stable docker-ce.x86_64 3:19.03.4-3.el7 docker-ce-stable docker-ce.x86_64 3:19.03.3-3.el7 docker-ce-stable docker-ce.x86_64 3:19.03.2-3.el7 docker-ce-stable docker-ce.x86_64 3:19.03.1-3.el7 docker-ce-stable docker-ce.x86_64 3:19.03.0-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.9-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.8-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.7-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.6-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.5-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.4-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.3-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.2-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.1-3.el7 docker-ce-stable docker-ce.x86_64 3:18.09.0-3.el7 docker-ce-stable docker-ce.x86_64 18.06.3.ce-3.el7 docker-ce-stable docker-ce.x86_64 18.06.2.ce-3.el7 docker-ce-stable docker-ce.x86_64 18.06.1.ce-3.el7 docker-ce-stable docker-ce.x86_64 18.06.0.ce-3.el7 docker-ce-stable docker-ce.x86_64 18.03.1.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 18.03.0.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.12.1.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.12.0.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.09.1.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.09.0.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.06.2.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.06.1.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.06.0.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.03.3.ce-1.el7 docker-ce-stable docker-ce.x86_64 17.03.2.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.03.1.ce-1.el7.centos docker-ce-stable docker-ce.x86_64 17.03.0.ce-1.el7.centos docker-ce-stable * base: mirrors.aliyun.com Available Packages
The list of docker versions currently supported by Kubernetes 1.16 is 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09.Version 18.09.7 of docker is installed here on each node.
yum makecache fast yum install -y --setopt=obsoletes=0 \ docker-ce-18.09.7-3.el7 systemctl start docker systemctl enable docker
Confirm that the default policy (pllicy) for the FOWARD chain in the iptables filter table is ACCEPT.
iptables -nvL
[root@k8s-master ~]# iptables -nvL
Chain INPUT (policy ACCEPT 20 packets, 2866 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DOCKER-ISOLATION-STAGE-1 all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 19 packets, 2789 bytes)
pkts bytes target prot opt in out source destination
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-ISOLATION-STAGE-2 all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * docker0 0.0.0.0/0 0.0.0.0/0
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
0 0 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 1.4 Modify docker cgroup driver to systemd
According to the content in the document CRI installation, for Linux distribution using systemd as init system, using systemd as the cgroup driver for docker can ensure that server nodes are more stable in resource-constrained situations, so modify the cgroup driver for docker on each node to be systemd here.
Create or modify/etc/docker/daemon.json:
vim /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"]
}Restart docker:
systemctl restart docker docker info | grep Cgroup Cgroup Driver: systemd