10. Kubernetes Practice Papers
10.1) Three ways to install Kubernetes (officially available)
10.1.1)minikube
Minikube is a tool that allows you to quickly run a single Kubernetes locally and try it out with Kubernetes or users of everyday development.It cannot be used in a production environment.
Official documents:
10.1.2)kubeadm
Kubeadm helps you quickly deploy a set of kubernetes clusters.The purpose of the kubeadm design is to provide a simple way for new users to start trying out kubernetes.It's currently in Beta.
Official documents:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
https://kubernetes.io/docs/setup/independent/install-kubeadm/
10.1.3) Binary Package
Download the distribution's binary packages from the official and manually deploy each component to form a Kubernetes cluster.This approach is currently used primarily in enterprise production environments.
Download address:
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#v1113
10.2) Kubernetes Installation Planning
10.2.1) Basic Resources
operating system
Ubuntu 16.04+
Debian 9
CentOS 7
RHEL 7
Fedora 25/26 (best-effort)
Other requirements:
Memory 2GB +, 2 Core CPU + (Production Planning)
Communication between cluster nodes
Unique host name, MAC address and product_uuid for each node
Check MAC address: use ip link or ifconfig-a
Check product_uuid:cat/sys/class/dmi/id/product_uuid
Prohibit swap partitioning so that kubelet works properly
10.2.2) Node Planning
192.168.111.134 node7 --node1 192.168.111.135 node8 --node2 192.168.111.136 node9 --master
10.3) Kubernetes Prepare Environment
10.3.1) Close the firewall
systemctl stop firewalld systemctl disable firewalld
10.3.2) Turn off selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config setenforce 0
10.3.3) Turn off swap
swapoff -a # temporary swapoff -a && sysctl -w vm.swappiness=0 vim /etc/fstab # permanent sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab
Change swap limits
cat /etc/sysconfig/kubelet KUBELET_EXTRA_ARGS=--fail-swap-on=false
10.3.4) hosts configuration
192.168.111.134 node7 192.168.111.135 node8 192.168.111.136 node9
10.3.5) Time Synchronization
ntpdate 1.cn.pool.ntp.org yum install ntpdate –y
#Configure ntp
client
Server: 
10.3.6) Add ssh mutual trust
ssh-keygen -t rsa for i in node7 node8 node9;do ssh-copy-id -i .ssh/id_rsa.pub $i;done yum install -y bridge-utils.x86_64
10.4) Kubernetes Cluster Installation (kubeadm)
10.4.1) System Resource Parameters
* hardnofile 65536 * softnofile 65536 * hardnproc 65536 * softnproc 65536
Edit the configuration file/etc/sysctl.conf to add the following:
net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_tw_recycle = 0 net.ipv4.ip_local_port_range = 10240 65535 net.ipv4.tcp_fin_timeout = 30 net.ipv4.tcp_max_syn_backlog = 8192 net.ipv4.tcp_max_tw_buckets = 36000 net.ipv4.tcp_keepalive_time = 1200 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.somaxconn = 16384
Execute sysctp-p after save exits
10.4.2) docker installation
2.6) Docker Installation Management
10.4.3) Installation of kubeadm-related tools
kubeadm: command to boot the cluster
kubelet: An agent that runs tasks in a cluster
kubectl: command line management tool
Add Aliyun YUM Software Source
#cat << EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF
Or:
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF yum install -y kubelet kubeadm kubectl kubernetes-cni yum install -y kubelet-1.13.5-0.x86_64 kubeadm-1.13.5-0.x86_64 kubectl-1.13.5-0.x86_64 kubernetes-cni systemctl enable kubelet && systemctl start kubelet
Note: When using Docker, kubeadm automatically checks the CGroup driver for kubelet and/var/lib/kubelet/kubeadm-flags.env sets it in the file at run time.If other CRI s are used, the cgroup-driver value must be modified to cgroupfs in/etc/default/kubelet:
cat /var/lib/kubelet/kubeadm-flags.env KUBELET_KUBEADM_ARGS=--cgroup-driver=cgroupfs --cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni systemctl daemon-reload systemctl restart kubelet
Common commands for kubeadm
helpHelp about any command initRun this command in order to set up the Kubernetes control plane. # Execute on master, initialize all master components joinRun this on any machine you wish to join an existing cluster # Execute on node, join master reset Run this to revert any changes made to this host by 'kubeadm init' or 'kubeadm join'. # Clean up init, join's environment token Manage bootstrap tokens. # Addition or deletion of token upgrade Upgrade your cluster smoothly to a newer version with this command. # Update Cluster version Print the version of kubeadm
10.4.4) Download a mirror of kubernetes
K8S_VERSION=v1.13.5
ETCD_VERSION=3.2.24
DASHBOARD_VERSION=v1.8.3
FLANNEL_VERSION=v0.10.0-amd64
DNS_VERSION=1.2.6
PAUSE_VERSION=3.1
Basic Components
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION
Network Components
docker pull quay.io/coreos/flannel:$FLANNEL_VERSION
Modify tag
docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver-amd64:$K8S_VERSION k8s.gcr.io/kube-apiserver:$K8S_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager-amd64:$K8S_VERSION k8s.gcr.io/kube-controller-manager:$K8S_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler-amd64:$K8S_VERSION k8s.gcr.io/kube-scheduler:$K8S_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy-amd64:$K8S_VERSION k8s.gcr.io/kube-proxy:$K8S_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/etcd-amd64:$ETCD_VERSION k8s.gcr.io/etcd:$ETCD_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/pause:$PAUSE_VERSION k8s.gcr.io/pause:$PAUSE_VERSION docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:$DNS_VERSION k8s.gcr.io/coredns:$DNS_VERSION 10.4.5)Function kubeadm init install master
Configure Domestic Mirror Acceleration
cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://registry.docker-cn.com" ]
}kubeadm config
kubeadm config upload from-file uploaded from configuration file to cluster to generate ConfigMap kubeadm config upload from-flags Generate ConfigMap from configuration parameters kubeadm config view to view configuration values in the current cluster kubeadm config print init-defaults output init-defaults default parameter file content kubeadm config print join-defaults output join-defaults default parameter file content Configuration conversion between old and new versions of kubeadm config migrate kubeadm config images list lists the required mirrors kubeadm config images pull to mirror locally
New init-config.yaml file custom mirror warehouse address and OD address segment
cat init-config.yaml
apiVersion: kubeadm.k8s.io/v1beta1 kind: ClusterConfiguration imageRepository: docker.io/dustise kubernetesVersion: v1.14.0 networking: podSubnet: "172.16.0.0/16 "
Download the required image
kubeadm config images pull --config=init-config.yaml [config/images] Pulled docker.io/dustise/kube-apiserver:v1.14.0 [config/images] Pulled docker.io/dustise/kube-controller-manager:v1.14.0 [config/images] Pulled docker.io/dustise/kube-scheduler:v1.14.0 [config/images] Pulled docker.io/dustise/kube-proxy:v1.14.0 [config/images] Pulled docker.io/dustise/pause:3.1 [config/images] Pulled docker.io/dustise/etcd:3.3.10 [config/images] Pulled docker.io/dustise/coredns:1.3.1
View default parameter file
kubeadm config print init-defaults
[WARNING IsDocker SystemdCheck] appears due to the inconsistency between docker's Group Driver and kubelet's Group Driver. Here you choose to modify the docker to be consistent with kubelet
docker info | grep Cgroup
Cgroup Driver: cgroupfs
Edit file/usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd --exec-opt native.cgroupdriver=systemd systemctl daemon-reload systemctl restart docker
docker info | grep Cgroup
Cgroup Driver: system
operation
Modify Configuration
sed -e 's/KUBELET_CGROUP_ARGS=--cgroup-driver=systemd/KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs/' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf DOCKER_CGROUPS=$(docker info | grep 'Cgroup' | cut -d' ' -f3) echo $DOCKER_CGROUPS echo 1 >/proc/sys/net/bridge/bridge-nf-call-iptables kubeadm init --kubernetes-version=1.13.5 --pod-network-cidr=172.16.0.0/16 --apiserver-advertise-address=192.168.111.136
kebeadm init requires additional parameters
Detailed parameter descriptions can be seen:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/
10.4.5) Run kubeadm init to install master
Configure Domestic Mirror Acceleration
cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://registry.docker-cn.com" ]
}kubeadm config
kubeadm config upload from-file uploaded from configuration file to cluster to generate ConfigMap kubeadm config upload from-flags Generate ConfigMap from configuration parameters kubeadm config view to view configuration values in the current cluster kubeadm config print init-defaults output init-defaults default parameter file content kubeadm config print join-defaults output join-defaults default parameter file content Configuration conversion between old and new versions of kubeadm config migrate kubeadm config images list lists the required mirrors kubeadm config images pull to mirror locally
New init-config.yaml file custom mirror warehouse address and OD address segment
#cat init-config.yaml apiVersion: kubeadm.k8s.io/v1beta1 kind: ClusterConfiguration imageRepository: docker.io/dustise kubernetesVersion: v1.14.0 networking: podSubnet: "172.16.0.0/16 "
Download the required image
kubeadm config images pull --config=init-config.yaml [config/images] Pulled docker.io/dustise/kube-apiserver:v1.14.0 [config/images] Pulled docker.io/dustise/kube-controller-manager:v1.14.0 [config/images] Pulled docker.io/dustise/kube-scheduler:v1.14.0 [config/images] Pulled docker.io/dustise/kube-proxy:v1.14.0 [config/images] Pulled docker.io/dustise/pause:3.1 [config/images] Pulled docker.io/dustise/etcd:3.3.10 [config/images] Pulled docker.io/dustise/coredns:1.3.1
#
DOCKER_CGROUPS=$(docker info | grep 'Cgroup' | cut -d' ' -f3)
echo $DOCKER_CGROUPS
echo 1 >/proc/sys/net/bridge/bridge-nf-call-iptables
kubeadm init --kubernetes-version=1.13.5 --pod-network-cidr=172.16.0.0/16 --apiserver-advertise-address=192.168.111.136
kebeadm init needs to be accompanied by detailed parameter descriptions as follows:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/
Use kubeadm reset to reset host state and reinitialize
10.4.6) General users access clusters using kubectl
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
10.4.7) Install the pod network plugin Flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml
Install the network plugin weave
kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')
10.4.8) View all pod s and nodes
10.4.9) Add Work Node
Switch to root account at Node:
kubeadm join 192.168.111.136:6443 --token gf25fd.xntkm8qy5klmhrv6 --discovery-token-ca-cert-hash sha256:f409b76900e0bf4e334f1bc2b629a89f4e031744489c6bfe8d8233f9af7ecdd7 #Format: kubeadm join --token <token> <master-ip>: <master-port> --discovery-token-ca-cert-hash sha256:<hash>
10.4.10) Installation Configuration Access dashboardb
Install dashboard
https://github.com/kubernetes/dashboard wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.0/src/deploy/recommended/kubernetes-dashboard.yaml wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.0/src/deploy/recommended/kubernetes-dashboard.yaml
Modify Dashboard Service to NodePort type
kind: Service apiVersion: v1 metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kube-system spec: type: NodePort ports: - port: 443 targetPort: 8443 nodePort: 30001 selector: k8s-app: kubernetes-dashboard
Deploy
kubectl create -f kubernetes-dashboard.yaml kubectl delete -f kubernetes-dashboard.yaml
Inspection
kubectl get svc --all-namespaces
Create Administrator
cat k8s-admin.yaml apiVersion: v1 kind: ServiceAccount metadata: labels: k8s-app: kubernetes-dashboard name: admin namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin namespace: kube-system
Log in using token
https://192.168.111.136:30001/#!/login
