Kubernetes
1. Introduction to kubernetes
1.1 evolution of application deployment mode
There are three main periods in the way of deploying applications:
-
Traditional deployment: in the early days of the Internet, applications will be deployed directly on physical machines
Advantages: simple, without the participation of other technologies
Disadvantages: resource usage boundaries cannot be defined for applications, it is difficult to allocate computing resources reasonably, and it is easy to have an impact between programs
-
Virtualization deployment: multiple virtual machines can be run on one physical machine, and each virtual machine is an independent environment
Advantages: the program environment will not affect each other, providing a certain degree of security
Disadvantages: it increases the operating system and wastes some resources
-
Container deployment: similar to virtualization, but sharing the operating system
advantage:
It can ensure that each container has its own file system, CPU, memory, process space, etc
The resources needed to run the application are wrapped in containers and decoupled from the underlying infrastructure
Containerized applications can be deployed across cloud service providers and Linux operating system distributions
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-hs70j2fu-163354912825) (kubenetes. Assets / image-20200505183738289. PNG)]
Container deployment brings a lot of convenience to, but there are also some problems, such as:
- If a container fails and stops, how can another container be started immediately to replace the stopped container
- How to expand the number of containers horizontally when the amount of concurrent access becomes larger
These container management problems are collectively referred to as container scheduling problems. In order to solve these container scheduling problems, some container scheduling software has been produced:
- Swarm: Docker's own container choreography tool
- Mesos: Apache's tool for unified resource management and control needs to be used in combination with Marathon
- Kubernetes: Google's open source container choreography tool
[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-xg7det8p-163354912828) (kubenetes. Assets / image-2020052415039551. PNG)]
1.2 introduction to kubernetes
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-1b96cnwp-163354912829) (kubenetes. Assets / image-20200406232838722. PNG)]
kubernetes, a new leading distributed architecture solution based on container technology, is an open source version of Borg system, a secret weapon strictly kept secret by Google for more than ten years. The first version was released in September 2014 and the first official version was released in July 2015.
kubernetes is essentially a group of server clusters. It can run specific programs on each node of the cluster to manage the containers in the node. The purpose is to realize the automation of resource management. It mainly provides the following main functions:
- Self repair: once a container crashes, it can quickly start a new container in about 1 second
- Elastic scaling: you can automatically adjust the number of running containers in the cluster as needed
- Service discovery: a service can find its dependent services in the form of automatic discovery
- Load balancing: if a service starts multiple containers, it can automatically realize the requested load balancing
- Version fallback: if you find a problem with the newly released program version, you can immediately fallback to the original version
- Storage Orchestration: storage volumes can be automatically created according to the needs of the container itself
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-8iigwdye-163354912835) (kubenetes. Assets / image-20200526203726071-1626780706899. PNG)]
1.3 kubernetes components
A kubernetes cluster is mainly composed of a master node and * * work node * *. Different components are installed on each node.
master: the control plane of the cluster, which is responsible for the decision-making (Management) of the cluster
ApiServer: the only entry for resource operation. It receives commands entered by users and provides mechanisms such as authentication, authorization, API registration and discovery
Scheduler: it is responsible for cluster resource scheduling and scheduling the Pod to the corresponding node node according to the scheduled scheduling policy
ControllerManager: responsible for maintaining the status of the cluster, such as program deployment arrangement, fault detection, automatic expansion, rolling update, etc
Etcd: it is responsible for storing the information of various resource objects in the cluster
node: the data plane of the cluster, which is responsible for providing the running environment (work) for the container
Kubelet: responsible for maintaining the life cycle of containers, that is, creating, updating and destroying containers by controlling docker s
KubeProxy: responsible for providing service discovery and load balancing within the cluster
Docker: responsible for various operations of containers on nodes
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-73dwszss-163354912836) (kubenetes. Assets / image-20200406184656917. PNG)]
Next, deploy an nginx service to illustrate the calling relationship of various components of kubernetes system:
-
First of all, it should be clear that once the kubernetes environment is started, both the master and node will store their own information in the etcd database
-
The installation request of an nginx service will first be sent to the API server component of the master node
-
The apiServer component will call the scheduler component to decide which node this service should be installed on
At this time, it will read the information of each node from etcd, then select according to a certain algorithm, and inform apiServer of the results
-
apiServer calls controller manager to schedule the Node to install nginx service
-
After kubelet receives the instruction, it will notify docker, and then docker will start an nginx pod
Pod is the smallest operation unit of kubernetes. The container must run in pod,
-
An nginx service is running. If you need to access nginx, you need to use Kube proxy to generate access proxy for pod
In this way, external users can access nginx services in the cluster
1.4 kubernetes concept
Master: cluster control node. Each cluster requires at least one master node to control the cluster
Node: workload node. The master allocates containers to these node work nodes, and the docker on the node node is responsible for the operation of the containers
Pod: the smallest control unit of kubernetes. All containers run in the pod. There can be one or more containers in a pod
Controller: controller, which is used to manage pods, such as starting, stopping, and scaling the number of pods
Service: the unified entrance of external service of pod. Below, you can maintain multiple pods of the same type
Label: label, used to classify pod s. Pods of the same type will have the same label
NameSpace: NameSpace used to isolate the running environment of the pod
2. Build kubernetes cluster environment
2.1 pre knowledge points
At present, there are two main ways to deploy Kubernetes cluster in production:
kubeadm
Kubedm is a K8s deployment tool that provides kubedm init and kubedm join for rapid deployment of Kubernetes clusters.
Official address: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
Binary package
Download the binary package of the distribution from github and manually deploy each component to form a Kubernetes cluster.
Kubedm lowered the deployment threshold, but shielded many details, making it difficult to troubleshoot problems. If you want to be more controllable, it is recommended to use binary packages to deploy Kubernetes clusters. Although manual deployment is troublesome, you can learn a lot of working principles during this period, which is also conducive to later maintenance.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-54wr1twj-163354912836) (kubenetes. Assets / image-20200404094800622. PNG)]
2.2 introduction to kubedm deployment mode
kubeadm is a tool launched by the official community for rapid deployment of kubernetes clusters. This tool can complete the deployment of a kubernetes cluster through two instructions:
- Create a Master node kubedm init
- Join the Node to the current cluster $kubedm join < IP and port of the master Node >
2.3 installation requirements
Before you start, you need to meet the following conditions to deploy Kubernetes cluster machines:
- One or more machines, operating system CentOS7.x-86_x64
- Hardware configuration: 2GB or more RAM, 2 CPUs or more CPUs, hard disk 30GB or more
- Network interworking between all machines in the cluster
- You can access the Internet. You need to pull the image
- Disable swap partition
2.4 final objective
- Install Docker and kubedm on all nodes
- Deploy Kubernetes Master
- Deploy container network plug-in
- Deploy the Kubernetes Node and add the node to the Kubernetes cluster
- Deploy the Dashboard Web page to visually view Kubernetes resources
2.5 preparation environment
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-lqb7kjsv-163354912837) (kubenetes. Assets / image-20210609000002940. PNG)]
| role | IP address | assembly |
|---|---|---|
| k8s-master01 | 192.168.5.3 | docker，kubectl，kubeadm，kubelet |
| k8s-node01 | 192.168.5.4 | docker，kubectl，kubeadm，kubelet |
| k8s-node02 | 192.168.5.5 | docker，kubectl，kubeadm，kubelet |
2.6 system initialization
2.6.1 set the mutual resolution of system Host name and Host file
hostnamectl set-hostname k8s-master01 && bash hostnamectl set-hostname k8s-node01 && bash hostnamectl set-hostname k8s-node02 && bash
cat <<EOF>> /etc/hosts 192.168.5.3 k8s-master01 192.168.5.4 k8s-node01 192.168.5.5 k8s-node02 EOF
scp /etc/hosts root@192.168.5.4:/etc/hosts scp /etc/hosts root@192.168.5.5:/etc/hosts
2.6.2 install dependent files (all nodes must operate)
yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git
2.6.3 set the firewall to Iptables and set empty rules (all nodes must operate)
systemctl stop firewalld && systemctl disable firewalld yum -y install iptables-services && systemctl start iptables && systemctl enable iptables && iptables -F && service iptables save
2.6.4 close SELINUX (all nodes must operate)
swapoff -a && sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab setenforce 0 && sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
2.6.5 adjust kernel parameters for K8S (all nodes must operate)
modprobe br_netfilter cat <<EOF> kubernetes.conf net.bridge.bridge-nf-call-iptables=1 net.bridge.bridge-nf-call-ip6tables=1 net.ipv4.ip_forward=1 net.ipv4.tcp_tw_recycle=0 vm.swappiness=0 # The use of swap space is prohibited and is allowed only when the system is OOM vm.overcommit_memory=1 # Do not check whether the physical memory is sufficient vm.panic_on_oom=0 # Open OOM fs.inotify.max_user_instances=8192 fs.inotify.max_user_watches=1048576 fs.file-max=52706963 fs.nr_open=52706963 net.ipv6.conf.all.disable_ipv6=1 net.netfilter.nf_conntrack_max=2310720 EOF cp kubernetes.conf /etc/sysctl.d/kubernetes.conf sysctl -p /etc/sysctl.d/kubernetes.conf
2.6.6 adjust the system time zone (all nodes must operate)
# Set the system time zone to China / Shanghai timedatectl set-timezone Asia/Shanghai # Writes the current UTC time to the hardware clock timedatectl set-local-rtc 0 # Restart system time dependent services systemctl restart rsyslog systemctl restart crond
2.6.7 set rsyslogd and SYSTEMd Journal (all nodes must be operated)
# Directory where logs are persisted mkdir /var/log/journal mkdir /etc/systemd/journald.conf.d cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF [Journal] # Persistent save to disk Storage=persistent # Compress history log Compress=yes SyncIntervalSec=5m RateLimitInterval=30s RateLimitBurst=1000 # Maximum occupied space 10G SystemMaxUse=10G # The maximum size of a single log file is 200M SystemMaxFileSize=200M # Log retention time: 2 weeks MaxRetentionSec=2week # Do not forward logs to syslog ForwardToSyslog=no EOF systemctl restart systemd-journald
2.6.8 preconditions for Kube proxy to enable ipvs (all nodes must operate)
cat <<EOF> /etc/sysconfig/modules/ipvs.modules #!/bin/bash modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOF chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4
2.6.9 install Docker software (all nodes must operate)
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce
## Create / etc/docker directory
mkdir /etc/docker
cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
}
}
EOF
mkdir -p /etc/systemd/system/docker.service.d
# Restart docker service
systemctl daemon-reload && systemctl restart docker && systemctl enable docker
Upload the file to / etc/yum.repos.d/ instead of Yum config Manager -- add repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo command
docker-ce.repo
[docker-ce-stable] name=Docker CE Stable - $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-stable-debuginfo] name=Docker CE Stable - Debuginfo $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/stable enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-stable-source] name=Docker CE Stable - Sources baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/stable enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-test] name=Docker CE Test - $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/test enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-test-debuginfo] name=Docker CE Test - Debuginfo $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/test enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-test-source] name=Docker CE Test - Sources baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/test enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-nightly] name=Docker CE Nightly - $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/nightly enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-nightly-debuginfo] name=Docker CE Nightly - Debuginfo $basearch baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/nightly enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg [docker-ce-nightly-source] name=Docker CE Nightly - Sources baseurl=https://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/nightly enabled=0 gpgcheck=1 gpgkey=https://mirrors.aliyun.com/docker-ce/linux/centos/gpg
2.6.10 install kubedm (all nodes must be operated)
cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF yum install -y kubelet kubeadm kubectl && systemctl enable kubelet
2.7 deployment of Kubernetes Master
2.7.1 initialize master node (master node operation)
kubeadm init --apiserver-advertise-address=192.168.5.3 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.21.1 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
2.7.2 add the master node and other work nodes
kubeadm join 192.168.5.3:6443 --token h0uelc.l46qp29nxscke7f7 \
--discovery-token-ca-cert-hash sha256:abc807778e24bff73362ceeb783cc7f6feec96f20b4fd707c3f8e8312294e28f
2.7.3 deployment network
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Below is the file
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: psp.flannel.unprivileged
annotations:
seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
spec:
privileged: false
volumes:
- configMap
- secret
- emptyDir
- hostPath
allowedHostPaths:
- pathPrefix: "/etc/cni/net.d"
- pathPrefix: "/etc/kube-flannel"
- pathPrefix: "/run/flannel"
readOnlyRootFilesystem: false
# Users and groups
runAsUser:
rule: RunAsAny
supplementalGroups:
rule: RunAsAny
fsGroup:
rule: RunAsAny
# Privilege Escalation
allowPrivilegeEscalation: false
defaultAllowPrivilegeEscalation: false
# Capabilities
allowedCapabilities: ['NET_ADMIN', 'NET_RAW']
defaultAddCapabilities: []
requiredDropCapabilities: []
# Host namespaces
hostPID: false
hostIPC: false
hostNetwork: true
hostPorts:
- min: 0
max: 65535
# SELinux
seLinux:
# SELinux is unused in CaaSP
rule: 'RunAsAny'
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
rules:
- apiGroups: ['extensions']
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames: ['psp.flannel.unprivileged']
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: flannel
namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kube-flannel-cfg
namespace: kube-system
labels:
tier: node
app: flannel
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kube-flannel-ds
namespace: kube-system
labels:
tier: node
app: flannel
spec:
selector:
matchLabels:
app: flannel
template:
metadata:
labels:
tier: node
app: flannel
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
hostNetwork: true
priorityClassName: system-node-critical
tolerations:
- operator: Exists
effect: NoSchedule
serviceAccountName: flannel
initContainers:
- name: install-cni
image: quay.io/coreos/flannel:v0.14.0
command:
- cp
args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
volumeMounts:
- name: cni
mountPath: /etc/cni/net.d
- name: flannel-cfg
mountPath: /etc/kube-flannel/
containers:
- name: kube-flannel
image: quay.io/coreos/flannel:v0.14.0
command:
- /opt/bin/flanneld
args:
- --ip-masq
- --kube-subnet-mgr
resources:
requests:
cpu: "100m"
memory: "50Mi"
limits:
cpu: "100m"
memory: "50Mi"
securityContext:
privileged: false
capabilities:
add: ["NET_ADMIN", "NET_RAW"]
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumeMounts:
- name: run
mountPath: /run/flannel
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
- name: run
hostPath:
path: /run/flannel
- name: cni
hostPath:
path: /etc/cni/net.d
- name: flannel-cfg
configMap:
name: kube-flannel-cfg
2.8 testing kubernetes clusters
2.8.1 deploy nginx test
kubectl create deployment nginx --image=nginx kubectl expose deployment nginx --port=80 --type=NodePort kubectl get pod,svc
3. Resource management
3.1 introduction to resource management
In kubernetes, all contents are abstracted as resources. Users need to manage kubernetes by operating resources.
Kubernetes is essentially a cluster system. Users can deploy various services in the cluster. The so-called deployment service actually means running containers in the kubernetes cluster and running the specified programs in the containers.
The smallest management unit of kubernetes is a pod rather than a container, so the container can only be placed in the pod. Generally, kubernetes does not directly manage the pod, but manages the pod through the pod controller.
After the Pod can provide services, you should consider how to access the services in the Pod. kubernetes provides Service resources to realize this function.
Of course, kubernetes also provides various storage systems if the data of programs in Pod needs to be persistent.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-828ssgim-163354912837) (kubenetes. Assets / image-20200406225334627. PNG)]
The core of learning kubernetes is to learn how to operate Pod, Pod controller, Service, storage and other resources on the cluster
3.2 introduction to yaml language
YAML is a markup language similar to XML and JSON. It emphasizes data centric rather than focusing on identification languages. Therefore, the definition of YAML itself is relatively simple, known as "a humanized data format language".
<heima>
<age>15</age>
<address>Beijing</address>
</heima>
heima: age: 15 address: Beijing
The syntax of YAML is relatively simple, mainly including the following:
- Case sensitive
- Use indentation to represent hierarchical relationships
- tab is not allowed for indentation, only spaces are allowed (lower version limit)
- The number of indented spaces is not important, as long as the elements of the same level are aligned to the left
- '#' indicates a comment
YAML supports the following data types:
- Scalar: a single, non separable value
- Object: a collection of key value pairs, also known as mapping / hash / dictionary
- Array: a set of values arranged in order, also known as sequence / list
# Scalar refers to a simple value, such as string, Boolean, integer, floating point number, Null, time and date
# 1 boolean type
c1: true (perhaps True)
# 2 integer
c2: 234
# 3 floating point
c3: 3.14
# 4 null type
c4: ~ # Use ~ for null
# 5 date type
c5: 2018-02-17 # The date must be in ISO 8601 format, i.e. yyyy mm DD
# 6 time type
c6: 2018-02-17T15:02:31+08:00 # The time is in ISO 8601 format, the T connection is used between the time and date, and the last + is used to represent the time zone
# 7 string type
c7: heima # It is simple to write the value directly. If there are special characters in the middle of the string, it must be wrapped in double quotation marks or single quotation marks
c8: line1
line2 # If there are too many strings, they can be split into multiple lines, and each line will be converted into a space
# object
# Form I (recommended):
heima:
age: 15
address: Beijing
# Form 2 (understanding):
heima: {age: 15,address: Beijing}
# array # Form I (recommended): address: - Shunyi - Changping # Form 2 (understanding): address: [Shunyi,Changping]
Tip:
When writing yaml, remember to add a space after it
2 if you need to put multi segment yaml configuration in one file, use - -- to separate them
3 the following is a yaml to json website, which can be used to verify whether yaml is written correctly
https://www.json2yaml.com/convert-yaml-to-json
3.3 resource management mode
-
Imperative object management: directly use commands to operate kubernetes resources
kubectl run nginx-pod --image=nginx:1.17.1 --port=80
-
Imperative object configuration: operate kubernetes resources through command configuration and configuration files
kubectl create/patch -f nginx-pod.yaml
-
Declarative object configuration: operate kubernetes resources through the apply command and configuration file
kubectl apply -f nginx-pod.yaml
| type | Operation object | Applicable environment | advantage | shortcoming |
|---|---|---|---|---|
| Imperative object management | object | test | simple | Only active objects can be operated, and cannot be audited or tracked |
| Imperative object configuration | file | development | Can audit and track | When the project is large, there are many configuration files and the operation is troublesome |
| Declarative object configuration | catalogue | development | Support directory operation | Difficult to debug under unexpected circumstances |
3.3.1 imperative object management
kubectl command
Kubectl is the command line tool of kubernetes cluster. It can manage the cluster itself and install and deploy container applications on the cluster. The syntax of the kubectl command is as follows:
kubectl [command] [type] [name] [flags]
comand: Specifies the operations to be performed on the resource, such as create, get, and delete
Type: Specifies the resource type, such as deployment, pod, and service
Name: Specifies the name of the resource. The name is case sensitive
flags: specify additional optional parameters
# View all pod s kubectl get pod # View a pod kubectl get pod pod_name # View a pod and display the results in yaml format kubectl get pod pod_name -o yaml
Resource type
All contents in kubernetes are abstracted as resources, which can be viewed through the following command:
kubectl api-resources
Frequently used resources include the following:
| Resource classification | Resource name | abbreviation | Resource role |
|---|---|---|---|
| Cluster level resources | nodes | no | Cluster components |
| namespaces | ns | Isolate Pod | |
| pod resources | pods | po | Loading container |
| pod resource controller | replicationcontrollers | rc | Control pod resources |
| replicasets | rs | Control pod resources | |
| deployments | deploy | Control pod resources | |
| daemonsets | ds | Control pod resources | |
| jobs | Control pod resources | ||
| cronjobs | cj | Control pod resources | |
| horizontalpodautoscalers | hpa | Control pod resources | |
| statefulsets | sts | Control pod resources | |
| Service discovery resources | services | svc | Unified pod external interface |
| ingress | ing | Unified pod external interface | |
| Storage resources | volumeattachments | storage | |
| persistentvolumes | pv | storage | |
| persistentvolumeclaims | pvc | storage | |
| Configure resources | configmaps | cm | to configure |
| secrets | to configure |
operation
kubernetes allows multiple operations on resources. You can view detailed operation commands through – help
kubectl --help
Frequently used operations include the following:
| Command classification | command | translate | Command function |
|---|---|---|---|
| Basic command | create | establish | Create a resource |
| edit | edit | Edit a resource | |
| get | obtain | Get a resource | |
| patch | to update | Update a resource | |
| delete | delete | Delete a resource | |
| explain | explain | Show resource documents | |
| Operation and commissioning | run | function | Run a specified mirror in the cluster |
| expose | expose | Exposed resource is Service | |
| describe | describe | Display resource internal information | |
| logs | Log output container logs in pod | Output container log in pod | |
| attach | Winding into a container in operation | Enter the container in operation | |
| exec | Execute a command in the container | Execute a command in the container | |
| cp | copy | Copy files inside and outside the Pod | |
| rollout | First show | Manage publishing of resources | |
| scale | scale | Number of expansion (contraction) Pod | |
| autoscale | Automatic adjustment | Automatically adjust the number of pods | |
| Advanced command | apply | rc | Configure resources through files |
| label | label | Update labels on resources | |
| Other commands | cluster-info | Cluster information | Display cluster information |
| version | edition | Displays the current Server and Client versions |
The following is a simple demonstration of how to use the following commands by creating and deleting a namespace / pod:
# Create a namespace [root@master ~]# kubectl create namespace dev namespace/dev created # Get namespace [root@master ~]# kubectl get ns NAME STATUS AGE default Active 21h dev Active 21s kube-node-lease Active 21h kube-public Active 21h kube-system Active 21h # Create and run an nginx Pod in this namespace [root@master ~]# kubectl run pod --image=nginx:latest -n dev kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead. deployment.apps/pod created # View the newly created pod [root@master ~]# kubectl get pod -n dev NAME READY STATUS RESTARTS AGE pod 1/1 Running 0 21s # Delete the specified pod [root@master ~]# kubectl delete pod pod-864f9875b9-pcw7x pod "pod" deleted # Deletes the specified namespace [root@master ~]# kubectl delete ns dev namespace "dev" deleted
3.3.2 command object configuration
Imperative object configuration is to use commands to operate kubernetes resources together with configuration files.
1) Create a nginxpod.yaml as follows:
apiVersion: v1
kind: Namespace
metadata:
name: dev
---
apiVersion: v1
kind: Pod
metadata:
name: nginxpod
namespace: dev
spec:
containers:
- name: nginx-containers
image: nginx:latest
2) Execute the create command to create resources:
[root@master ~]# kubectl create -f nginxpod.yaml namespace/dev created pod/nginxpod created
At this time, it is found that two resource objects are created, namely namespace and pod
3) Execute the get command to view resources:
[root@master ~]# kubectl get -f nginxpod.yaml NAME STATUS AGE namespace/dev Active 18s NAME READY STATUS RESTARTS AGE pod/nginxpod 1/1 Running 0 17s
The information of the two resource objects is displayed
4) Execute the delete command to delete resources:
[root@master ~]# kubectl delete -f nginxpod.yaml namespace "dev" deleted pod "nginxpod" deleted
At this time, two resource objects are found to be deleted
summary:
To operate resources in the way of command object configuration, you can simply think of: command + yaml Configuration file (which contains various parameters required by the command)
3.3.3 declarative object configuration
Declarative object configuration is similar to imperative object configuration, but it has only one command, apply.
# First, execute the kubectl apply -f yaml file once and find that the resource is created [root@master ~]# kubectl apply -f nginxpod.yaml namespace/dev created pod/nginxpod created # Execute the kubectl apply -f yaml file again and find that the resource has not changed [root@master ~]# kubectl apply -f nginxpod.yaml namespace/dev unchanged pod/nginxpod unchanged
summary:
In fact, declarative object configuration is to use apply Describe the final state of a resource (in yaml (status defined in)
use apply Operational resources:
If the resource does not exist, it is created, which is equivalent to kubectl create
If the resource already exists, it is updated, which is equivalent to kubectl patch
Extension: can kubectl run on node nodes?
kubectl needs to be configured to run. Its configuration file is $HOME/.kube. If you want to run this command on the node node, you need to copy the. kube file on the master node to the node node, that is, perform the following operations on the master node:
scp -r HOME/.kube node1: HOME/
Usage recommendation: how to use the three methods?
Create / update resources configure kubectl apply -f XXX.yaml using declarative objects
Deleting resources configuring kubectl delete -f XXX.yaml using imperative objects
Query resources use the command object to manage kubectl get(describe) resource names
4. Introduction to actual combat
This chapter describes how to deploy and access an nginx service in a kubernetes cluster.
4.1 Namespace
Namespace is a very important resource in kubernetes system. Its main function is to realize resource isolation of multiple environments or multi tenant resources.
By default, all pods in the kubernetes cluster can access each other. However, in practice, you may not want two pods to access each other. At this time, you can divide the two pods into different namespaces. Kubernetes can form logical "groups" by allocating resources in the cluster to different namespaces, so as to facilitate the isolated use and management of resources in different groups.
Different namespace s can be managed by different tenants through the authorization mechanism of kubernetes, so as to realize multi tenant resource isolation. At this time, the resource quota mechanism of kubernetes can be combined to limit the resources that can be occupied by different tenants, such as CPU usage, memory usage, etc., so as to realize the management of available resources for tenants.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-hnayssue-163354912838) (kubenetes. Assets / image-20200407100850484. PNG)]
kubernetes will create several namespace s by default after the cluster is started
[root@master ~]# kubectl get namespace NAME STATUS AGE default Active 45h # All objects that do not specify a Namespace are assigned in the default Namespace kube-node-lease Active 45h # The heartbeat maintenance between cluster nodes is introduced in v1.13 kube-public Active 45h # Resources under this namespace can be accessed by everyone (including unauthenticated users) kube-system Active 45h # All resources created by the Kubernetes system are in this namespace
Let's look at the specific operations of the namespace resource:
see
# 1 view all ns commands: kubectl get ns [root@master ~]# kubectl get ns NAME STATUS AGE default Active 45h kube-node-lease Active 45h kube-public Active 45h kube-system Active 45h # 2 view the specified ns command: kubectl get ns name [root@master ~]# kubectl get ns default NAME STATUS AGE default Active 45h # 3 specify output format command: kubectl get ns name - o format parameter # kubernetes supports many formats, such as wide, json and yaml [root@master ~]# kubectl get ns default -o yaml apiVersion: v1 kind: Namespace metadata: creationTimestamp: "2021-05-08T04:44:16Z" name: default resourceVersion: "151" selfLink: /api/v1/namespaces/default uid: 7405f73a-e486-43d4-9db6-145f1409f090 spec: finalizers: - kubernetes status: phase: Active # 4. View ns details command: kubectl describe ns name [root@master ~]# kubectl describe ns default Name: default Labels: <none> Annotations: <none> Status: Active # Active namespace in use Terminating deleting namespace # ResourceQuota resource restrictions for namespace # LimitRange is the resource limit for each component in the namespace No resource quota. No LimitRange resource.
establish
# Create namespace [root@master ~]# kubectl create ns dev namespace/dev created
delete
# Delete namespace [root@master ~]# kubectl delete ns dev namespace "dev" deleted
collocation method
First, prepare a yaml file: ns-dev.yaml
apiVersion: v1 kind: Namespace metadata: name: dev
Then you can execute the corresponding create and delete commands:
Create: kubectl create - f ns dev.yaml
Delete: kubectl delete - f ns dev.yaml
4.2 Pod
Pod is the smallest unit for kubernetes cluster management. The program must be deployed in the container to run, and the container must exist in pod.
A Pod can be considered as a package of containers. One or more containers can exist in a Pod.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-2kcovyhk-163354912838) (kubenetes. Assets / image-20200407121501907. PNG)]
After kubernetes starts the cluster, all components in the cluster also run in Pod mode. You can view it with the following command:
[root@master ~]# kubectl get pod -n kube-system NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-6955765f44-68g6v 1/1 Running 0 2d1h kube-system coredns-6955765f44-cs5r8 1/1 Running 0 2d1h kube-system etcd-master 1/1 Running 0 2d1h kube-system kube-apiserver-master 1/1 Running 0 2d1h kube-system kube-controller-manager-master 1/1 Running 0 2d1h kube-system kube-flannel-ds-amd64-47r25 1/1 Running 0 2d1h kube-system kube-flannel-ds-amd64-ls5lh 1/1 Running 0 2d1h kube-system kube-proxy-685tk 1/1 Running 0 2d1h kube-system kube-proxy-87spt 1/1 Running 0 2d1h kube-system kube-scheduler-master 1/1 Running 0 2d1h
Create and run
kubernetes does not provide the command to run Pod separately, but is implemented through the Pod controller
# Command format: kubectl run (pod controller name) [parameter] # --Image specifies the image of the Pod # --Port specifies the port # --Namespace specifies the namespace [root@master ~]# kubectl run nginx --image=nginx:latest --port=80 --namespace dev deployment.apps/nginx created
View pod information
# View Pod basic information
[root@master ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 43s
# View Pod details
[root@master ~]# kubectl describe pod nginx -n dev
Name: nginx
Namespace: dev
Priority: 0
Node: node1/192.168.5.4
Start Time: Wed, 08 May 2021 09:29:24 +0800
Labels: pod-template-hash=5ff7956ff6
run=nginx
Annotations: <none>
Status: Running
IP: 10.244.1.23
IPs:
IP: 10.244.1.23
Controlled By: ReplicaSet/nginx
Containers:
nginx:
Container ID: docker://4c62b8c0648d2512380f4ffa5da2c99d16e05634979973449c98e9b829f6253c
Image: nginx:latest
Image ID: docker-pullable://nginx@sha256:485b610fefec7ff6c463ced9623314a04ed67e3945b9c08d7e53a47f6d108dc7
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Wed, 08 May 2021 09:30:01 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-hwvvw (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-hwvvw:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-hwvvw
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned dev/nginx-5ff7956ff6-fg2db to node1
Normal Pulling 4m11s kubelet, node1 Pulling image "nginx:latest"
Normal Pulled 3m36s kubelet, node1 Successfully pulled image "nginx:latest"
Normal Created 3m36s kubelet, node1 Created container nginx
Normal Started 3m36s kubelet, node1 Started container nginx
Access Pod
# Get podIP [root@master ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ... nginx 1/1 Running 0 190s 10.244.1.23 node1 ... #Access POD [root@master ~]# curl http://10.244.1.23:80 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> </head> <body> <p><em>Thank you for using nginx.</em></p> </body> </html>
Delete specified Pod
# Delete specified Pod [root@master ~]# kubectl delete pod nginx -n dev pod "nginx" deleted # At this point, it is displayed that the Pod was deleted successfully, but after querying again, it is found that a new one has been generated [root@master ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 21s # This is because the current Pod is created by the Pod controller. The controller will monitor the Pod status. Once the Pod is found dead, it will be rebuilt immediately # To delete the Pod, you must delete the Pod controller # Let's first query the Pod controller in the current namespace [root@master ~]# kubectl get deploy -n dev NAME READY UP-TO-DATE AVAILABLE AGE nginx 1/1 1 1 9m7s # Next, delete this PodPod controller [root@master ~]# kubectl delete deploy nginx -n dev deployment.apps "nginx" deleted # Wait a moment, then query the Pod and find that the Pod has been deleted [root@master ~]# kubectl get pods -n dev No resources found in dev namespace.
Configuration operation
Create a pod-nginx.yaml as follows:
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
Then you can execute the corresponding create and delete commands:
Create: kubectl create -f pod-nginx.yaml
Delete: kubectl delete -f pod-nginx.yaml
4.3 Label
Label is an important concept in kubernetes system. Its function is to add identification on resources to distinguish and select them.
Features of Label:
- A Label will be attached to various objects in the form of key/value pairs, such as Node, Pod, Service, etc
- A resource object can define any number of labels, and the same Label can also be added to any number of resource objects
- The Label is usually determined when the resource object is defined. Of course, it can also be dynamically added or deleted after the object is created
Multi dimensional grouping of resources can be realized through Label, so that resource allocation, scheduling, configuration, deployment and other management can be carried out flexibly and conveniently.
Some common Label examples are as follows:
- Version label: "version": "release", "version": "stable"
- Environment label: "environment": "dev", "environment": "test", "environment": "pro"
- Schema label: "tier": "frontend", "tier": "backend"
After the label is defined, the label selection should also be considered, which requires the use of the Label Selector, namely:
Label is used to define an identity for a resource object
Label Selector is used to query and filter resource objects with certain labels
There are currently two types of label selectors:
-
Equation based Label Selector
name = slave: select all objects with key="name" and value="slave" in the Label
env != production: select all objects including key="env" in Label and whose value is not equal to "production"
-
Collection based Label Selector
name in (master, slave): select all objects with key="name" and value="master" or "slave" in the Label
name not in (frontend): select all objects that contain key="name" in Label and whose value is not equal to "frontend"
Multiple label selectors can be used for label selection. In this case, multiple label selectors can be combined and separated by comma "," and ". For example:
name=slave，env!=production
name not in (frontend)，env!=production
Command mode
# Tag pod resources [root@master ~]# kubectl label pod nginx-pod version=1.0 -n dev pod/nginx-pod labeled # Update tags for pod resources [root@master ~]# kubectl label pod nginx-pod version=2.0 -n dev --overwrite pod/nginx-pod labeled # View label [root@master ~]# kubectl get pod nginx-pod -n dev --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx-pod 1/1 Running 0 10m version=2.0 # Filter Tags [root@master ~]# kubectl get pod -n dev -l version=2.0 --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx-pod 1/1 Running 0 17m version=2.0 [root@master ~]# kubectl get pod -n dev -l version!=2.0 --show-labels No resources found in dev namespace. #delete a tap [root@master ~]# kubectl label pod nginx-pod version- -n dev pod/nginx-pod labeled
collocation method
apiVersion: v1
kind: Pod
metadata:
name: nginx
namespace: dev
labels:
version: "3.0"
env: "test"
spec:
containers:
- image: nginx:latest
name: pod
ports:
- name: nginx-port
containerPort: 80
protocol: TCP
Then you can execute the corresponding update command: kubectl apply -f pod-nginx.yaml
4.4 Deployment
In kubernetes, pod is the smallest control unit, but kubernetes rarely directly controls pod, which is generally completed through pod controller. The pod controller is used for pod management to ensure that the pod resources meet the expected state. When the pod resources fail, it will try to restart or rebuild the pod.
There are many types of Pod controllers in kubernetes. This chapter only introduces one: Deployment.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-hiofrpj1-163354912839) (kubenetes. Assets / image-20200408193950807. PNG)]
Command operation
# Command format: kubectl create deployment name [parameter]
# --Image specifies the image of the pod
# --Port specifies the port
# --replicas specifies the number of created pod s
# --Namespace specifies the namespace
[root@master ~]# kubectl create deploy nginx --image=nginx:latest --port=80 --replicas=3 -n dev
deployment.apps/nginx created
# View the created Pod
[root@master ~]# kubectl get pods -n dev
NAME READY STATUS RESTARTS AGE
nginx-5ff7956ff6-6k8cb 1/1 Running 0 19s
nginx-5ff7956ff6-jxfjt 1/1 Running 0 19s
nginx-5ff7956ff6-v6jqw 1/1 Running 0 19s
# View deployment information
[root@master ~]# kubectl get deploy -n dev
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 3/3 3 3 2m42s
# UP-TO-DATE: number of copies successfully upgraded
# AVAILABLE: number of AVAILABLE copies
[root@master ~]# kubectl get deploy -n dev -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 3/3 3 3 2m51s nginx nginx:latest run=nginx
# View deployment details
[root@master ~]# kubectl describe deploy nginx -n dev
Name: nginx
Namespace: dev
CreationTimestamp: Wed, 08 May 2021 11:14:14 +0800
Labels: run=nginx
Annotations: deployment.kubernetes.io/revision: 1
Selector: run=nginx
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: run=nginx
Containers:
nginx:
Image: nginx:latest
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-5ff7956ff6 (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 5m43s deployment-controller Scaled up replicaset nginx-5ff7956ff6 to 3
# delete
[root@master ~]# kubectl delete deploy nginx -n dev
deployment.apps "nginx" deleted
Configuration operation
Create a deploy-nginx.yaml as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
run: nginx
template:
metadata:
labels:
run: nginx
spec:
containers:
- image: nginx:latest
name: nginx
ports:
- containerPort: 80
protocol: TCP
Then you can execute the corresponding create and delete commands:
Create: kubectl create -f deploy-nginx.yaml
Delete: kubectl delete -f deploy-nginx.yaml
4.5 Service
Through the previous lesson, we have been able to use Deployment to create a group of pods to provide services with high availability.
Although each Pod is assigned a separate Pod IP, there are two problems:
- Pod IP will change with the reconstruction of pod
- Pod IP is only a virtual IP visible in the cluster and cannot be accessed externally
This makes it difficult to access the Service. Therefore, kubernetes designed Service to solve this problem.
Service can be regarded as a group of external access interfaces of similar pods. With the help of service, applications can easily realize service discovery and load balancing.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-rrl8glvu-163354912839) (kubenetes. Assets / image-20200408194716912. PNG)]
Operation 1: create an accessible Service in the cluster
# Expose Service [root@master ~]# kubectl expose deploy nginx --name=svc-nginx1 --type=ClusterIP --port=80 --target-port=80 -n dev service/svc-nginx1 exposed # View service [root@master ~]# kubectl get svc svc-nginx1 -n dev -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc-nginx1 ClusterIP 10.109.179.231 <none> 80/TCP 3m51s run=nginx # A CLUSTER-IP is generated here, which is the IP of the service. This address will not change during the service life cycle # You can access the POD corresponding to the current service through this IP [root@master ~]# curl 10.109.179.231:80 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> </head> <body> <h1>Welcome to nginx!</h1> ....... </body> </html>
Operation 2: create a Service that can also be accessed outside the cluster
# The type of the Service created above is ClusterIP. This ip address is accessible only within the cluster # If you need to create a Service that can also be accessed externally, you need to modify the type to NodePort [root@master ~]# kubectl expose deploy nginx --name=svc-nginx2 --type=NodePort --port=80 --target-port=80 -n dev service/svc-nginx2 exposed # At this time, you will find that a Service of NodePort type appears, and there is a pair of ports (80:31928/TC) [root@master ~]# kubectl get svc svc-nginx2 -n dev -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR svc-nginx2 NodePort 10.100.94.0 <none> 80:31928/TCP 9s run=nginx # Next, you can access the node IP:31928 to access the service through the host outside the cluster # For example, access the following address through a browser on the host computer http://192.168.5.4:31928/
Delete Service
[root@master ~]# kubectl delete svc svc-nginx-1 -n dev service "svc-nginx-1" deleted
collocation method
Create an svc-nginx.yaml as follows:
apiVersion: v1
kind: Service
metadata:
name: svc-nginx
namespace: dev
spec:
clusterIP: 10.109.179.231 #Fixed svc intranet ip
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: nginx
type: ClusterIP
Then you can execute the corresponding create and delete commands:
Create: kubectl create -f svc-nginx.yaml
Delete: kubectl delete -f svc-nginx.yaml
Summary
So far, you have mastered the basic operations of Namespace, Pod, Deployment and Service resources. With these operations, you can realize the simple Deployment and access of a Service in kubernetes cluster. However, if you want to make better use of kubernetes, you need to deeply study the details and principles of these resources.
5. Detailed explanation of pod
5.1 Pod introduction
5.1.1 Pod structure
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-zi9wcouh-163354912839) (kubenetes. Assets / image-20200407121501907-1626781151898. PNG)]
Each Pod can contain one or more containers, which can be divided into two categories:
-
The number of containers where the user program is located can be more or less
-
Pause container, which is a root container for each Pod. It has two functions:
-
It can be used as a basis to evaluate the health status of the whole Pod
-
The Ip address can be set on the root container, and other containers can use this Ip (Pod IP) to realize the network communication within the Pod
Here is Pod Internal communications, Pod The communication between is realized by virtual two-layer network technology. Our current environment uses Flannel
-
5.1.2 definition of pod
The following is a list of Pod resources:
apiVersion: v1 #Required, version number, e.g. v1
kind: Pod #Required, resource type, such as Pod
metadata: #Required, metadata
name: string #Required, Pod name
namespace: string #The namespace to which the Pod belongs. The default is "default"
labels: #Custom label list
- name: string
spec: #Required, detailed definition of container in Pod
containers: #Required, container list in Pod
- name: string #Required, container name
image: string #Required, the image name of the container
imagePullPolicy: [ Always|Never|IfNotPresent ] #Get the policy of the mirror
command: [string] #The list of startup commands of the container. If not specified, the startup commands used during packaging shall be used
args: [string] #List of startup command parameters for the container
workingDir: string #Working directory of the container
volumeMounts: #Configuration of storage volumes mounted inside the container
- name: string #Refer to the name of the shared storage volume defined by the pod. The volume name defined in the volumes [] section is required
mountPath: string #The absolute path of the storage volume mount in the container should be less than 512 characters
readOnly: boolean #Is it read-only mode
ports: #List of port library numbers to be exposed
- name: string #Name of the port
containerPort: int #The port number that the container needs to listen on
hostPort: int #The port number that the host of the Container needs to listen to. It is the same as the Container by default
protocol: string #Port protocol, support TCP and UDP, default TCP
env: #List of environment variables to be set before the container runs
- name: string #Environment variable name
value: string #Value of environment variable
resources: #Setting of resource limits and requests
limits: #Resource limit settings
cpu: string #Cpu limit, in the number of core s, will be used for the docker run -- Cpu shares parameter
memory: string #Memory limit. The unit can be Mib/Gib. It will be used for the docker run --memory parameter
requests: #Settings for resource requests
cpu: string #Cpu request, initial available quantity of container startup
memory: string #Memory request, initial available number of container starts
lifecycle: #Lifecycle hook
postStart: #This hook is executed immediately after the container is started. If the execution fails, it will be restarted according to the restart policy
preStop: #Execute this hook before the container terminates, and the container will terminate regardless of the result
livenessProbe: #For the setting of health check of each container in the Pod, when there is no response for several times, the container will be restarted automatically
exec: #Set the inspection mode in the Pod container to exec mode
command: [string] #Command or script required for exec mode
httpGet: #Set the health check method of containers in Pod to HttpGet, and specify Path and port
path: string
port: number
host: string
scheme: string
HttpHeaders:
- name: string
value: string
tcpSocket: #Set the health check mode of containers in the Pod to tcpSocket mode
port: number
initialDelaySeconds: 0 #The time of the first detection after the container is started, in seconds
timeoutSeconds: 0 #Timeout for container health check probe waiting for response, unit: seconds, default: 1 second
periodSeconds: 0 #Set the periodic detection time for container monitoring and inspection, in seconds, once every 10 seconds by default
successThreshold: 0
failureThreshold: 0
securityContext:
privileged: false
restartPolicy: [Always | Never | OnFailure] #Restart strategy of Pod
nodeName: <string> #Setting NodeName means that the Pod is scheduled to the node node with the specified name
nodeSelector: obeject #Setting NodeSelector means scheduling the Pod to the node containing the label
imagePullSecrets: #The secret name used to Pull the image, specified in the format of key: secret key
- name: string
hostNetwork: false #Whether to use the host network mode. The default is false. If it is set to true, it means to use the host network
volumes: #Define a list of shared storage volumes on this pod
- name: string #Shared storage volume name (there are many types of volumes)
emptyDir: {} #The storage volume of type emtyDir is a temporary directory with the same life cycle as Pod. Null value
hostPath: string #The storage volume of type hostPath represents the directory of the host where the Pod is mounted
path: string #The directory of the host where the Pod is located will be used for the directory of mount in the same period
secret: #For the storage volume of type secret, mount the cluster and the defined secret object inside the container
scretname: string
items:
- key: string
path: string
configMap: #The storage volume of type configMap mounts predefined configMap objects inside the container
name: string
items:
- key: string
path: string
#Tip: # Here, you can view the configurable items of each resource through a command # kubectl explain resource type view the first level attributes that can be configured for a resource # kubectl explain resource type. View the sub attributes of the attribute [root@k8s-master01 ~]# kubectl explain pod KIND: Pod VERSION: v1 FIELDS: apiVersion <string> kind <string> metadata <Object> spec <Object> status <Object> [root@k8s-master01 ~]# kubectl explain pod.metadata KIND: Pod VERSION: v1 RESOURCE: metadata <Object> FIELDS: annotations <map[string]string> clusterName <string> creationTimestamp <string> deletionGracePeriodSeconds <integer> deletionTimestamp <string> finalizers <[]string> generateName <string> generation <integer> labels <map[string]string> managedFields <[]Object> name <string> namespace <string> ownerReferences <[]Object> resourceVersion <string> selfLink <string> uid <string>
In kubernetes, the primary attributes of almost all resources are the same, mainly including five parts:
- The apiVersion version is internally defined by kubernetes. The version number must be available through kubectl API versions
- kind type is internally defined by kubernetes. The version number must be available through kubectl API resources
- Metadata metadata, mainly resource identification and description, commonly used are name, namespace, labels, etc
- spec description, which is the most important part of configuration, contains a detailed description of various resource configurations
- Status status information, the contents of which do not need to be defined, is automatically generated by kubernetes
Among the above attributes, spec is the focus of next research. Continue to look at its common sub attributes:
- Containers < [] Object > container list, used to define container details
- NodeName schedules the pod to the specified Node according to the value of nodeName
- NodeSelector < map [] > select and schedule the Pod to the Node containing these label s according to the information defined in NodeSelector
- Whether the hostNetwork uses the host network mode. The default is false. If it is set to true, it indicates that the host network is used
- Volumes < [] Object > storage volume is used to define the storage information hung on the Pod
- restartPolicy is the restart policy, which indicates the processing policy of Pod in case of failure
5.2 Pod configuration
This section mainly studies the pod.spec.containers attribute, which is also the most critical configuration in pod configuration.
[root@k8s-master01 ~]# kubectl explain pod.spec.containers KIND: Pod VERSION: v1 RESOURCE: containers <[]Object> # Array, representing that there can be multiple containers FIELDS: name <string> # Container name image <string> # The mirror address required by the container imagePullPolicy <string> # Image pull strategy command <[]string> # The list of startup commands of the container. If not specified, the startup commands used during packaging shall be used args <[]string> # List of parameters required for the container's start command env <[]Object> # Configuration of container environment variables ports <[]Object> # List of port numbers that the container needs to expose resources <Object> # Setting of resource limits and resource requests
5.2.1 basic configuration
Create the pod-base.yaml file as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-base
namespace: dev
labels:
user: heima
spec:
containers:
- name: nginx
image: nginx:1.17.1
- name: busybox
image: busybox:1.30
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-kvmvkczf-163354912840) (kubenetes. Assets / image-20210617223823675-1626781695411. PNG)]
The above defines a relatively simple Pod configuration with two containers:
- Nginx: created with the nginx image of version 1.17.1, (nginx is a lightweight web container)
- Busybox: created with the busybox image of version 1.30, (busybox is a small collection of linux commands)
# Create Pod [root@k8s-master01 pod]# kubectl apply -f pod-base.yaml pod/pod-base created # View Pod status # READY 1/2: indicates that there are 2 containers in the current Pod, of which 1 is ready and 1 is not ready # RESTARTS: the number of RESTARTS. Because one container failed, Pod has been restarting trying to recover it [root@k8s-master01 pod]# kubectl get pod -n dev NAME READY STATUS RESTARTS AGE pod-base 1/2 Running 4 95s # You can view internal details through describe # At this point, a basic Pod has been running, although it has a problem for the time being [root@k8s-master01 pod]# kubectl describe pod pod-base -n dev
5.2.2 image pulling
Create the pod-imagepullpolicy.yaml file as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-imagepullpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
imagePullPolicy: Never # Used to set the image pull policy
- name: busybox
image: busybox:1.30
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-irv3slbu-163354912840) (kubenetes. Assets / image-20210617223923659. PNG)]
imagePullPolicy is used to set the image pull policy. kubernetes supports configuring three kinds of pull policies:
- Always: always pull images from remote warehouses (always download remotely)
- IfNotPresent: if there is a local image, the local image is used. If there is no local image, the image is pulled from the remote warehouse (if there is a local image, there is no remote download)
- Never: only use the local image, never pull from the remote warehouse, and report an error if there is no local image (always use the local image)
Default value Description:
If the image tag is a specific version number, the default policy is IfNotPresent
If the image tag is: latest (final version), the default policy is always
# Create Pod [root@k8s-master01 pod]# kubectl create -f pod-imagepullpolicy.yaml pod/pod-imagepullpolicy created # View Pod details # At this time, it is obvious that the nginx image has a process of Pulling image "nginx:1.17.1" [root@k8s-master01 pod]# kubectl describe pod pod-imagepullpolicy -n dev ...... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned dev/pod-imagePullPolicy to node1 Normal Pulling 32s kubelet, node1 Pulling image "nginx:1.17.1" Normal Pulled 26s kubelet, node1 Successfully pulled image "nginx:1.17.1" Normal Created 26s kubelet, node1 Created container nginx Normal Started 25s kubelet, node1 Started container nginx Normal Pulled 7s (x3 over 25s) kubelet, node1 Container image "busybox:1.30" already present on machine Normal Created 7s (x3 over 25s) kubelet, node1 Created container busybox Normal Started 7s (x3 over 25s) kubelet, node1 Started container busybox
5.2.3 start command
In the previous case, there has always been a problem that has not been solved, that is, the busybox container has not been running successfully. What is the reason for the failure of this container?
Originally, busybox is not a program, but a collection of tool classes. After the kubernetes cluster starts management, it will close automatically. The solution is to keep it running, which requires command configuration.
Create the pod-command.yaml file as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-command
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","touch /tmp/hello.txt;while true;do /bin/echo $(date +%T) >> /tmp/hello.txt; sleep 3; done;"]
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-g8qsocyo-163354912840) (kubenetes. Assets / image-20210617224457945. PNG)]
Command, which is used to run a command after the initialization of the container in the pod.
Explain the meaning of the above command a little:
"/ bin/sh", "- c", execute the command with sh
touch /tmp/hello.txt; Create a / TMP / hello.txt file
while true; do /bin/echo $(date +%T) >> /tmp/hello.txt; sleep 3; done; Write the current time to the file every 3 seconds
# Create Pod [root@k8s-master01 pod]# kubectl create -f pod-command.yaml pod/pod-command created # View Pod status # At this time, it is found that both pod s are running normally [root@k8s-master01 pod]# kubectl get pods pod-command -n dev NAME READY STATUS RESTARTS AGE pod-command 2/2 Runing 0 2s # Enter the busybox container in the pod to view the contents of the file # Add a command: kubectl exec pod name - n namespace - it -c container name / bin/sh execute the command inside the container # Using this command, you can enter the interior of a container and then perform related operations # For example, you can view the contents of a txt file [root@k8s-master01 pod]# kubectl exec pod-command -n dev -it -c busybox /bin/sh / # tail -f /tmp/hello.txt 14:44:19 14:44:22 14:44:25
Special note:
Found above command The function of starting commands and passing parameters can be completed. Why do you provide one here args Options for passing parameters?This is actually the same as docker It doesn't matter, kubernetes Medium command,args The two are actually to achieve coverage Dockerfile in ENTRYPOINT The function of.
1 If command and args It's not written, so use it Dockerfile Configuration of.
2 If command Yes, but args No, so Dockerfile The default configuration will be ignored and the entered configuration will be executed command
3 If command No, but args Yes, so Dockerfile Configured in ENTRYPOINT The command will be executed using the current args Parameters of
4 If command and args It's all written, so Dockerfile The configuration of is ignored and cannot be executed command And add args parameter
5.2.4 environmental variables
Create the pod-env.yaml file as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-env
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do /bin/echo $(date +%T);sleep 60; done;"]
env: # Set environment variable list
- name: "username"
value: "admin"
- name: "password"
value: "123456"
env, environment variable, used to set the environment variable in the container in pod.
# Create Pod [root@k8s-master01 ~]# kubectl create -f pod-env.yaml pod/pod-env created # Enter the container and output the environment variable [root@k8s-master01 ~]# kubectl exec pod-env -n dev -c busybox -it /bin/sh / # echo $username admin / # echo $password 123456
This method is not recommended. It is recommended to store these configurations separately in the configuration file. This method will be described later.
5.2.5 port setting
This section describes the port settings of containers, that is, the ports option of containers.
First, look at the sub options supported by ports:
[root@k8s-master01 ~]# kubectl explain pod.spec.containers.ports KIND: Pod VERSION: v1 RESOURCE: ports <[]Object> FIELDS: name <string> # The port name, if specified, must be unique in the pod containerPort<integer> # Port on which the container will listen (0 < x < 65536) hostPort <integer> # The port on which the container is to be exposed on the host. If set, only one copy of the container can be run on the host (generally omitted) hostIP <string> # Host IP to which the external port is bound (generally omitted) protocol <string> # Port protocol. Must be UDP, TCP, or SCTP. The default is TCP.
Next, write a test case to create pod-ports.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-ports
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports: # Sets the list of ports exposed by the container
- name: nginx-port
containerPort: 80
protocol: TCP
# Create Pod
[root@k8s-master01 ~]# kubectl create -f pod-ports.yaml
pod/pod-ports created
# View pod
# You can see the configuration information clearly below
[root@k8s-master01 ~]# kubectl get pod pod-ports -n dev -o yaml
......
spec:
containers:
- image: nginx:1.17.1
imagePullPolicy: IfNotPresent
name: nginx
ports:
- containerPort: 80
name: nginx-port
protocol: TCP
......
To access the programs in the container, you need to use Podip:containerPort
5.2.6 resource quota
To run a program in a container, it must occupy certain resources, such as cpu and memory. If the resources of a container are not limited, it may eat a lot of resources and make other containers unable to run. In this case, kubernetes provides a quota mechanism for memory and cpu resources. This mechanism is mainly implemented through the resources option. It has two sub options:
- Limits: used to limit the maximum resources occupied by the container at runtime. When the resources occupied by the container exceed the limits, it will be terminated and restarted
- requests: used to set the minimum resources required by the container. If the environment resources are insufficient, the container will not start
You can set the upper and lower limits of resources through the above two options.
Next, write a test case to create pod-resources.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-resources
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
resources: # Resource quota
limits: # Limit resources (upper limit)
cpu: "2" # CPU limit, in core s
memory: "10Gi" # Memory limit
requests: # Requested resources (lower limit)
cpu: "1" # CPU limit, in core s
memory: "10Mi" # Memory limit
Here is a description of the units of cpu and memory:
- cpu: core number, which can be integer or decimal
- Memory: memory size, which can be in the form of Gi, Mi, G, M, etc
# Run Pod
[root@k8s-master01 ~]# kubectl create -f pod-resources.yaml
pod/pod-resources created
# It is found that the pod is running normally
[root@k8s-master01 ~]# kubectl get pod pod-resources -n dev
NAME READY STATUS RESTARTS AGE
pod-resources 1/1 Running 0 39s
# Next, stop Pod
[root@k8s-master01 ~]# kubectl delete -f pod-resources.yaml
pod "pod-resources" deleted
# Edit pod and change the value of resources.requests.memory to 10Gi
[root@k8s-master01 ~]# vim pod-resources.yaml
# Start the pod again
[root@k8s-master01 ~]# kubectl create -f pod-resources.yaml
pod/pod-resources created
# Check the Pod status and find that the Pod failed to start
[root@k8s-master01 ~]# kubectl get pod pod-resources -n dev -o wide
NAME READY STATUS RESTARTS AGE
pod-resources 0/1 Pending 0 20s
# Check the details of pod and you will find the following prompt
[root@k8s-master01 ~]# kubectl describe pod pod-resources -n dev
......
Warning FailedScheduling 35s default-scheduler 0/3 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 2 Insufficient memory.(insufficient memory)
5.3 Pod life cycle
We generally call the period from the creation to the end of a pod object as the life cycle of a pod. It mainly includes the following processes:
- pod creation process
- Run the init container process
- Run main container
- After the container is started (post start), before the container is terminated (pre stop)
- Survivability probe and readiness probe of container
- pod termination process
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-grlebpur-163354912841) (kubenetes. Assets / image-2020041111402706-1626782188724. PNG)]
In the whole life cycle, Pod will have five states (phases), as follows:
- Pending: apiserver has created the pod resource object, but it has not been scheduled or is still in the process of downloading the image
- Running: the pod has been scheduled to a node, and all containers have been created by kubelet
- Succeeded: all containers in the pod have been successfully terminated and will not be restarted
- Failed: all containers have been terminated, but at least one container failed to terminate, that is, the container returned an exit status with a value other than 0
- Unknown: apiserver cannot normally obtain the status information of pod object, which is usually caused by network communication failure
5.3.1 creation and termination
Creation process of pod
-
Users submit the pod information to be created to apiServer through kubectl or other api clients
-
apiServer starts to generate the information of the pod object, stores the information in etcd, and then returns the confirmation information to the client
-
apiServer starts to reflect the changes of pod objects in etcd, and other components use the watch mechanism to track and check the changes on apiServer
-
The scheduler finds that a new pod object needs to be created, starts assigning hosts to the pod and updates the result information to apiServer
-
The kubelet on the node finds that a pod has been dispatched, tries to call docker to start the container, and sends the result back to apiServer
-
apiServer stores the received pod status information into etcd
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-2disg0wc-163354912841) (kubenetes. Assets / image-20200406184656917-1626782168787. PNG)]
Termination process of pod
- The user sends a command to apiServer to delete the pod object
- The pod object information in apiserver will be updated over time. Within the grace period (30s by default), pod is regarded as dead
- Mark pod as terminating
- kubelet starts the closing process of pod while monitoring that the pod object is in the terminating state
- When the endpoint controller monitors the closing behavior of the pod object, it removes it from the endpoint list of all service resources matching this endpoint
- If the current pod object defines a preStop hook processor, it will start execution synchronously after it is marked as terminating
- The container process in the pod object received a stop signal
- After the grace period, if there are still running processes in the pod, the pod object will receive an immediate termination signal
- kubelet requests apiServer to set the grace period of this pod resource to 0 to complete the deletion operation. At this time, the pod is no longer visible to the user
5.3.2 initialize container
Initialization container is a container to be run before the main container of pod is started. It mainly does some pre work of the main container. It has two characteristics:
- The initialization container must run until it is completed. If an initialization container fails to run, kubernetes needs to restart it until it is completed successfully
- The initialization container must be executed in the defined order. If and only after the current one succeeds, the latter one can run
There are many application scenarios for initializing containers. The following are the most common:
- Provide tools, programs, or custom codes that are not available in the main container image
- The initialization container must be started and run serially before the application container, so it can be used to delay the startup of the application container until its dependent conditions are met
Next, make a case to simulate the following requirements:
Suppose you want to run nginx in the main container, but you need to be able to connect to the server where mysql and redis are located before running nginx
To simplify the test, specify the addresses of mysql(192.168.5.4) and redis(192.168.5.5) servers in advance
Create pod-initcontainer.yaml as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-initcontainer
namespace: dev
spec:
containers:
- name: main-container
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
initContainers:
- name: test-mysql
image: busybox:1.30
command: ['sh', '-c', 'until ping 192.168.159.101 -c 1 ; do echo waiting for mysql...; sleep 2; done;']
- name: test-redis
image: busybox:1.30
command: ['sh', '-c', 'until ping 192.168.159.102 -c 1 ; do echo waiting for reids...; sleep 2; done;']
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-initcontainer.yaml pod/pod-initcontainer created # View pod status # It is found that the following containers will not run when the pod card starts the first initialization container root@k8s-master01 ~]# kubectl describe pod pod-initcontainer -n dev ........ Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 49s default-scheduler Successfully assigned dev/pod-initcontainer to node1 Normal Pulled 48s kubelet, node1 Container image "busybox:1.30" already present on machine Normal Created 48s kubelet, node1 Created container test-mysql Normal Started 48s kubelet, node1 Started container test-mysql # View pod dynamically [root@k8s-master01 ~]# kubectl get pods pod-initcontainer -n dev -w NAME READY STATUS RESTARTS AGE pod-initcontainer 0/1 Init:0/2 0 15s pod-initcontainer 0/1 Init:1/2 0 52s pod-initcontainer 0/1 Init:1/2 0 53s pod-initcontainer 0/1 PodInitializing 0 89s pod-initcontainer 1/1 Running 0 90s # Next, open a new shell, add two IPS to the current server, and observe the change of pod [root@k8s-master01 ~]# ifconfig ens33:1 192.168.5.14 netmask 255.255.255.0 up [root@k8s-master01 ~]# ifconfig ens33:2 192.168.5.15 netmask 255.255.255.0 up
5.3.3 hook function
Hook functions can sense the events in their own life cycle and run the user specified program code when the corresponding time comes.
kubernetes provides two hook functions after the main container is started and before it is stopped:
- post start: executed after the container is created. If it fails, the container will be restarted
- pre stop: executed before the container terminates. After the execution is completed, the container will terminate successfully. The operation of deleting the container will be blocked before it is completed
The hook processor supports defining actions in the following three ways:
-
Exec command: execute the command once inside the container
...... lifecycle: postStart: exec: command: - cat - /tmp/healthy ...... -
TCPSocket: attempts to access the specified socket in the current container
...... lifecycle: postStart: tcpSocket: port: 8080 ...... -
HTTPGet: initiates an http request to a url in the current container
...... lifecycle: postStart: httpGet: path: / #URI address port: 80 #Port number host: 192.168.5.3 #Host address scheme: HTTP #Supported protocols, http or https ......
Next, take the exec method as an example to demonstrate the use of the following hook function and create a pod-hook-exec.yaml file, which is as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-hook-exec
namespace: dev
spec:
containers:
- name: main-container
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
lifecycle:
postStart:
exec: # When the container starts, execute a command to modify the default home page content of nginx
command: ["/bin/sh", "-c", "echo postStart... > /usr/share/nginx/html/index.html"]
preStop:
exec: # Stop the nginx service before the container stops
command: ["/usr/sbin/nginx","-s","quit"]
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-hook-exec.yaml pod/pod-hook-exec created # View pod [root@k8s-master01 ~]# kubectl get pods pod-hook-exec -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE pod-hook-exec 1/1 Running 0 29s 10.244.2.48 node2 # Access pod [root@k8s-master01 ~]# curl 10.244.2.48 postStart...
5.3.4 container detection
Container detection is used to detect whether the application instances in the container work normally. It is a traditional mechanism to ensure service availability. If the status of the instance does not meet the expectation after detection, kubernetes will "remove" the problem instance and do not undertake business traffic. Kubernetes provides two probes to realize container detection, namely:
- liveness probes: Live probes are used to detect whether the application instance is currently in normal operation. If not, k8s the container will be restarted
- readiness probes: readiness probes are used to detect whether the application instance can receive requests. If not, k8s it will not forward traffic
livenessProbe decides whether to restart the container, and readinessProbe decides whether to forward the request to the container.
The above two probes currently support three detection modes:
-
Exec command: execute the command once in the container. If the exit code of the command is 0, the program is considered normal, otherwise it is not normal
...... livenessProbe: exec: command: - cat - /tmp/healthy ...... -
TCPSocket: will try to access the port of a user container. If this connection can be established, the program is considered normal, otherwise it is not normal
...... livenessProbe: tcpSocket: port: 8080 ...... -
HTTPGet: call the URL of the Web application in the container. If the returned status code is between 200 and 399, the program is considered normal, otherwise it is not normal
...... livenessProbe: httpGet: path: / #URI address port: 80 #Port number host: 127.0.0.1 #Host address scheme: HTTP #Supported protocols, http or https ......
Let's take liveness probes as an example to demonstrate:
Method 1: Exec
Create pod-liveness-exec.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-exec
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
exec:
command: ["/bin/cat","/tmp/hello.txt"] # Execute a command to view files
Create a pod and observe the effect
# Create Pod [root@k8s-master01 ~]# kubectl create -f pod-liveness-exec.yaml pod/pod-liveness-exec created # View Pod details [root@k8s-master01 ~]# kubectl describe pods pod-liveness-exec -n dev ...... Normal Created 20s (x2 over 50s) kubelet, node1 Created container nginx Normal Started 20s (x2 over 50s) kubelet, node1 Started container nginx Normal Killing 20s kubelet, node1 Container nginx failed liveness probe, will be restarted Warning Unhealthy 0s (x5 over 40s) kubelet, node1 Liveness probe failed: cat: can't open '/tmp/hello11.txt': No such file or directory # Observing the above information, you will find that the health check is carried out after the nginx container is started # After the check fails, the container is kill ed, and then restart is attempted (this is the function of the restart strategy, which will be explained later) # After a while, observe the pod information, and you can see that the RESTARTS is no longer 0, but has been growing [root@k8s-master01 ~]# kubectl get pods pod-liveness-exec -n dev NAME READY STATUS RESTARTS AGE pod-liveness-exec 0/1 CrashLoopBackOff 2 3m19s # Of course, next, you can modify it to an existing file, such as / tmp/hello.txt. Try again, and the result will be normal
Mode 2: TCPSocket
Create pod-liveness-tcpsocket.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-tcpsocket
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
tcpSocket:
port: 8080 # Trying to access port 8080
Create a pod and observe the effect
# Create Pod [root@k8s-master01 ~]# kubectl create -f pod-liveness-tcpsocket.yaml pod/pod-liveness-tcpsocket created # View Pod details [root@k8s-master01 ~]# kubectl describe pods pod-liveness-tcpsocket -n dev ...... Normal Scheduled 31s default-scheduler Successfully assigned dev/pod-liveness-tcpsocket to node2 Normal Pulled <invalid> kubelet, node2 Container image "nginx:1.17.1" already present on machine Normal Created <invalid> kubelet, node2 Created container nginx Normal Started <invalid> kubelet, node2 Started container nginx Warning Unhealthy <invalid> (x2 over <invalid>) kubelet, node2 Liveness probe failed: dial tcp 10.244.2.44:8080: connect: connection refused # Observing the above information, I found that I tried to access port 8080, but failed # After a while, observe the pod information, and you can see that the RESTARTS is no longer 0, but has been growing [root@k8s-master01 ~]# kubectl get pods pod-liveness-tcpsocket -n dev NAME READY STATUS RESTARTS AGE pod-liveness-tcpsocket 0/1 CrashLoopBackOff 2 3m19s # Of course, next, you can change it to an accessible port, such as 80. Try again, and the result will be normal
Method 3: HTTPGet
Create pod-liveness-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-httpget
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet: # It's actually a visit http://127.0.0.1:80/hello
scheme: HTTP #Supported protocols, http or https
port: 80 #Port number
path: /hello #URI address
Create a pod and observe the effect
# Create Pod [root@k8s-master01 ~]# kubectl create -f pod-liveness-httpget.yaml pod/pod-liveness-httpget created # View Pod details [root@k8s-master01 ~]# kubectl describe pod pod-liveness-httpget -n dev ....... Normal Pulled 6s (x3 over 64s) kubelet, node1 Container image "nginx:1.17.1" already present on machine Normal Created 6s (x3 over 64s) kubelet, node1 Created container nginx Normal Started 6s (x3 over 63s) kubelet, node1 Started container nginx Warning Unhealthy 6s (x6 over 56s) kubelet, node1 Liveness probe failed: HTTP probe failed with statuscode: 404 Normal Killing 6s (x2 over 36s) kubelet, node1 Container nginx failed liveness probe, will be restarted # Observe the above information and try to access the path, but it is not found. A 404 error occurs # After a while, observe the pod information, and you can see that the RESTARTS is no longer 0, but has been growing [root@k8s-master01 ~]# kubectl get pod pod-liveness-httpget -n dev NAME READY STATUS RESTARTS AGE pod-liveness-httpget 1/1 Running 5 3m17s # Of course, next, you can modify it to an accessible path, such as /. Try again, and the result will be normal
So far, the liveness Probe has been used to demonstrate three detection methods. However, looking at the child properties of liveness Probe, you will find that there are other configurations in addition to these three methods, which are explained here:
[root@k8s-master01 ~]# kubectl explain pod.spec.containers.livenessProbe FIELDS: exec <Object> tcpSocket <Object> httpGet <Object> initialDelaySeconds <integer> # How many seconds do you wait for the first probe after the container starts timeoutSeconds <integer> # Probe timeout. Default 1 second, minimum 1 second periodSeconds <integer> # The frequency at which the probe is performed. The default is 10 seconds and the minimum is 1 second failureThreshold <integer> # How many times does a continuous probe fail before it is considered a failure. The default is 3. The minimum value is 1 successThreshold <integer> # How many times are successive detections considered successful. The default is 1
Here are two slightly configured to demonstrate the following effects:
[root@k8s-master01 ~]# more pod-liveness-httpget.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-liveness-httpget
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet:
scheme: HTTP
port: 80
path: /
initialDelaySeconds: 30 # Detection starts 30 s after the container is started
timeoutSeconds: 5 # The detection timeout is 5s
5.3.5 restart strategy
In the previous section, once there is a problem with the container detection, kubernetes will restart the pod where the container is located. In fact, this is determined by the pod restart strategy. There are three kinds of pod restart strategies, as follows:
- Always: automatically restart the container when it fails, which is also the default value.
- OnFailure: restart when the container terminates and the exit code is not 0
- Never: do not restart the container regardless of the status
The restart strategy is applicable to all containers in the pod object. The container that needs to be restarted for the first time will be restarted immediately when it needs to be restarted. The operation that needs to be restarted again will be delayed by kubelet for a period of time, and the delay length of repeated restart operations is 10s, 20s, 40s, 80s, 160s and 300s. 300s is the maximum delay length.
Create pod-restartpolicy.yaml:
apiVersion: v1
kind: Pod
metadata:
name: pod-restartpolicy
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- name: nginx-port
containerPort: 80
livenessProbe:
httpGet:
scheme: HTTP
port: 80
path: /hello
restartPolicy: Never # Set the restart policy to Never
Run Pod test
# Create Pod [root@k8s-master01 ~]# kubectl create -f pod-restartpolicy.yaml pod/pod-restartpolicy created # Check the Pod details and find that nginx container failed [root@k8s-master01 ~]# kubectl describe pods pod-restartpolicy -n dev ...... Warning Unhealthy 15s (x3 over 35s) kubelet, node1 Liveness probe failed: HTTP probe failed with statuscode: 404 Normal Killing 15s kubelet, node1 Container nginx failed liveness probe # Wait a little longer, and then observe the restart times of the pod. It is found that it has been 0 and has not been restarted [root@k8s-master01 ~]# kubectl get pods pod-restartpolicy -n dev NAME READY STATUS RESTARTS AGE pod-restartpolicy 0/1 Running 0 5min42s
5.4 Pod scheduling
By default, the Node on which a Pod runs is calculated by the Scheduler component using the corresponding algorithm. This process is not controlled manually. However, in actual use, this does not meet the needs of users, because in many cases, we want to control some pods to reach some nodes, so what should we do? This requires understanding the scheduling rules of kubernetes for Pod. Kubernetes provides four types of scheduling methods:
- Automatic scheduling: the node on which the Scheduler runs is completely calculated by a series of algorithms
- Directional scheduling: NodeName, NodeSelector
- Affinity scheduling: NodeAffinity, PodAffinity, PodAntiAffinity
- Stain tolerance scheduling: Taints, tolerance
5.4.1 directional dispatching
Directed scheduling refers to scheduling the pod to the desired node node by declaring nodeName or nodeSelector on the pod. Note that the scheduling here is mandatory, which means that even if the target node to be scheduled does not exist, it will be scheduled to the above, but the pod operation fails.
NodeName
NodeName is used to force the constraint to schedule the Pod to the Node with the specified Name. This method actually directly skips the Scheduler's scheduling logic and directly schedules the Pod to the Node with the specified Name.
Next, experiment: create a pod-nodename.yaml file
apiVersion: v1
kind: Pod
metadata:
name: pod-nodename
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: node1 # Specify scheduling to node1 node
#Create Pod [root@k8s-master01 ~]# kubectl create -f pod-nodename.yaml pod/pod-nodename created #Check the NODE attribute of Pod scheduling. It is indeed scheduled to node1 NODE [root@k8s-master01 ~]# kubectl get pods pod-nodename -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-nodename 1/1 Running 0 56s 10.244.1.87 node1 ...... # Next, delete the pod and change the value of nodeName to node3 (no node3 node) [root@k8s-master01 ~]# kubectl delete -f pod-nodename.yaml pod "pod-nodename" deleted [root@k8s-master01 ~]# vim pod-nodename.yaml [root@k8s-master01 ~]# kubectl create -f pod-nodename.yaml pod/pod-nodename created #Check again. It is found that the node3 node has been scheduled, but the pod cannot run normally because there is no node3 node [root@k8s-master01 ~]# kubectl get pods pod-nodename -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-nodename 0/1 Pending 0 6s <none> node3 ......
NodeSelector
NodeSelector is used to schedule the pod to the node node with the specified label added. It is implemented through kubernetes' label selector mechanism, that is, before the pod is created, the scheduler will use the MatchNodeSelector scheduling policy to match the label, find the target node, and then schedule the pod to the target node. The matching rule is a mandatory constraint.
Next, experiment:
1 first add labels for the node nodes respectively
[root@k8s-master01 ~]# kubectl label nodes node1 nodeenv=pro node/node2 labeled [root@k8s-master01 ~]# kubectl label nodes node2 nodeenv=test node/node2 labeled
2 create a pod-nodeselector.yaml file and use it to create a Pod
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeselector
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeSelector:
nodeenv: pro # Specifies to schedule to a node with nodeenv=pro tag
#Create Pod [root@k8s-master01 ~]# kubectl create -f pod-nodeselector.yaml pod/pod-nodeselector created #Check the NODE attribute of Pod scheduling. It is indeed scheduled to node1 NODE [root@k8s-master01 ~]# kubectl get pods pod-nodeselector -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-nodeselector 1/1 Running 0 47s 10.244.1.87 node1 ...... # Next, delete the pod and change the value of nodeSelector to nodeenv: xxxx (there is no node with this label) [root@k8s-master01 ~]# kubectl delete -f pod-nodeselector.yaml pod "pod-nodeselector" deleted [root@k8s-master01 ~]# vim pod-nodeselector.yaml [root@k8s-master01 ~]# kubectl create -f pod-nodeselector.yaml pod/pod-nodeselector created #Check again. It is found that the pod cannot operate normally, and the value of Node is none [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE pod-nodeselector 0/1 Pending 0 2m20s <none> <none> # Check the details and find the prompt of node selector matching failure [root@k8s-master01 ~]# kubectl describe pods pod-nodeselector -n dev ....... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling <unknown> default-scheduler 0/3 nodes are available: 3 node(s) didn't match node selector.
5.4.2 affinity scheduling
In the previous section, we introduced two directional scheduling methods, which are very convenient to use, but there are also some problems, that is, if there are no qualified nodes, the Pod will not be run, even if there is a list of available nodes in the cluster, which limits its use scenario.
Based on the above problem, kubernetes also provides an Affinity scheduling. It is extended on the basis of NodeSelector. Through configuration, it can give priority to the nodes that meet the conditions for scheduling. If not, it can also be scheduled to the nodes that do not meet the conditions, making the scheduling more flexible.
Affinity is mainly divided into three categories:
- node affinity: Aiming at nodes, it solves the problem of which nodes a pod can schedule
- Podaffinity: Aiming at pods, it solves the problem of which existing pods can be deployed in the same topology domain
- Pod anti affinity: Aiming at pod, it solves the problem that pod cannot be deployed in the same topology domain with existing pods
Description of affinity (anti affinity) usage scenarios:
Affinity: if two applications interact frequently, it is necessary to use affinity to make the two applications as close as possible, so as to reduce the performance loss caused by network communication.
Anti affinity: when the application is deployed with multiple replicas, it is necessary to use anti affinity to make each application instance scattered and distributed on each node, which can improve the high availability of the service.
NodeAffinity
First, let's take a look at the configurable items of NodeAffinity:
pod.spec.affinity.nodeAffinity
requiredDuringSchedulingIgnoredDuringExecution Node The node must meet all the specified rules, which is equivalent to a hard limit
nodeSelectorTerms Node selection list
matchFields List of node selector requirements by node field
matchExpressions List of node selector requirements by node label(recommend)
key key
values value
operator Relational character support Exists, DoesNotExist, In, NotIn, Gt, Lt
preferredDuringSchedulingIgnoredDuringExecution Priority scheduling to meet the specified rules Node，Equivalent to soft limit (inclination)
preference A node selector item associated with the corresponding weight
matchFields List of node selector requirements by node field
matchExpressions List of node selector requirements by node label(recommend)
key key
values value
operator Relational character support In, NotIn, Exists, DoesNotExist, Gt, Lt
weight Propensity weight, in range 1-100.
Instructions for using relation characters:
- matchExpressions:
- key: nodeenv # Match the node with nodeenv as the key with label
operator: Exists
- key: nodeenv # The key of the matching tag is nodeenv and the value is "xxx" or "yyy"
operator: In
values: ["xxx","yyy"]
- key: nodeenv # The key of the matching tag is nodeenv and the value is greater than "xxx"
operator: Gt
values: "xxx"
Next, first demonstrate the required duringschedulingignored duringexecution,
Create pod-nodeaffinity-required.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #Affinity settings
nodeAffinity: #Set node affinity
requiredDuringSchedulingIgnoredDuringExecution: # Hard limit
nodeSelectorTerms:
- matchExpressions: # Match the label of the value of env in ["xxx","yyy"]
- key: nodeenv
operator: In
values: ["xxx","yyy"]
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-required.yaml pod/pod-nodeaffinity-required created # View pod status (failed to run) [root@k8s-master01 ~]# kubectl get pods pod-nodeaffinity-required -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-nodeaffinity-required 0/1 Pending 0 16s <none> <none> ...... # View Pod details # If the scheduling fails, the node selection fails [root@k8s-master01 ~]# kubectl describe pod pod-nodeaffinity-required -n dev ...... Warning FailedScheduling <unknown> default-scheduler 0/3 nodes are available: 3 node(s) didn't match node selector. Warning FailedScheduling <unknown> default-scheduler 0/3 nodes are available: 3 node(s) didn't match node selector. #Next, stop the pod [root@k8s-master01 ~]# kubectl delete -f pod-nodeaffinity-required.yaml pod "pod-nodeaffinity-required" deleted # Modify the file and set values: ["XXX", "YYY"] ---- > ["pro", "YYY"] [root@k8s-master01 ~]# vim pod-nodeaffinity-required.yaml # Restart [root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-required.yaml pod/pod-nodeaffinity-required created # At this time, it is found that the scheduling is successful and the pod has been scheduled to node1 [root@k8s-master01 ~]# kubectl get pods pod-nodeaffinity-required -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-nodeaffinity-required 1/1 Running 0 11s 10.244.1.89 node1 ......
Next, let's demonstrate the required duringschedulingignored duringexecution,
Create pod-nodeaffinity-preferred.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-nodeaffinity-preferred
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #Affinity settings
nodeAffinity: #Set node affinity
preferredDuringSchedulingIgnoredDuringExecution: # Soft limit
- weight: 1
preference:
matchExpressions: # The tag matching the value of env in ["xxx","yyy"] (not available in the current environment)
- key: nodeenv
operator: In
values: ["xxx","yyy"]
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-nodeaffinity-preferred.yaml pod/pod-nodeaffinity-preferred created # View the pod status (running successfully) [root@k8s-master01 ~]# kubectl get pod pod-nodeaffinity-preferred -n dev NAME READY STATUS RESTARTS AGE pod-nodeaffinity-preferred 1/1 Running 0 40s
NodeAffinity Precautions for rule setting:
1 If defined at the same time nodeSelector and nodeAffinity，Then both conditions must be met, Pod To run on the specified Node upper
2 If nodeAffinity Multiple specified nodeSelectorTerms，Then only one of them can match successfully
3 If one nodeSelectorTerms Multiple in matchExpressions ，Then a node must meet all requirements to match successfully
4 If one pod Where Node stay Pod Its label has changed during operation and is no longer in compliance with this requirement Pod The system will ignore this change
PodAffinity
PodAffinity mainly implements the function of making the newly created pod and the reference pod in the same area with the running pod as the reference.
First, let's take a look at the configurable items of PodAffinity:
pod.spec.affinity.podAffinity
requiredDuringSchedulingIgnoredDuringExecution Hard limit
namespaces Specify reference pod of namespace
topologyKey Specify scheduling scope
labelSelector tag chooser
matchExpressions List of node selector requirements by node label(recommend)
key key
values value
operator Relational character support In, NotIn, Exists, DoesNotExist.
matchLabels Refers to multiple matchExpressions Mapped content
preferredDuringSchedulingIgnoredDuringExecution Soft limit
podAffinityTerm option
namespaces
topologyKey
labelSelector
matchExpressions
key key
values value
operator
matchLabels
weight Propensity weight, in range 1-100
topologyKey Used to specify the dispatch time scope,for example:
If specified as kubernetes.io/hostname，That is to Node The node is a differentiated range
If specified as beta.kubernetes.io/os,Then Node The operating system type of the node
Next, demonstrate the requiredDuringSchedulingIgnoredDuringExecution,
1) First, create a reference Pod, pod-podaffinity-target.yaml:
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-target
namespace: dev
labels:
podenv: pro #Set label
spec:
containers:
- name: nginx
image: nginx:1.17.1
nodeName: node1 # Specify the target pod name on node1
# Start target pod [root@k8s-master01 ~]# kubectl create -f pod-podaffinity-target.yaml pod/pod-podaffinity-target created # View pod status [root@k8s-master01 ~]# kubectl get pods pod-podaffinity-target -n dev NAME READY STATUS RESTARTS AGE pod-podaffinity-target 1/1 Running 0 4s
2) Create pod-podaffinity-required.yaml as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-podaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #Affinity settings
podAffinity: #Set pod affinity
requiredDuringSchedulingIgnoredDuringExecution: # Hard limit
- labelSelector:
matchExpressions: # Match the label of the value of env in ["xxx","yyy"]
- key: podenv
operator: In
values: ["xxx","yyy"]
topologyKey: kubernetes.io/hostname
The above configuration means that the new pod must be on the same Node as the pod with the label nodeenv=xxx or nodeenv=yyy. Obviously, there is no such pod now. Next, run the test.
# Start pod [root@k8s-master01 ~]# kubectl create -f pod-podaffinity-required.yaml pod/pod-podaffinity-required created # Check the pod status and find that it is not running [root@k8s-master01 ~]# kubectl get pods pod-podaffinity-required -n dev NAME READY STATUS RESTARTS AGE pod-podaffinity-required 0/1 Pending 0 9s # View details [root@k8s-master01 ~]# kubectl describe pods pod-podaffinity-required -n dev ...... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling <unknown> default-scheduler 0/3 nodes are available: 2 node(s) didn't match pod affinity rules, 1 node(s) had taints that the pod didn't tolerate. # Next, modify values: ["XXX", "YYY"] ---- > values: ["pro", "YYY"] # This means that the new pod must be on the same Node as the pod with the label nodeenv=xxx or nodeenv=yyy [root@k8s-master01 ~]# vim pod-podaffinity-required.yaml # Then recreate the pod to see the effect [root@k8s-master01 ~]# kubectl delete -f pod-podaffinity-required.yaml pod "pod-podaffinity-required" deleted [root@k8s-master01 ~]# kubectl create -f pod-podaffinity-required.yaml pod/pod-podaffinity-required created # It is found that the Pod is running normally at this time [root@k8s-master01 ~]# kubectl get pods pod-podaffinity-required -n dev NAME READY STATUS RESTARTS AGE LABELS pod-podaffinity-required 1/1 Running 0 6s <none>
The preferred during scheduling ignored during execution of PodAffinity will not be demonstrated here.
PodAntiAffinity
PodAntiAffinity mainly implements the function of taking the running pod as the reference, so that the newly created pod and the reference pod are not in the same area.
Its configuration mode and options are the same as those of podaffinity. There is no detailed explanation here. Just make a test case.
1) Continue to use the target pod in the previous case
[root@k8s-master01 ~]# kubectl get pods -n dev -o wide --show-labels NAME READY STATUS RESTARTS AGE IP NODE LABELS pod-podaffinity-required 1/1 Running 0 3m29s 10.244.1.38 node1 <none> pod-podaffinity-target 1/1 Running 0 9m25s 10.244.1.37 node1 podenv=pro
2) Create pod-podandiaffinity-required.yaml as follows:
apiVersion: v1
kind: Pod
metadata:
name: pod-podantiaffinity-required
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
affinity: #Affinity settings
podAntiAffinity: #Set pod affinity
requiredDuringSchedulingIgnoredDuringExecution: # Hard limit
- labelSelector:
matchExpressions: # Match the label of the value of podenv in ["pro"]
- key: podenv
operator: In
values: ["pro"]
topologyKey: kubernetes.io/hostname
The above configuration means that the new pod must not be on the same Node as the pod with the label nodeenv=pro. Run the test.
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-podantiaffinity-required.yaml pod/pod-podantiaffinity-required created # View pod # It is found that the schedule is on node2 [root@k8s-master01 ~]# kubectl get pods pod-podantiaffinity-required -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE .. pod-podantiaffinity-required 1/1 Running 0 30s 10.244.1.96 node2 ..
5.4.3 stain and tolerance
Taints
The previous scheduling methods are based on the point of view of the Pod. By adding attributes to the Pod, we can determine whether the Pod should be scheduled to the specified Node. In fact, we can also decide whether to allow the Pod to be scheduled from the point of view of the Node by adding stain attributes to the Node.
After the Node is tainted, there is a mutually exclusive relationship between the Node and the Pod, and then the Pod scheduling is rejected, and the existing Pod can even be expelled.
The format of the stain is: key = value: effect. Key and value are the labels of the stain. Effect describes the function of the stain. The following three options are supported:
- PreferNoSchedule: kubernetes will try to avoid scheduling pods to nodes with this stain unless there are no other nodes to schedule
- NoSchedule: kubernetes will not schedule the Pod to the Node with the stain, but will not affect the existing Pod on the current Node
- NoExecute: kubernetes will not schedule the Pod to the Node with the stain, but will also drive the existing Pod on the Node away
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-yqv2zheh-163354912842) (kubenetes. Assets / image-20200605021831545. PNG)]
Examples of commands for setting and removing stains using kubectl are as follows:
# Set stain kubectl taint nodes node1 key=value:effect # Remove stains kubectl taint nodes node1 key:effect- # Remove all stains kubectl taint nodes node1 key-
Next, demonstrate the effect of the stain:
- Prepare node node1 (to make the demonstration effect more obvious, temporarily stop node node2)
- Set a stain for node1 node: tag=heima:PreferNoSchedule; Then create pod1 (pod1 can)
- Modify node1 node to set a stain: tag=heima:NoSchedule; Then create pod2 (pod1 is normal and pod2 fails)
- Modify to set a stain for node1 node: tag=heima:NoExecute; Then create pod3 (all three pods fail)
# Set stain for node1 (PreferNoSchedule) [root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:PreferNoSchedule # Create pod1 [root@k8s-master01 ~]# kubectl run taint1 --image=nginx:1.17.1 -n dev [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE taint1-7665f7fd85-574h4 1/1 Running 0 2m24s 10.244.1.59 node1 # Set stain for node1 (cancel PreferNoSchedule and set NoSchedule) [root@k8s-master01 ~]# kubectl taint nodes node1 tag:PreferNoSchedule- [root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:NoSchedule # Create pod2 [root@k8s-master01 ~]# kubectl run taint2 --image=nginx:1.17.1 -n dev [root@k8s-master01 ~]# kubectl get pods taint2 -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE taint1-7665f7fd85-574h4 1/1 Running 0 2m24s 10.244.1.59 node1 taint2-544694789-6zmlf 0/1 Pending 0 21s <none> <none> # Set stain for node1 (cancel NoSchedule and set NoExecute) [root@k8s-master01 ~]# kubectl taint nodes node1 tag:NoSchedule- [root@k8s-master01 ~]# kubectl taint nodes node1 tag=heima:NoExecute # Create pod3 [root@k8s-master01 ~]# kubectl run taint3 --image=nginx:1.17.1 -n dev [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED taint1-7665f7fd85-htkmp 0/1 Pending 0 35s <none> <none> <none> taint2-544694789-bn7wb 0/1 Pending 0 35s <none> <none> <none> taint3-6d78dbd749-tktkq 0/1 Pending 0 6s <none> <none> <none>
Tip:
use kubeadm The cluster will be set up by default master Add a stain mark to the node,therefore pod It won't be scheduled master On node.
Tolerance
The above describes the role of stains. We can add stains on nodes to reject pod scheduling. However, what should we do if we want to schedule a pod to a node with stains? This requires tolerance.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-w3cjngiw-163354912842) (kubenetes. Assets / image-20200514095913741. PNG)]
Stain means rejection, tolerance means neglect. Node rejects the pod through stain, and pod ignores rejection through tolerance
Let's look at the effect through a case:
- In the previous section, the node1 node has been marked with the stain of NoExecute. At this time, the pod cannot be scheduled
- In this section, you can add tolerance to the pod and then schedule it
Create pod-tolerance.yaml as follows
apiVersion: v1
kind: Pod
metadata:
name: pod-toleration
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
tolerations: # Add tolerance
- key: "tag" # The key to the stain to be tolerated
operator: "Equal" # Operator
value: "heima" # The value of tolerated stains
effect: "NoExecute" # Add the tolerance rule, which must be the same as the stain rule of the mark
# Add pod before tolerance [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED pod-toleration 0/1 Pending 0 3s <none> <none> <none> # Add pod after tolerance [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED pod-toleration 1/1 Running 0 3s 10.244.1.62 node1 <none>
Let's take a look at the detailed configuration of tolerance:
[root@k8s-master01 ~]# kubectl explain pod.spec.tolerations ...... FIELDS: key # Corresponding to the key of the stain to be tolerated, null means to match all the keys value # The value corresponding to the stain to be tolerated operator # The key value operator supports Equal and Exists (default) effect # For the effect corresponding to the stain, null means to match all effects tolerationSeconds # Tolerance time, which takes effect when the effect is NoExecute, indicates the residence time of the pod on the Node
6. Detailed explanation of pod controller
6.1 introduction to pod controller
Pod is the smallest management unit of kubernetes. In kubernetes, it can be divided into two categories according to the creation method of Pod:
- Autonomous Pod: a pod directly created by kubernetes. This kind of pod will not exist and will not be rebuilt after deletion
- Pod created by the controller: the pod created by kubernetes through the controller. After this pod is deleted, it will be rebuilt automatically
What is a Pod controller
The pod controller is the middle layer for managing pods. After using the pod controller, you only need to tell the pod controller how many and what kind of pods you want. It will create qualified pods and ensure that each pod resource is in the desired target state. If the pod resource fails during operation, it rearranges the pod based on the specified policy.
In kubernetes, there are many types of pod controllers, each of which has its own suitable scenario. The common ones are as follows:
- ReplicationController: compared with the original pod controller, it has been abandoned and replaced by ReplicaSet
- ReplicaSet: ensure that the number of replicas is always maintained at the expected value, and support the expansion and contraction of pod number and image version upgrade
- Deployment: controls Pod by controlling ReplicaSet, and supports rolling upgrade and rollback version
- Horizontal Pod Autoscaler: the number of pods can be automatically adjusted according to the cluster load level to realize peak shaving and valley filling
- Daemon set: runs on the specified Node in the cluster and only runs one copy. It is generally used for the tasks of the daemon class
- Job: the created pod will exit as soon as the task is completed. It does not need to restart or rebuild. It is used to perform one-time tasks
- Cronjob: the Pod it creates is responsible for periodic task control and does not need to run continuously in the background
- Stateful set: manage stateful applications
6.2 ReplicaSet(RS)
The main function of ReplicaSet is to ensure the normal operation of a certain number of pods. It will continuously monitor the operation status of these pods. Once the pod fails, it will restart or rebuild. At the same time, it also supports the expansion and contraction of the number of pods and the upgrading and upgrading of the image version.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-z3jzyohg-163354912842) (kubenetes. Assets / image-20200612005334159. PNG)]
Resource manifest file for ReplicaSet:
apiVersion: apps/v1 # Version number
kind: ReplicaSet # type
metadata: # metadata
name: # rs name
namespace: # Namespace
labels: #label
controller: rs
spec: # Detailed description
replicas: 3 # Number of copies
selector: # Selector, which specifies which pod s the controller manages
matchLabels: # Labels matching rule
app: nginx-pod
matchExpressions: # Expressions matching rule
- {key: app, operator: In, values: [nginx-pod]}
template: # Template. When the number of copies is insufficient, a pod copy will be created according to the following template
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
Here, the configuration items that need to be newly understood are the following options of spec:
-
Replicas: Specifies the number of replicas, which is actually the number of pod s created by the current rs. the default value is 1
-
Selector: selector, which is used to establish the association relationship between pod controller and pod, and adopts the Label Selector mechanism
Define a label on the pod template and a selector on the controller to indicate which pods the current controller can manage
-
Template: template is the template used by the current controller to create a pod. In fact, it is the definition of pod learned in the previous chapter
Create ReplicaSet
Create a pc-replicaset.yaml file as follows:
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: pc-replicaset
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
# Create rs [root@k8s-master01 ~]# kubectl create -f pc-replicaset.yaml replicaset.apps/pc-replicaset created # View rs # Specified: expected number of replicas # CURRENT: CURRENT copy quantity # READY: number of copies READY for service [root@k8s-master01 ~]# kubectl get rs pc-replicaset -n dev -o wide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR pc-replicaset 3 3 3 22s nginx nginx:1.17.1 app=nginx-pod # View the pod created by the current controller # It is found that the name of the pod created by the controller is spliced with - xxxxx random code after the controller name [root@k8s-master01 ~]# kubectl get pod -n dev NAME READY STATUS RESTARTS AGE pc-replicaset-6vmvt 1/1 Running 0 54s pc-replicaset-fmb8f 1/1 Running 0 54s pc-replicaset-snrk2 1/1 Running 0 54s
Expansion and contraction capacity
# Edit the number of copies of rs and modify spec:replicas: 6 [root@k8s-master01 ~]# kubectl edit rs pc-replicaset -n dev replicaset.apps/pc-replicaset edited # View pod [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-replicaset-6vmvt 1/1 Running 0 114m pc-replicaset-cftnp 1/1 Running 0 10s pc-replicaset-fjlm6 1/1 Running 0 10s pc-replicaset-fmb8f 1/1 Running 0 114m pc-replicaset-s2whj 1/1 Running 0 10s pc-replicaset-snrk2 1/1 Running 0 114m # Of course, you can also use commands directly # Use the scale command to expand or shrink the capacity, and then -- replicas=n directly specify the target quantity [root@k8s-master01 ~]# kubectl scale rs pc-replicaset --replicas=2 -n dev replicaset.apps/pc-replicaset scaled # After the command is run, check it immediately and find that there are 4 ready to exit [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-replicaset-6vmvt 0/1 Terminating 0 118m pc-replicaset-cftnp 0/1 Terminating 0 4m17s pc-replicaset-fjlm6 0/1 Terminating 0 4m17s pc-replicaset-fmb8f 1/1 Running 0 118m pc-replicaset-s2whj 0/1 Terminating 0 4m17s pc-replicaset-snrk2 1/1 Running 0 118m #Wait a moment, there are only two left [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-replicaset-fmb8f 1/1 Running 0 119m pc-replicaset-snrk2 1/1 Running 0 119m
Image upgrade
# Edit container image of rs - image: nginx:1.17.2 [root@k8s-master01 ~]# kubectl edit rs pc-replicaset -n dev replicaset.apps/pc-replicaset edited # Check again and find that the image version has changed [root@k8s-master01 ~]# kubectl get rs -n dev -o wide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES ... pc-replicaset 2 2 2 140m nginx nginx:1.17.2 ... # In the same way, you can use commands to do this # Kubectl set image RS name container = mirror version - n namespace [root@k8s-master01 ~]# kubectl set image rs pc-replicaset nginx=nginx:1.17.1 -n dev replicaset.apps/pc-replicaset image updated # Check again and find that the image version has changed [root@k8s-master01 ~]# kubectl get rs -n dev -o wide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES ... pc-replicaset 2 2 2 145m nginx nginx:1.17.1 ...
Delete ReplicaSet
# Using the kubectl delete command deletes this RS and the Pod it manages # Before kubernetes deletes RS, the replicasclear of RS will be adjusted to 0. After all pods are deleted, delete RS objects [root@k8s-master01 ~]# kubectl delete rs pc-replicaset -n dev replicaset.apps "pc-replicaset" deleted [root@k8s-master01 ~]# kubectl get pod -n dev -o wide No resources found in dev namespace. # If you want to delete only RS objects (keep Pod), you can add -- cascade=false option when using kubectl delete command (not recommended). [root@k8s-master01 ~]# kubectl delete rs pc-replicaset -n dev --cascade=false replicaset.apps "pc-replicaset" deleted [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-replicaset-cl82j 1/1 Running 0 75s pc-replicaset-dslhb 1/1 Running 0 75s # You can also use yaml to delete directly (recommended) [root@k8s-master01 ~]# kubectl delete -f pc-replicaset.yaml replicaset.apps "pc-replicaset" deleted
6.3 Deployment(Deploy)
In order to better solve the problem of service orchestration, kubernetes introduced the Deployment controller in V1.2. It is worth mentioning that this controller does not directly manage pod, but introduces the management of pod by managing ReplicaSet, that is, Deployment manages ReplicaSet and ReplicaSet manages pod. Therefore, Deployment is more powerful than ReplicaSet.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-mzny9mgu-163354912843) (kubenetes. Assets / image-20200612005524778. PNG)]
The main functions of Deployment are as follows:
- All features of ReplicaSet are supported
- Support the stop and continue of publishing
- Support rolling upgrade and rollback version
Resource manifest file for Deployment:
apiVersion: apps/v1 # Version number
kind: Deployment # type
metadata: # metadata
name: # rs name
namespace: # Namespace
labels: #label
controller: deploy
spec: # Detailed description
replicas: 3 # Number of copies
revisionHistoryLimit: 3 # Keep historical version
paused: false # Pause deployment. The default value is false
progressDeadlineSeconds: 600 # Deployment timeout (s). The default is 600
strategy: # strategy
type: RollingUpdate # Rolling update strategy
rollingUpdate: # Rolling update
maxSurge: 30% # The maximum number of copies that can exist, either as a percentage or as an integer
maxUnavailable: 30% # The maximum value of the Pod in the maximum unavailable state, which can be either a percentage or an integer
selector: # Selector, which specifies which pod s the controller manages
matchLabels: # Labels matching rule
app: nginx-pod
matchExpressions: # Expressions matching rule
- {key: app, operator: In, values: [nginx-pod]}
template: # Template. When the number of copies is insufficient, a pod copy will be created according to the following template
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
Create deployment
Create pc-deployment.yaml as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
# Create deployment [root@k8s-master01 ~]# kubectl create -f pc-deployment.yaml --record=true deployment.apps/pc-deployment created # View deployment # UP-TO-DATE number of pod s of the latest version # AVAILABLE the number of currently AVAILABLE pod s [root@k8s-master01 ~]# kubectl get deploy pc-deployment -n dev NAME READY UP-TO-DATE AVAILABLE AGE pc-deployment 3/3 3 3 15s # View rs # It is found that the name of rs is a 10 digit random string added after the name of the original deployment [root@k8s-master01 ~]# kubectl get rs -n dev NAME DESIRED CURRENT READY AGE pc-deployment-6696798b78 3 3 3 23s # View pod [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-deployment-6696798b78-d2c8n 1/1 Running 0 107s pc-deployment-6696798b78-smpvp 1/1 Running 0 107s pc-deployment-6696798b78-wvjd8 1/1 Running 0 107s
Expansion and contraction capacity
# The number of changed copies is 5 [root@k8s-master01 ~]# kubectl scale deploy pc-deployment --replicas=5 -n dev deployment.apps/pc-deployment scaled # View deployment [root@k8s-master01 ~]# kubectl get deploy pc-deployment -n dev NAME READY UP-TO-DATE AVAILABLE AGE pc-deployment 5/5 5 5 2m # View pod [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-deployment-6696798b78-d2c8n 1/1 Running 0 4m19s pc-deployment-6696798b78-jxmdq 1/1 Running 0 94s pc-deployment-6696798b78-mktqv 1/1 Running 0 93s pc-deployment-6696798b78-smpvp 1/1 Running 0 4m19s pc-deployment-6696798b78-wvjd8 1/1 Running 0 4m19s # Edit the number of copies of deployment and modify spec:replicas: 4 [root@k8s-master01 ~]# kubectl edit deploy pc-deployment -n dev deployment.apps/pc-deployment edited # View pod [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-deployment-6696798b78-d2c8n 1/1 Running 0 5m23s pc-deployment-6696798b78-jxmdq 1/1 Running 0 2m38s pc-deployment-6696798b78-smpvp 1/1 Running 0 5m23s pc-deployment-6696798b78-wvjd8 1/1 Running 0 5m23s
Mirror update
deployment supports two update strategies: rebuild update and rolling update. You can specify the policy type through strategy and support two attributes:
strategy: Specify a new Pod Replace old Pod The policy supports two properties:
type: Specify the policy type and support two policies
Recreate: Create a new Pod We'll kill all the existing before Pod
RollingUpdate: Rolling update is to kill a part and start a part. During the update process, there are two versions Pod
rollingUpdate: When type by RollingUpdate Effective when used for RollingUpdate Setting parameters supports two properties:
maxUnavailable: Used to specify not available during upgrade Pod The maximum quantity of is 25 by default%.
maxSurge: Used to specify that you can exceed expectations during the upgrade process Pod The maximum quantity of is 25 by default%.
Rebuild update
- Edit pc-deployment.yaml and add an update policy under the spec node
spec:
strategy: # strategy
type: Recreate # Rebuild update
- Create deploy for validation
# Change image [root@k8s-master01 ~]# kubectl set image deployment pc-deployment nginx=nginx:1.17.2 -n dev deployment.apps/pc-deployment image updated # Observe the upgrade process [root@k8s-master01 ~]# kubectl get pods -n dev -w NAME READY STATUS RESTARTS AGE pc-deployment-5d89bdfbf9-65qcw 1/1 Running 0 31s pc-deployment-5d89bdfbf9-w5nzv 1/1 Running 0 31s pc-deployment-5d89bdfbf9-xpt7w 1/1 Running 0 31s pc-deployment-5d89bdfbf9-xpt7w 1/1 Terminating 0 41s pc-deployment-5d89bdfbf9-65qcw 1/1 Terminating 0 41s pc-deployment-5d89bdfbf9-w5nzv 1/1 Terminating 0 41s pc-deployment-675d469f8b-grn8z 0/1 Pending 0 0s pc-deployment-675d469f8b-hbl4v 0/1 Pending 0 0s pc-deployment-675d469f8b-67nz2 0/1 Pending 0 0s pc-deployment-675d469f8b-grn8z 0/1 ContainerCreating 0 0s pc-deployment-675d469f8b-hbl4v 0/1 ContainerCreating 0 0s pc-deployment-675d469f8b-67nz2 0/1 ContainerCreating 0 0s pc-deployment-675d469f8b-grn8z 1/1 Running 0 1s pc-deployment-675d469f8b-67nz2 1/1 Running 0 1s pc-deployment-675d469f8b-hbl4v 1/1 Running 0 2s
Rolling update
- Edit pc-deployment.yaml and add an update policy under the spec node
spec:
strategy: # strategy
type: RollingUpdate # Rolling update strategy
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
- Create deploy for validation
# Change image [root@k8s-master01 ~]# kubectl set image deployment pc-deployment nginx=nginx:1.17.3 -n dev deployment.apps/pc-deployment image updated # Observe the upgrade process [root@k8s-master01 ~]# kubectl get pods -n dev -w NAME READY STATUS RESTARTS AGE pc-deployment-c848d767-8rbzt 1/1 Running 0 31m pc-deployment-c848d767-h4p68 1/1 Running 0 31m pc-deployment-c848d767-hlmz4 1/1 Running 0 31m pc-deployment-c848d767-rrqcn 1/1 Running 0 31m pc-deployment-966bf7f44-226rx 0/1 Pending 0 0s pc-deployment-966bf7f44-226rx 0/1 ContainerCreating 0 0s pc-deployment-966bf7f44-226rx 1/1 Running 0 1s pc-deployment-c848d767-h4p68 0/1 Terminating 0 34m pc-deployment-966bf7f44-cnd44 0/1 Pending 0 0s pc-deployment-966bf7f44-cnd44 0/1 ContainerCreating 0 0s pc-deployment-966bf7f44-cnd44 1/1 Running 0 2s pc-deployment-c848d767-hlmz4 0/1 Terminating 0 34m pc-deployment-966bf7f44-px48p 0/1 Pending 0 0s pc-deployment-966bf7f44-px48p 0/1 ContainerCreating 0 0s pc-deployment-966bf7f44-px48p 1/1 Running 0 0s pc-deployment-c848d767-8rbzt 0/1 Terminating 0 34m pc-deployment-966bf7f44-dkmqp 0/1 Pending 0 0s pc-deployment-966bf7f44-dkmqp 0/1 ContainerCreating 0 0s pc-deployment-966bf7f44-dkmqp 1/1 Running 0 2s pc-deployment-c848d767-rrqcn 0/1 Terminating 0 34m # At this point, the new version of pod is created and the version of pod is destroyed # The intermediate process is rolling, that is, creating while destroying
Rolling update process:
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-mzphqkx6-163354912843) (kubenetes. Assets / image-20200416140251491. PNG)]
Changes of rs in mirror update
# Looking at rs, it is found that the original rs still exists, but the number of pods becomes 0, and then a new rs is generated, and the number of pods is 4 # In fact, this is the secret of version fallback in deployment, which will be explained in detail later [root@k8s-master01 ~]# kubectl get rs -n dev NAME DESIRED CURRENT READY AGE pc-deployment-6696798b78 0 0 0 7m37s pc-deployment-6696798b11 0 0 0 5m37s pc-deployment-c848d76789 4 4 4 72s
Version fallback
deployment supports many functions such as pause, resume and version fallback during version upgrade, as shown below
kubectl rollout: functions related to version upgrade. The following options are supported:
- Status displays the current upgrade status
- History displays the upgrade history
- Pause pause the version upgrade process
- resume resumes the suspended version upgrade process
- Restart restart the version upgrade process
- undo rollback to the previous version (you can use – to revision to rollback to the specified version)
# View the status of the current upgraded version [root@k8s-master01 ~]# kubectl rollout status deploy pc-deployment -n dev deployment "pc-deployment" successfully rolled out # View upgrade history [root@k8s-master01 ~]# kubectl rollout history deploy pc-deployment -n dev deployment.apps/pc-deployment REVISION CHANGE-CAUSE 1 kubectl create --filename=pc-deployment.yaml --record=true 2 kubectl create --filename=pc-deployment.yaml --record=true 3 kubectl create --filename=pc-deployment.yaml --record=true # It can be found that there are three version records, indicating that two upgrades have been completed # Version rollback # Here, you can directly use -- to revision = 1 to roll back to version 1. If you omit this option, you will roll back to the previous version, which is version 2 [root@k8s-master01 ~]# kubectl rollout undo deployment pc-deployment --to-revision=1 -n dev deployment.apps/pc-deployment rolled back # The first version can be found through the nginx image version [root@k8s-master01 ~]# kubectl get deploy -n dev -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES pc-deployment 4/4 4 4 74m nginx nginx:1.17.1 # Looking at the rs, it is found that four pods in the first rs are running, and the pods in the later two versions of rs are running # In fact, the reason why deployment can roll back the version is to record the history rs, # Once you want to rollback to which version, you only need to reduce the number of pods of the current version to 0, and then increase the number of pods of the rollback version to the target number [root@k8s-master01 ~]# kubectl get rs -n dev NAME DESIRED CURRENT READY AGE pc-deployment-6696798b78 4 4 4 78m pc-deployment-966bf7f44 0 0 0 37m pc-deployment-c848d767 0 0 0 71m
Canary release
The Deployment controller supports control during the update process, such as "pause" or "resume" update operations.
For example, when a batch of new Pod resources are created, the update process is suspended immediately. At this time, there are only some new versions of applications, and the main part is still the old version. Then, filter a small number of user requests to route to the new version of Pod application, and continue to observe whether it can run stably in the desired way. After confirming that there is no problem, continue to complete the rolling update of the remaining Pod resources, otherwise roll back the update operation immediately. This is the so-called Canary release.
# Update the version of deployment and configure to suspend deployment [root@k8s-master01 ~]# kubectl set image deploy pc-deployment nginx=nginx:1.17.4 -n dev && kubectl rollout pause deployment pc-deployment -n dev deployment.apps/pc-deployment image updated deployment.apps/pc-deployment paused #Observe update status [root@k8s-master01 ~]# kubectl rollout status deploy pc-deployment -n dev Waiting for deployment "pc-deployment" rollout to finish: 2 out of 4 new replicas have been updated... # Monitoring the update process, we can see that a new resource has been added, but an old resource has not been deleted according to the expected state, because the pause command is used [root@k8s-master01 ~]# kubectl get rs -n dev -o wide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES pc-deployment-5d89bdfbf9 3 3 3 19m nginx nginx:1.17.1 pc-deployment-675d469f8b 0 0 0 14m nginx nginx:1.17.2 pc-deployment-6c9f56fcfb 2 2 2 3m16s nginx nginx:1.17.4 [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-deployment-5d89bdfbf9-rj8sq 1/1 Running 0 7m33s pc-deployment-5d89bdfbf9-ttwgg 1/1 Running 0 7m35s pc-deployment-5d89bdfbf9-v4wvc 1/1 Running 0 7m34s pc-deployment-6c9f56fcfb-996rt 1/1 Running 0 3m31s pc-deployment-6c9f56fcfb-j2gtj 1/1 Running 0 3m31s # Make sure that the updated pod is OK and continue to update [root@k8s-master01 ~]# kubectl rollout resume deploy pc-deployment -n dev deployment.apps/pc-deployment resumed # View the last update [root@k8s-master01 ~]# kubectl get rs -n dev -o wide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES pc-deployment-5d89bdfbf9 0 0 0 21m nginx nginx:1.17.1 pc-deployment-675d469f8b 0 0 0 16m nginx nginx:1.17.2 pc-deployment-6c9f56fcfb 4 4 4 5m11s nginx nginx:1.17.4 [root@k8s-master01 ~]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE pc-deployment-6c9f56fcfb-7bfwh 1/1 Running 0 37s pc-deployment-6c9f56fcfb-996rt 1/1 Running 0 5m27s pc-deployment-6c9f56fcfb-j2gtj 1/1 Running 0 5m27s pc-deployment-6c9f56fcfb-rf84v 1/1 Running 0 37s
Delete Deployment
# When the deployment is deleted, the rs and pod under it will also be deleted [root@k8s-master01 ~]# kubectl delete -f pc-deployment.yaml deployment.apps "pc-deployment" deleted
6.4 Horizontal Pod Autoscaler(HPA)
In the previous course, we can manually execute kubectl scale command to realize pod capacity expansion or reduction, but this obviously does not meet the positioning goal of kubernetes - automation and intelligence. Kubernetes expects to realize the automatic adjustment of the number of pods by monitoring the use of pods, so a controller such as Horizontal Pod Autoscaler (HPA) is produced.
HPA can obtain the utilization rate of each Pod, then compare it with the indicators defined in HPA, calculate the specific value to be scaled, and finally adjust the number of pods. In fact, like the previous Deployment, HPA also belongs to a Kubernetes resource object. It tracks and analyzes the load changes of all target pods controlled by RC to determine whether it is necessary to adjust the number of copies of the target Pod. This is the implementation principle of HPA.
[external chain image transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-xws9zljf-163354912843) (kubenetes. Assets / image-20200608155858271. PNG)]
Next, let's do an experiment
1. Install metrics server
Metrics server can be used to collect resource usage in a cluster
# Install git [root@k8s-master01 ~]# yum install git -y # Get the metrics server and pay attention to the version used [root@k8s-master01 ~]# git clone -b v0.3.6 https://github.com/kubernetes-incubator/metrics-server # When modifying the deployment, note that the image and initialization parameters are modified [root@k8s-master01 ~]# cd /root/metrics-server/deploy/1.8+/ [root@k8s-master01 1.8+]# vim metrics-server-deployment.yaml Add the following options as shown hostNetwork: true image: registry.cn-hangzhou.aliyuncs.com/google_containers/metrics-server-amd64:v0.3.6 args: - --kubelet-insecure-tls - --kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-nbwvciqd-163354912844) (kubenetes. Assets / image-20200608163326496. PNG)]
# Installing metrics server [root@k8s-master01 1.8+]# kubectl apply -f ./ # Check the operation of pod [root@k8s-master01 1.8+]# kubectl get pod -n kube-system metrics-server-6b976979db-2xwbj 1/1 Running 0 90s # Use kubectl top node to view resource usage [root@k8s-master01 1.8+]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8s-master01 289m 14% 1582Mi 54% k8s-node01 81m 4% 1195Mi 40% k8s-node02 72m 3% 1211Mi 41% [root@k8s-master01 1.8+]# kubectl top pod -n kube-system NAME CPU(cores) MEMORY(bytes) coredns-6955765f44-7ptsb 3m 9Mi coredns-6955765f44-vcwr5 3m 8Mi etcd-master 14m 145Mi ... # At this point, the metrics server installation is complete
2 prepare deployment and servie
Create a pc-hpa-pod.yaml file as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: dev
spec:
strategy: # strategy
type: RollingUpdate # Rolling update strategy
replicas: 1
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
resources: # Resource quota
limits: # Limit resources (upper limit)
cpu: "1" # CPU limit, in core s
requests: # Requested resources (lower limit)
cpu: "100m" # CPU limit, in core s
# Create service [root@k8s-master01 1.8+]# kubectl expose deployment nginx --type=NodePort --port=80 -n dev
# see [root@k8s-master01 1.8+]# kubectl get deployment,pod,svc -n dev NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/nginx 1/1 1 1 47s NAME READY STATUS RESTARTS AGE pod/nginx-7df9756ccc-bh8dr 1/1 Running 0 47s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/nginx NodePort 10.101.18.29 <none> 80:31830/TCP 35s
3 deploy HPA
Create a pc-hpa.yaml file as follows:
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: pc-hpa
namespace: dev
spec:
minReplicas: 1 #Minimum pod quantity
maxReplicas: 10 #Maximum number of pod s
targetCPUUtilizationPercentage: 3 # CPU utilization index
scaleTargetRef: # Specify the nginx information to control
apiVersion: apps/v1
kind: Deployment
name: nginx
# Create hpa
[root@k8s-master01 1.8+]# kubectl create -f pc-hpa.yaml
horizontalpodautoscaler.autoscaling/pc-hpa created
# View hpa
[root@k8s-master01 1.8+]# kubectl get hpa -n dev
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
pc-hpa Deployment/nginx 0%/3% 1 10 1 62s
4 test
Use the pressure measurement tool to measure the service address 192.168.5.4:31830, and then check the changes of hpa and pod through the console
hpa change
[root@k8s-master01 ~]# kubectl get hpa -n dev -w NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE pc-hpa Deployment/nginx 0%/3% 1 10 1 4m11s pc-hpa Deployment/nginx 0%/3% 1 10 1 5m19s pc-hpa Deployment/nginx 22%/3% 1 10 1 6m50s pc-hpa Deployment/nginx 22%/3% 1 10 4 7m5s pc-hpa Deployment/nginx 22%/3% 1 10 8 7m21s pc-hpa Deployment/nginx 6%/3% 1 10 8 7m51s pc-hpa Deployment/nginx 0%/3% 1 10 8 9m6s pc-hpa Deployment/nginx 0%/3% 1 10 8 13m pc-hpa Deployment/nginx 0%/3% 1 10 1 14m
deployment change
[root@k8s-master01 ~]# kubectl get deployment -n dev -w NAME READY UP-TO-DATE AVAILABLE AGE nginx 1/1 1 1 11m nginx 1/4 1 1 13m nginx 1/4 1 1 13m nginx 1/4 1 1 13m nginx 1/4 4 1 13m nginx 1/8 4 1 14m nginx 1/8 4 1 14m nginx 1/8 4 1 14m nginx 1/8 8 1 14m nginx 2/8 8 2 14m nginx 3/8 8 3 14m nginx 4/8 8 4 14m nginx 5/8 8 5 14m nginx 6/8 8 6 14m nginx 7/8 8 7 14m nginx 8/8 8 8 15m nginx 8/1 8 8 20m nginx 8/1 8 8 20m nginx 1/1 1 1 20m
pod change
[root@k8s-master01 ~]# kubectl get pods -n dev -w NAME READY STATUS RESTARTS AGE nginx-7df9756ccc-bh8dr 1/1 Running 0 11m nginx-7df9756ccc-cpgrv 0/1 Pending 0 0s nginx-7df9756ccc-8zhwk 0/1 Pending 0 0s nginx-7df9756ccc-rr9bn 0/1 Pending 0 0s nginx-7df9756ccc-cpgrv 0/1 ContainerCreating 0 0s nginx-7df9756ccc-8zhwk 0/1 ContainerCreating 0 0s nginx-7df9756ccc-rr9bn 0/1 ContainerCreating 0 0s nginx-7df9756ccc-m9gsj 0/1 Pending 0 0s nginx-7df9756ccc-g56qb 0/1 Pending 0 0s nginx-7df9756ccc-sl9c6 0/1 Pending 0 0s nginx-7df9756ccc-fgst7 0/1 Pending 0 0s nginx-7df9756ccc-g56qb 0/1 ContainerCreating 0 0s nginx-7df9756ccc-m9gsj 0/1 ContainerCreating 0 0s nginx-7df9756ccc-sl9c6 0/1 ContainerCreating 0 0s nginx-7df9756ccc-fgst7 0/1 ContainerCreating 0 0s nginx-7df9756ccc-8zhwk 1/1 Running 0 19s nginx-7df9756ccc-rr9bn 1/1 Running 0 30s nginx-7df9756ccc-m9gsj 1/1 Running 0 21s nginx-7df9756ccc-cpgrv 1/1 Running 0 47s nginx-7df9756ccc-sl9c6 1/1 Running 0 33s nginx-7df9756ccc-g56qb 1/1 Running 0 48s nginx-7df9756ccc-fgst7 1/1 Running 0 66s nginx-7df9756ccc-fgst7 1/1 Terminating 0 6m50s nginx-7df9756ccc-8zhwk 1/1 Terminating 0 7m5s nginx-7df9756ccc-cpgrv 1/1 Terminating 0 7m5s nginx-7df9756ccc-g56qb 1/1 Terminating 0 6m50s nginx-7df9756ccc-rr9bn 1/1 Terminating 0 7m5s nginx-7df9756ccc-m9gsj 1/1 Terminating 0 6m50s nginx-7df9756ccc-sl9c6 1/1 Terminating 0 6m50s
6.5 DaemonSet(DS)
A DaemonSet type controller can ensure that a replica runs on each (or specified) node in the cluster. It is generally applicable to log collection, node monitoring and other scenarios. That is to say, if the functions provided by a Pod are node level (each node requires and only needs one), then this kind of Pod is suitable for creating with a controller of type DaemonSet.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-3ccpiygk-163354912844) (kubenetes. Assets / image-20200612010223537. PNG)]
Features of DaemonSet controller:
- Whenever a node is added to the cluster, the specified Pod copy is also added to the node
- When a node is removed from the cluster, the Pod is garbage collected
Let's take a look at the resource manifest file of DaemonSet
apiVersion: apps/v1 # Version number
kind: DaemonSet # type
metadata: # metadata
name: # rs name
namespace: # Namespace
labels: #label
controller: daemonset
spec: # Detailed description
revisionHistoryLimit: 3 # Keep historical version
updateStrategy: # Update strategy
type: RollingUpdate # Rolling update strategy
rollingUpdate: # Rolling update
maxUnavailable: 1 # The maximum value of the Pod in the maximum unavailable state, which can be either a percentage or an integer
selector: # Selector, which specifies which pod s the controller manages
matchLabels: # Labels matching rule
app: nginx-pod
matchExpressions: # Expressions matching rule
- {key: app, operator: In, values: [nginx-pod]}
template: # Template. When the number of copies is insufficient, a pod copy will be created according to the following template
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
Create pc-daemon.yaml as follows:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: pc-daemonset
namespace: dev
spec:
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
# Create daemonset [root@k8s-master01 ~]# kubectl create -f pc-daemonset.yaml daemonset.apps/pc-daemonset created # View daemonset [root@k8s-master01 ~]# kubectl get ds -n dev -o wide NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES pc-daemonset 2 2 2 2 2 24s nginx nginx:1.17.1 # Check the pod and find that a pod is running on each Node [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE pc-daemonset-9bck8 1/1 Running 0 37s 10.244.1.43 node1 pc-daemonset-k224w 1/1 Running 0 37s 10.244.2.74 node2 # Delete daemonset [root@k8s-master01 ~]# kubectl delete -f pc-daemonset.yaml daemonset.apps "pc-daemonset" deleted
6.6 Job
Job, which is mainly used for * * batch processing (processing a specified number of tasks at a time) short one-time (each task ends after running only once) * * tasks. Job features are as follows:
- When the execution of the pod created by the Job is completed successfully, the Job will record the number of successfully completed pods
- When the number of successfully completed pod s reaches the specified number, the Job will complete execution
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (IMG sukotpzz-163354912844) (kubenetes. Assets / image-20200618213054113. PNG)]
Job resource manifest file:
apiVersion: batch/v1 # Version number
kind: Job # type
metadata: # metadata
name: # rs name
namespace: # Namespace
labels: #label
controller: job
spec: # Detailed description
completions: 1 # Specifies the number of times a job needs to run Pods successfully. Default: 1
parallelism: 1 # Specifies the number of Pods that a job should run concurrently at any one time. Default: 1
activeDeadlineSeconds: 30 # Specify the time period within which the job can run. If it is not over, the system will try to terminate it.
backoffLimit: 6 # Specifies the number of retries after a job fails. The default is 6
manualSelector: true # Can I use the selector to select pod? The default is false
selector: # Selector, which specifies which pod s the controller manages
matchLabels: # Labels matching rule
app: counter-pod
matchExpressions: # Expressions matching rule
- {key: app, operator: In, values: [counter-pod]}
template: # Template. When the number of copies is insufficient, a pod copy will be created according to the following template
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never # The restart policy can only be set to Never or OnFailure
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 2;done"]
Description of restart policy settings:
If specified as OnFailure，be job Will be pod Restart the container on failure instead of creating pod，failed Times unchanged
If specified as Never，be job Will be pod Create a new on failure pod，And fault pod It won't disappear or restart, failed Times plus 1
If specified as Always If so, it means always restarting, which means job The task will be executed repeatedly. Of course, it is wrong, so it cannot be set to Always
Create pc-job.yaml as follows:
apiVersion: batch/v1
kind: Job
metadata:
name: pc-job
namespace: dev
spec:
manualSelector: true
selector:
matchLabels:
app: counter-pod
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 3;done"]
# Create job [root@k8s-master01 ~]# kubectl create -f pc-job.yaml job.batch/pc-job created # View job [root@k8s-master01 ~]# kubectl get job -n dev -o wide -w NAME COMPLETIONS DURATION AGE CONTAINERS IMAGES SELECTOR pc-job 0/1 21s 21s counter busybox:1.30 app=counter-pod pc-job 1/1 31s 79s counter busybox:1.30 app=counter-pod # By observing the status of the pod, you can see that the pod will change to the Completed status after running the task [root@k8s-master01 ~]# kubectl get pods -n dev -w NAME READY STATUS RESTARTS AGE pc-job-rxg96 1/1 Running 0 29s pc-job-rxg96 0/1 Completed 0 33s # Next, adjust the total number and parallel number of pod runs, that is, set the following two options under spec # completions: 6 # The number of times that the specified job needs to run Pods successfully is 6 # parallelism: 3 # Specify that the number of job s running Pods concurrently is 3 # Then re run the job and observe the effect. At this time, it will be found that the job will run 3 pods at a time, and a total of 6 pods have been executed [root@k8s-master01 ~]# kubectl get pods -n dev -w NAME READY STATUS RESTARTS AGE pc-job-684ft 1/1 Running 0 5s pc-job-jhj49 1/1 Running 0 5s pc-job-pfcvh 1/1 Running 0 5s pc-job-684ft 0/1 Completed 0 11s pc-job-v7rhr 0/1 Pending 0 0s pc-job-v7rhr 0/1 Pending 0 0s pc-job-v7rhr 0/1 ContainerCreating 0 0s pc-job-jhj49 0/1 Completed 0 11s pc-job-fhwf7 0/1 Pending 0 0s pc-job-fhwf7 0/1 Pending 0 0s pc-job-pfcvh 0/1 Completed 0 11s pc-job-5vg2j 0/1 Pending 0 0s pc-job-fhwf7 0/1 ContainerCreating 0 0s pc-job-5vg2j 0/1 Pending 0 0s pc-job-5vg2j 0/1 ContainerCreating 0 0s pc-job-fhwf7 1/1 Running 0 2s pc-job-v7rhr 1/1 Running 0 2s pc-job-5vg2j 1/1 Running 0 3s pc-job-fhwf7 0/1 Completed 0 12s pc-job-v7rhr 0/1 Completed 0 12s pc-job-5vg2j 0/1 Completed 0 12s # Delete job [root@k8s-master01 ~]# kubectl delete -f pc-job.yaml job.batch "pc-job" deleted
6.7 CronJob(CJ)
The CronJob controller takes the job controller resource as its control object and uses it to manage the pod resource object. The job tasks defined by the job controller will be executed immediately after its controller resource is created, but CronJob can control its running time point and repeated running mode in a way similar to the periodic task job plan of the Linux operating system. That is, CronJob can run job tasks at a specific point in time (repeatedly).
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-du2fqso1-163354912845) (kubenetes. Assets / image-20200618213149531. PNG)]
Resource manifest file for CronJob:
apiVersion: batch/v1beta1 # Version number
kind: CronJob # type
metadata: # metadata
name: # rs name
namespace: # Namespace
labels: #label
controller: cronjob
spec: # Detailed description
schedule: # cron format job scheduling run time point, which is used to control when tasks are executed
concurrencyPolicy: # Concurrent execution policy is used to define whether and how to run the next job when the previous job run has not been completed
failedJobHistoryLimit: # The number of history records reserved for failed task execution. The default is 1
successfulJobHistoryLimit: # The number of history records reserved for successful task execution. The default is 3
startingDeadlineSeconds: # Timeout length of start job error
jobTemplate: # The job controller template is used to generate a job object for the cronjob controller; The following is actually the definition of job
metadata:
spec:
completions: 1
parallelism: 1
activeDeadlineSeconds: 30
backoffLimit: 6
manualSelector: true
selector:
matchLabels:
app: counter-pod
matchExpressions: rule
- {key: app, operator: In, values: [counter-pod]}
template:
metadata:
labels:
app: counter-pod
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 20;done"]
Several key options to explain:
schedule: cron Expression that specifies the execution time of the task
*/1 * * * *
<minute> <hour> <day> <month> <week>
Minute values range from 0 to 59.
Hour values range from 0 to 23.
Daily values range from 1 to 31.
Monthly values range from 1 to 12.
Week values range from 0 to 6, 0 On behalf of Sunday
Multiple times can be separated by commas; The range can be given with hyphens;*Can be used as wildcards; /Represents each...
concurrencyPolicy:
Allow: allow Jobs Concurrent operation(default)
Forbid: Concurrent running is prohibited. If the previous run has not been completed, the next run will be skipped
Replace: Replace, cancel the currently running job and replace it with a new job
Create pc-cronjob.yaml as follows:
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: pc-cronjob
namespace: dev
labels:
controller: cronjob
spec:
schedule: "*/1 * * * *"
jobTemplate:
metadata:
spec:
template:
spec:
restartPolicy: Never
containers:
- name: counter
image: busybox:1.30
command: ["bin/sh","-c","for i in 9 8 7 6 5 4 3 2 1; do echo $i;sleep 3;done"]
# Create cronjob [root@k8s-master01 ~]# kubectl create -f pc-cronjob.yaml cronjob.batch/pc-cronjob created # View cronjob [root@k8s-master01 ~]# kubectl get cronjobs -n dev NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE pc-cronjob */1 * * * * False 0 <none> 6s # View job [root@k8s-master01 ~]# kubectl get jobs -n dev NAME COMPLETIONS DURATION AGE pc-cronjob-1592587800 1/1 28s 3m26s pc-cronjob-1592587860 1/1 28s 2m26s pc-cronjob-1592587920 1/1 28s 86s # View pod [root@k8s-master01 ~]# kubectl get pods -n dev pc-cronjob-1592587800-x4tsm 0/1 Completed 0 2m24s pc-cronjob-1592587860-r5gv4 0/1 Completed 0 84s pc-cronjob-1592587920-9dxxq 1/1 Running 0 24s # Delete cronjob [root@k8s-master01 ~]# kubectl delete -f pc-cronjob.yaml cronjob.batch "pc-cronjob" deleted
7. Detailed explanation
7.1 Service introduction
In kubernetes, pod is the carrier of application program. We can access the application program through the ip of pod, but the ip address of pod is not fixed, which means that it is not convenient to directly use the ip of pod to access the service.
To solve this problem, kubernetes provides service resources. Service aggregates multiple pods that provide the same service and provides a unified entry address. You can access the following pod services by accessing the service entry address.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-k2qmc40r-163354912845) (kubenetes. Assets / image-20200408194716912-1626783758946. PNG)]
In many cases, service is just a concept. What really works is the Kube proxy service process. A Kube proxy service process is running on each Node. When creating a service, it will write the information of the created service to etcd through API server, and Kube proxy will find the changes of this service based on the listening mechanism, and then it will convert the latest service information into the corresponding access rules.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-6ihzozt4-163354912845) (kubenetes. Assets / image-20200509121254425. PNG)]
# 10.97.97.97:80 is the access portal provided by the service # When accessing this portal, you can find that there are three pod services waiting to be called, # Kube proxy distributes requests to one of the pod s based on rr (polling) policy # This rule will be generated on all nodes in the cluster at the same time, so it can be accessed on any node. [root@node1 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.97.97.97:80 rr -> 10.244.1.39:80 Masq 1 0 0 -> 10.244.1.40:80 Masq 1 0 0 -> 10.244.2.33:80 Masq 1 0 0
Kube proxy currently supports three working modes:
userspace mode
In userspace mode, Kube proxy will create a listening port for each Service. The request to Cluster IP will be redirected to the listening port of Kube proxy by Iptables rules. Kube proxy will select a Pod providing services according to LB algorithm and establish a link with it to forward the request to Pod. In this mode, Kube proxy acts as a four layer equalizer. Because Kube proxy runs in userspace, the data copy between kernel and user space will be increased during forwarding. Although it is relatively stable, it is inefficient.
[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-rfpgkmjp-163354912846) (kubenetes. Assets / image-202005091514280. PNG)]
iptables mode
In iptables mode, Kube proxy creates corresponding iptables rules for each Pod on the back end of the service, and directly redirects the request to the Cluster IP to a Pod IP. In this mode, Kube proxy does not assume the role of four-layer equalizer, but is only responsible for creating iptables rules. The advantage of this mode is that it is more efficient than the userspace mode, but it cannot provide a flexible LB strategy, and it cannot retry when the back-end Pod is unavailable.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ivvbjiaik-163354912846) (kubenetes. Assets / image-20200509152947714. PNG)]
ipvs mode
ipvs mode is similar to iptables. Kube proxy monitors Pod changes and creates corresponding ipvs rules. ipvs is more efficient than iptables. In addition, ipvs supports more LB algorithms.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-yjfhopug-163354912846) (kubenetes. Assets / image-20200509153731363. PNG)]
# ipvs kernel module must be installed in this mode, otherwise it will be degraded to iptables # Turn on ipvs [root@k8s-master01 ~]# kubectl edit cm kube-proxy -n kube-system # Modify mode: "ipvs" [root@k8s-master01 ~]# kubectl delete pod -l k8s-app=kube-proxy -n kube-system [root@node1 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.97.97.97:80 rr -> 10.244.1.39:80 Masq 1 0 0 -> 10.244.1.40:80 Masq 1 0 0 -> 10.244.2.33:80 Masq 1 0 0
7.2 Service type
Resource manifest file for Service:
kind: Service # Resource type
apiVersion: v1 # Resource version
metadata: # metadata
name: service # Resource name
namespace: dev # Namespace
spec: # describe
selector: # Tag selector, which is used to determine which pod s are represented by the current service
app: nginx
type: # Service type, specifying the access method of the service
clusterIP: # ip address of virtual service
sessionAffinity: # session affinity supports ClientIP and None options
ports: # port information
- protocol: TCP
port: 3017 # service port
targetPort: 5003 # pod port
nodePort: 31122 # Host port
- ClusterIP: the default value. It is the virtual IP automatically assigned by Kubernetes system and can only be accessed inside the cluster
- NodePort: expose the Service to the outside through the port on the specified Node. Through this method, you can access the Service outside the cluster
- LoadBalancer: use the external load balancer to complete the load distribution to the service. Note that this mode needs the support of external cloud environment
- ExternalName: introduce services outside the cluster into the cluster and use them directly
7.3 Service usage
7.3.1 preparation of experimental environment
Before using the service, first create three pods with Deployment. Note that the tag app = nginx pod should be set for the pod
Create deployment.yaml as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: pc-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
[root@k8s-master01 ~]# kubectl create -f deployment.yaml deployment.apps/pc-deployment created # View pod details [root@k8s-master01 ~]# kubectl get pods -n dev -o wide --show-labels NAME READY STATUS IP NODE LABELS pc-deployment-66cb59b984-8p84h 1/1 Running 10.244.1.39 node1 app=nginx-pod pc-deployment-66cb59b984-vx8vx 1/1 Running 10.244.2.33 node2 app=nginx-pod pc-deployment-66cb59b984-wnncx 1/1 Running 10.244.1.40 node1 app=nginx-pod # In order to facilitate the following tests, modify the index.html page of the next three nginx (the modified IP addresses of the three are inconsistent) # kubectl exec -it pc-deployment-66cb59b984-8p84h -n dev /bin/sh # echo "10.244.1.39" > /usr/share/nginx/html/index.html #After modification, access the test [root@k8s-master01 ~]# curl 10.244.1.39 10.244.1.39 [root@k8s-master01 ~]# curl 10.244.2.33 10.244.2.33 [root@k8s-master01 ~]# curl 10.244.1.40 10.244.1.40
7.3.2 ClusterIP type Service
Create the service-clusterip.yaml file
apiVersion: v1
kind: Service
metadata:
name: service-clusterip
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: 10.97.97.97 # The ip address of the service. If it is not written, it will be generated by default
type: ClusterIP
ports:
- port: 80 # Service port
targetPort: 80 # pod port
# Create service [root@k8s-master01 ~]# kubectl create -f service-clusterip.yaml service/service-clusterip created # View service [root@k8s-master01 ~]# kubectl get svc -n dev -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service-clusterip ClusterIP 10.97.97.97 <none> 80/TCP 13s app=nginx-pod # View service details # Here is a list of Endpoints, which is the service entry that the current service can load [root@k8s-master01 ~]# kubectl describe svc service-clusterip -n dev Name: service-clusterip Namespace: dev Labels: <none> Annotations: <none> Selector: app=nginx-pod Type: ClusterIP IP: 10.97.97.97 Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: 10.244.1.39:80,10.244.1.40:80,10.244.2.33:80 Session Affinity: None Events: <none> # View mapping rules for ipvs [root@k8s-master01 ~]# ipvsadm -Ln TCP 10.97.97.97:80 rr -> 10.244.1.39:80 Masq 1 0 0 -> 10.244.1.40:80 Masq 1 0 0 -> 10.244.2.33:80 Masq 1 0 0 # Visit 10.97.97.97:80 to observe the effect [root@k8s-master01 ~]# curl 10.97.97.97:80 10.244.2.33
Endpoint
Endpoint is a resource object in kubernetes. It is stored in etcd and used to record the access addresses of all pod s corresponding to a service. It is generated according to the selector description in the service configuration file.
A service consists of a group of pods, which are exposed through endpoints, which are the collection of endpoints that implement the actual service. In other words, the connection between service and pod is realized through endpoints.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-2dsdhj0b-163354912846) (kubenetes. Assets / image-2020050917069. PNG)]
Load distribution policy
Access to the Service is distributed to the backend Pod. At present, kubernetes provides two load distribution strategies:
-
If it is not defined, the Kube proxy policy is used by default, such as random and polling
-
Session persistence mode based on client address, that is, all requests from the same client will be forwarded to a fixed Pod
This mode enables you to add the sessionAffinity:ClientIP option to the spec
# View the mapping rules of ipvs [rr polling] [root@k8s-master01 ~]# ipvsadm -Ln TCP 10.97.97.97:80 rr -> 10.244.1.39:80 Masq 1 0 0 -> 10.244.1.40:80 Masq 1 0 0 -> 10.244.2.33:80 Masq 1 0 0 # Cyclic access test [root@k8s-master01 ~]# while true;do curl 10.97.97.97:80; sleep 5; done; 10.244.1.40 10.244.1.39 10.244.2.33 10.244.1.40 10.244.1.39 10.244.2.33 # Modify distribution policy - sessionAffinity:ClientIP # View ipvs rules [persistent stands for persistent] [root@k8s-master01 ~]# ipvsadm -Ln TCP 10.97.97.97:80 rr persistent 10800 -> 10.244.1.39:80 Masq 1 0 0 -> 10.244.1.40:80 Masq 1 0 0 -> 10.244.2.33:80 Masq 1 0 0 # Cyclic access test [root@k8s-master01 ~]# while true;do curl 10.97.97.97; sleep 5; done; 10.244.2.33 10.244.2.33 10.244.2.33 # Delete service [root@k8s-master01 ~]# kubectl delete -f service-clusterip.yaml service "service-clusterip" deleted
7.3.3 Service of headliner type
In some scenarios, developers may not want to use the load balancing function provided by the service, but want to control the load balancing policy. In this case, kubernetes provides a headlines service, which does not allocate Cluster IP. If they want to access the service, they can only query through the domain name of the service.
Create service-headline.yaml
apiVersion: v1
kind: Service
metadata:
name: service-headliness
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: None # Set clusterIP to None to create the headline service
type: ClusterIP
ports:
- port: 80
targetPort: 80
# Create service [root@k8s-master01 ~]# kubectl create -f service-headliness.yaml service/service-headliness created # Get the service and find that CLUSTER-IP is not allocated [root@k8s-master01 ~]# kubectl get svc service-headliness -n dev -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR service-headliness ClusterIP None <none> 80/TCP 11s app=nginx-pod # View service details [root@k8s-master01 ~]# kubectl describe svc service-headliness -n dev Name: service-headliness Namespace: dev Labels: <none> Annotations: <none> Selector: app=nginx-pod Type: ClusterIP IP: None Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: 10.244.1.39:80,10.244.1.40:80,10.244.2.33:80 Session Affinity: None Events: <none> # View the resolution of the domain name [root@k8s-master01 ~]# kubectl exec -it pc-deployment-66cb59b984-8p84h -n dev /bin/sh / # cat /etc/resolv.conf nameserver 10.96.0.10 search dev.svc.cluster.local svc.cluster.local cluster.local [root@k8s-master01 ~]# dig @10.96.0.10 service-headliness.dev.svc.cluster.local service-headliness.dev.svc.cluster.local. 30 IN A 10.244.1.40 service-headliness.dev.svc.cluster.local. 30 IN A 10.244.1.39 service-headliness.dev.svc.cluster.local. 30 IN A 10.244.2.33
7.3.4 NodePort type Service
In the previous example, the ip address of the created service can only be accessed inside the cluster. If you want to expose the service to external use, you need to use another type of service, called NodePort. The working principle of NodePort is to map the port of the service to a port of the Node, and then you can access the service through NodeIp:NodePort.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-pd1h9nqe-163354912847) (kubenetes. Assets / image-20200620175731338. PNG)]
Create service-nodeport.yaml
apiVersion: v1
kind: Service
metadata:
name: service-nodeport
namespace: dev
spec:
selector:
app: nginx-pod
type: NodePort # service type
ports:
- port: 80
nodePort: 30002 # Specify the port of the bound node (the default value range is 30000-32767). If it is not specified, it will be allocated by default
targetPort: 80
# Create service [root@k8s-master01 ~]# kubectl create -f service-nodeport.yaml service/service-nodeport created # View service [root@k8s-master01 ~]# kubectl get svc -n dev -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) SELECTOR service-nodeport NodePort 10.105.64.191 <none> 80:30002/TCP app=nginx-pod # Next, you can access port 30002 of any nodeip in the cluster through the browser of the host computer to access the pod
7.3.5 LoadBalancer type Service
LoadBalancer is very similar to NodePort. The purpose is to expose a port to the outside. The difference is that LoadBalancer will make a load balancing device outside the cluster. If the device needs the support of the external environment, the requests sent by external services to the device will be loaded by the device and forwarded to the cluster.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-yrh63hzb-163354912847) (kubenetes. Assets / image-20200510103945494. PNG)]
7.3.6 Service of externalname type
The service of externalname type is used to introduce services outside the cluster. It specifies the address of an external service through the externalname attribute, and then accesses the service inside the cluster to access the external service.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ngzxaasp-163354912847) (kubenetes. Assets / image-2020051013311209. PNG)]
apiVersion: v1 kind: Service metadata: name: service-externalname namespace: dev spec: type: ExternalName # service type externalName: www.baidu.com #You can also change it to an ip address
# Create service [root@k8s-master01 ~]# kubectl create -f service-externalname.yaml service/service-externalname created # Domain name resolution [root@k8s-master01 ~]# dig @10.96.0.10 service-externalname.dev.svc.cluster.local service-externalname.dev.svc.cluster.local. 30 IN CNAME www.baidu.com. www.baidu.com. 30 IN CNAME www.a.shifen.com. www.a.shifen.com. 30 IN A 39.156.66.18 www.a.shifen.com. 30 IN A 39.156.66.14
7.4 introduction to ingress
As mentioned in the previous course, there are two main ways for Service to expose services outside the cluster: NotePort and LoadBalancer, but both of them have certain disadvantages:
- The disadvantage of NodePort mode is that it will occupy many ports of cluster machines. This disadvantage becomes more and more obvious when there are more cluster services
- The disadvantage of LB mode is that each service needs an lb, which is wasteful and troublesome, and requires the support of devices other than kubernetes
Based on this situation, kubernetes provides an ingress resource object. Ingress only needs a NodePort or an LB to meet the requirement of exposing multiple services. The working mechanism is roughly shown in the figure below:
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-itjn4ih-163354912848) (kubenetes. Assets / image-20200623092808049. PNG)]
In fact, ingress is equivalent to a 7-layer load balancer. It is an abstraction of kubernetes' reverse proxy. Its working principle is similar to Nginx. It can be understood as establishing many mapping rules in ingress. Ingress Controller monitors these configuration rules and converts them into Nginx's reverse proxy configuration, and then provides services to the outside. Here are two core concepts:
- ingress: an object in kubernetes that defines rules for how requests are forwarded to service s
- ingress controller: a program that implements reverse proxy and load balancing. It parses the rules defined by ingress and forwards requests according to the configured rules. There are many ways to implement it, such as Nginx, Contour, Haproxy and so on
The working principle of Ingres (taking Nginx as an example) is as follows:
- The user writes an Ingress rule to specify which domain name corresponds to which Service in the kubernetes cluster
- The Ingress controller dynamically senses the changes of Ingress service rules, and then generates a corresponding Nginx reverse proxy configuration
- The Ingress controller will write the generated Nginx configuration to a running Nginx service and update it dynamically
- So far, what is really working is an Nginx, which is internally configured with user-defined request forwarding rules
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-xeuj8cq3-163354912848) (kubenetes. Assets / image-2020051612704764. PNG)]
7.5 use of ingress
7.5.1 environmental preparation
Setting up the ingress environment
# create folder [root@k8s-master01 ~]# mkdir ingress-controller [root@k8s-master01 ~]# cd ingress-controller/ # Obtain ingress nginx. Version 0.30 is used in this case [root@k8s-master01 ingress-controller]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml [root@k8s-master01 ingress-controller]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml # Modify the repository in the mandatory.yaml file # Modify quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0 # quay-mirror.qiniu.com/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0 # Create ingress nginx [root@k8s-master01 ingress-controller]# kubectl apply -f ./ # View ingress nginx [root@k8s-master01 ingress-controller]# kubectl get pod -n ingress-nginx NAME READY STATUS RESTARTS AGE pod/nginx-ingress-controller-fbf967dd5-4qpbp 1/1 Running 0 12h # View service [root@k8s-master01 ingress-controller]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx NodePort 10.98.75.163 <none> 80:32240/TCP,443:31335/TCP 11h
Prepare service and pod
For the convenience of later experiments, create the model shown in the figure below
[the external link image transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the image and upload it directly (img-reghpdde-163354912849) (kubenetes. Assets / image-202005161241998. PNG)]
Create tomcat-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat-pod
template:
metadata:
labels:
app: tomcat-pod
spec:
containers:
- name: tomcat
image: tomcat:8.5-jre10-slim
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: dev
spec:
selector:
app: nginx-pod
clusterIP: None
type: ClusterIP
ports:
- port: 80
targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
namespace: dev
spec:
selector:
app: tomcat-pod
clusterIP: None
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
# establish [root@k8s-master01 ~]# kubectl create -f tomcat-nginx.yaml # see [root@k8s-master01 ~]# kubectl get svc -n dev NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE nginx-service ClusterIP None <none> 80/TCP 48s tomcat-service ClusterIP None <none> 8080/TCP 48s
7.5.2 Http agent
Create ingress-http.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-http
namespace: dev
spec:
rules:
- host: nginx.itheima.com
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
- host: tomcat.itheima.com
http:
paths:
- path: /
backend:
serviceName: tomcat-service
servicePort: 8080
# establish [root@k8s-master01 ~]# kubectl create -f ingress-http.yaml ingress.extensions/ingress-http created # see [root@k8s-master01 ~]# kubectl get ing ingress-http -n dev NAME HOSTS ADDRESS PORTS AGE ingress-http nginx.itheima.com,tomcat.itheima.com 80 22s # View details [root@k8s-master01 ~]# kubectl describe ing ingress-http -n dev ... Rules: Host Path Backends ---- ---- -------- nginx.itheima.com / nginx-service:80 (10.244.1.96:80,10.244.1.97:80,10.244.2.112:80) tomcat.itheima.com / tomcat-service:8080(10.244.1.94:8080,10.244.1.95:8080,10.244.2.111:8080) ... # Next, configure the host file on the local computer and resolve the above two domain names to 192.168.109.100(master) # Then, you can visit tomcat.itheima.com:32240 and nginx.itheima.com:32240 to see the effect
7.5.3 Https agent
Create certificate
# Generate certificate openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/C=CN/ST=BJ/L=BJ/O=nginx/CN=itheima.com" # Create key kubectl create secret tls tls-secret --key tls.key --cert tls.crt
Create ingress-https.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-https
namespace: dev
spec:
tls:
- hosts:
- nginx.itheima.com
- tomcat.itheima.com
secretName: tls-secret # Specify secret key
rules:
- host: nginx.itheima.com
http:
paths:
- path: /
backend:
serviceName: nginx-service
servicePort: 80
- host: tomcat.itheima.com
http:
paths:
- path: /
backend:
serviceName: tomcat-service
servicePort: 8080
# establish [root@k8s-master01 ~]# kubectl create -f ingress-https.yaml ingress.extensions/ingress-https created # see [root@k8s-master01 ~]# kubectl get ing ingress-https -n dev NAME HOSTS ADDRESS PORTS AGE ingress-https nginx.itheima.com,tomcat.itheima.com 10.104.184.38 80, 443 2m42s # View details [root@k8s-master01 ~]# kubectl describe ing ingress-https -n dev ... TLS: tls-secret terminates nginx.itheima.com,tomcat.itheima.com Rules: Host Path Backends ---- ---- -------- nginx.itheima.com / nginx-service:80 (10.244.1.97:80,10.244.1.98:80,10.244.2.119:80) tomcat.itheima.com / tomcat-service:8080(10.244.1.99:8080,10.244.2.117:8080,10.244.2.120:8080) ... # The following can be accessed through the browser https://nginx.itheima.com:31335 And https://tomcat.itheima.com:31335 Here to see
8. Data storage
As mentioned earlier, containers may have a short life cycle and are frequently created and destroyed. When the container is destroyed, the data saved in the container will also be cleared. This result is undesirable to users in some cases. In order to persist the container data, kubernetes introduces the concept of Volume.
Volume is a shared directory that can be accessed by multiple containers in a Pod. It is defined on the Pod and then mounted to a specific file directory by multiple containers in a Pod. kubernetes realizes data sharing and persistent storage between different containers in the same Pod through volume. The life container of a volume is not related to the life cycle of a single container in the Pod. When the container is terminated or restarted, the data in the volume will not be lost.
kubernetes Volume supports many types, including the following:
- Simple storage: EmptyDir, HostPath, NFS
- Advanced storage: PV, PVC
- Configuration storage: ConfigMap, Secret
8.1 basic storage
8.1.1 EmptyDir
EmptyDir is the most basic Volume type. An EmptyDir is an empty directory on the Host.
EmptyDir is created when the Pod is allocated to the Node. Its initial content is empty, and there is no need to specify the corresponding directory file on the host, because kubernetes will automatically allocate a directory. When the Pod is destroyed, the data in EmptyDir will also be permanently deleted. The purpose of EmptyDir is as follows:
- Temporary space, such as a temporary directory required for some applications to run, and does not need to be permanently reserved
- A directory where one container needs to get data from another container (multi container shared directory)
Next, use EmptyDir through a case of file sharing between containers.
Prepare two containers nginx and busybox in a Pod, and then declare that a Volume is hung in the directories of the two containers respectively. Then the nginx container is responsible for writing logs to the Volume, and busybox reads the log contents to the console through commands.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-8m1xvs6b-163354912849) (kubenetes. Assets / image-20200413174713773. PNG)]
Create a volume-emptydir.yaml
apiVersion: v1
kind: Pod
metadata:
name: volume-emptydir
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts: # Hang logs volume in the nginx container, and the corresponding directory is / var/log/nginx
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"] # Initial command to dynamically read the contents of the specified file
volumeMounts: # Hang logs volume in the busybox container, and the corresponding directory is / logs
- name: logs-volume
mountPath: /logs
volumes: # Declare volume, the name is logs volume, and the type is emptyDir
- name: logs-volume
emptyDir: {}
# Create Pod [root@k8s-master01 ~]# kubectl create -f volume-emptydir.yaml pod/volume-emptydir created # View pod [root@k8s-master01 ~]# kubectl get pods volume-emptydir -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... volume-emptydir 2/2 Running 0 97s 10.42.2.9 node1 ...... # Accessing nginx through podIp [root@k8s-master01 ~]# curl 10.42.2.9 ...... # View the standard output of the specified container through the kubectl logs command [root@k8s-master01 ~]# kubectl logs -f volume-emptydir -n dev -c busybox 10.42.1.0 - - [27/Jun/2021:15:08:54 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"
8.1.2 HostPath
As mentioned in the last lesson, the data in EmptyDir will not be persisted. It will be destroyed with the end of Pod. If you want to simply persist the data to the host, you can choose HostPath.
HostPath is to hang an actual directory in the Node host in the Pod for use by the container. This design can ensure that the Pod is destroyed, but the data basis can exist on the Node host.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-cknw64ro-163354912849) (kubenetes. Assets / image-20200413214031331. PNG)]
Create a volume-hostpath.yaml:
apiVersion: v1
kind: Pod
metadata:
name: volume-hostpath
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts:
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"]
volumeMounts:
- name: logs-volume
mountPath: /logs
volumes:
- name: logs-volume
hostPath:
path: /root/logs
type: DirectoryOrCreate # If the directory exists, it will be used. If it does not exist, it will be created first and then used
about type A description of the value of:
DirectoryOrCreate If the directory exists, it will be used. If it does not exist, it will be created first and then used
Directory Directory must exist
FileOrCreate If a file exists, it will be used. If it does not exist, it will be created first and then used
File File must exist
Socket unix Socket must exist
CharDevice Character device must exist
BlockDevice Block device must exist
# Create Pod [root@k8s-master01 ~]# kubectl create -f volume-hostpath.yaml pod/volume-hostpath created # View Pod [root@k8s-master01 ~]# kubectl get pods volume-hostpath -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE ...... pod-volume-hostpath 2/2 Running 0 16s 10.42.2.10 node1 ...... #Access nginx [root@k8s-master01 ~]# curl 10.42.2.10 # Next, you can view the stored files in the / root/logs directory of the host ### Note: the following operations need to run on the node where the Pod is located (node1 in the case) [root@node1 ~]# ls /root/logs/ access.log error.log # Similarly, if you create a file in this directory, you can see it in the container
8.1.3 NFS
HostPath can solve the problem of data persistence, but once the Node fails, the Pod will have problems if it is transferred to another Node. At this time, it is necessary to prepare a separate network storage system. NFS and CIFS are commonly used.
NFS is a network file storage system. You can build an NFS server, and then directly connect the storage in the Pod to the NFS system. In this way, no matter how the Pod is transferred on the Node, as long as the connection between the Node and NFS is OK, the data can be accessed successfully.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ksg8wqv2-163354912850) (kubenetes. Assets / image-20200413215133559. PNG)]
1) First, prepare the nfs server. For simplicity, the master node is directly used as the nfs server
# Installing nfs services on nfs [root@nfs ~]# yum install nfs-utils -y # Prepare a shared directory [root@nfs ~]# mkdir /root/data/nfs -pv # Expose the shared directory to all hosts in the 192.168.5.0/24 network segment with read and write permissions [root@nfs ~]# vim /etc/exports [root@nfs ~]# more /etc/exports /root/data/nfs 192.168.5.0/24(rw,no_root_squash) # Start nfs service [root@nfs ~]# systemctl restart nfs
2) Next, you need to install nfs on each node node so that the node node can drive nfs devices
# Install the nfs service on the node. Note that it does not need to be started [root@k8s-master01 ~]# yum install nfs-utils -y
3) Next, you can write the configuration file of pod and create volume-nfs.yaml
apiVersion: v1
kind: Pod
metadata:
name: volume-nfs
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
volumeMounts:
- name: logs-volume
mountPath: /var/log/nginx
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","tail -f /logs/access.log"]
volumeMounts:
- name: logs-volume
mountPath: /logs
volumes:
- name: logs-volume
nfs:
server: 192.168.5.6 #nfs server address
path: /root/data/nfs #Shared file path
4) Finally, run the pod and observe the results
# Create pod [root@k8s-master01 ~]# kubectl create -f volume-nfs.yaml pod/volume-nfs created # View pod [root@k8s-master01 ~]# kubectl get pods volume-nfs -n dev NAME READY STATUS RESTARTS AGE volume-nfs 2/2 Running 0 2m9s # Check the shared directory on the nfs server and find that there are already files [root@k8s-master01 ~]# ls /root/data/ access.log error.log
8.2 advanced storage
We have learned to use NFS to provide storage. At this time, users are required to build NFS systems and configure NFS in yaml. Since kubernetes supports many storage systems, it is obviously unrealistic for customers to master them all. In order to shield the details of the underlying storage implementation and facilitate users, kubernetes introduces PV and PVC resource objects.
PV (Persistent Volume) means Persistent Volume. It is an abstraction of the underlying shared storage. Generally, PV is created and configured by kubernetes administrator. It is related to the underlying specific shared storage technology, and connects with shared storage through plug-ins.
PVC (Persistent Volume Claim) is a persistent volume declaration, which means a user's declaration of storage requirements. In other words, PVC is actually a resource demand application sent by users to kubernetes system.
[external chain picture transfer fails, and the source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-olv4fd1p-163354912850) (kubenetes. Assets / image-20200514194111567. PNG)]
After using PV and PVC, the work can be further subdivided:
- Storage: Storage Engineer Maintenance
- PV: kubernetes administrator maintenance
- PVC: kubernetes user maintenance
8.2.1 PV
PV is the abstraction of storage resources. The following is the resource manifest file:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv2
spec:
nfs: # Storage type, corresponding to the underlying real storage
capacity: # Storage capacity. At present, only storage space settings are supported
storage: 2Gi
accessModes: # Access mode
storageClassName: # Storage category
persistentVolumeReclaimPolicy: # Recycling strategy
Description of key configuration parameters of PV:
-
Storage type
kubernetes supports multiple storage types, and the configuration of each storage type is different
-
Storage capacity
At present, only the storage space setting (storage=1Gi) is supported, but the configuration of IOPS, throughput and other indicators may be added in the future
-
Access modes
It is used to describe the user application's access rights to storage resources. The access rights include the following ways:
- ReadWriteOnce (RWO): read and write permission, but can only be mounted by a single node
- ReadOnlyMany (ROX): read-only permission, which can be mounted by multiple nodes
- ReadWriteMany (RWX): read-write permission, which can be mounted by multiple nodes
It should be noted that different underlying storage types may support different access modes
-
Persistent volumereclaimpolicy
When PV is no longer used, the treatment of it. Currently, three strategies are supported:
- Retain to retain data, which requires the administrator to manually clean up the data
- Recycle clears the data in the PV. The effect is equivalent to executing rm -rf /thevolume/*
- Delete: the back-end storage connected to the PV completes the deletion of volume. Of course, this is common in the storage services of cloud service providers
It should be noted that different underlying storage types may support different recycling strategies
-
Storage category
PV can specify a storage category through the storageClassName parameter
- A PV with a specific category can only be bound to a PVC that requests that category
- PV without a category can only be bound with PVC that does not request any category
-
status
In the life cycle of a PV, it may be in 4 different stages:
- Available: indicates the available status and has not been bound by any PVC
- Bound: indicates that PV has been bound by PVC
- Released: indicates that the PVC has been deleted, but the resource has not been redeclared by the cluster
- Failed: indicates that the automatic recovery of the PV failed
experiment
Use NFS as storage to demonstrate the use of PVS. Create three PVS corresponding to three exposed paths in NFS.
- Prepare NFS environment
# Create directory
[root@nfs ~]# mkdir /root/data/{pv1,pv2,pv3} -pv
# Exposure services
[root@nfs ~]# more /etc/exports
/root/data/pv1 192.168.5.0/24(rw,no_root_squash)
/root/data/pv2 192.168.5.0/24(rw,no_root_squash)
/root/data/pv3 192.168.5.0/24(rw,no_root_squash)
# Restart service
[root@nfs ~]# systemctl restart nfs
- Create pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv1
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv1
server: 192.168.5.6
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv2
spec:
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv2
server: 192.168.5.6
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv3
spec:
capacity:
storage: 3Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
nfs:
path: /root/data/pv3
server: 192.168.5.6
# Create pv [root@k8s-master01 ~]# kubectl create -f pv.yaml persistentvolume/pv1 created persistentvolume/pv2 created persistentvolume/pv3 created # View pv [root@k8s-master01 ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS AGE VOLUMEMODE pv1 1Gi RWX Retain Available 10s Filesystem pv2 2Gi RWX Retain Available 10s Filesystem pv3 3Gi RWX Retain Available 9s Filesystem
8.2.2 PVC
PVC is a resource application, which is used to declare the demand information for storage space, access mode and storage category. Here is the resource manifest file:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc
namespace: dev
spec:
accessModes: # Access mode
selector: # PV selection with label
storageClassName: # Storage category
resources: # Request space
requests:
storage: 5Gi
Description of key configuration parameters of PVC:
- Access modes
Used to describe the access rights of user applications to storage resources
-
Selection criteria (selector)
Through the setting of Label Selector, PVC can filter the existing PV in the system
-
Storage category (storageClassName)
When defining PVC, you can set the required back-end storage category. Only the pv with this class can be selected by the system
-
Resource requests
Describes the request for storage resources
experiment
- Create pvc.yaml and apply for pv
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc1
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc2
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc3
namespace: dev
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
# Create pvc [root@k8s-master01 ~]# kubectl create -f pvc.yaml persistentvolumeclaim/pvc1 created persistentvolumeclaim/pvc2 created persistentvolumeclaim/pvc3 created # View pvc [root@k8s-master01 ~]# kubectl get pvc -n dev -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE pvc1 Bound pv1 1Gi RWX 15s Filesystem pvc2 Bound pv2 2Gi RWX 15s Filesystem pvc3 Bound pv3 3Gi RWX 15s Filesystem # View pv [root@k8s-master01 ~]# kubectl get pv -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM AGE VOLUMEMODE pv1 1Gi RWx Retain Bound dev/pvc1 3h37m Filesystem pv2 2Gi RWX Retain Bound dev/pvc2 3h37m Filesystem pv3 3Gi RWX Retain Bound dev/pvc3 3h37m Filesystem
- Create pods.yaml and use pv
apiVersion: v1
kind: Pod
metadata:
name: pod1
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do echo pod1 >> /root/out.txt; sleep 10; done;"]
volumeMounts:
- name: volume
mountPath: /root/
volumes:
- name: volume
persistentVolumeClaim:
claimName: pvc1
readOnly: false
---
apiVersion: v1
kind: Pod
metadata:
name: pod2
namespace: dev
spec:
containers:
- name: busybox
image: busybox:1.30
command: ["/bin/sh","-c","while true;do echo pod2 >> /root/out.txt; sleep 10; done;"]
volumeMounts:
- name: volume
mountPath: /root/
volumes:
- name: volume
persistentVolumeClaim:
claimName: pvc2
readOnly: false
# Create pod [root@k8s-master01 ~]# kubectl create -f pods.yaml pod/pod1 created pod/pod2 created # View pod [root@k8s-master01 ~]# kubectl get pods -n dev -o wide NAME READY STATUS RESTARTS AGE IP NODE pod1 1/1 Running 0 14s 10.244.1.69 node1 pod2 1/1 Running 0 14s 10.244.1.70 node1 # View pvc [root@k8s-master01 ~]# kubectl get pvc -n dev -o wide NAME STATUS VOLUME CAPACITY ACCESS MODES AGE VOLUMEMODE pvc1 Bound pv1 1Gi RWX 94m Filesystem pvc2 Bound pv2 2Gi RWX 94m Filesystem pvc3 Bound pv3 3Gi RWX 94m Filesystem # View pv [root@k8s-master01 ~]# kubectl get pv -n dev -o wide NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM AGE VOLUMEMODE pv1 1Gi RWX Retain Bound dev/pvc1 5h11m Filesystem pv2 2Gi RWX Retain Bound dev/pvc2 5h11m Filesystem pv3 3Gi RWX Retain Bound dev/pvc3 5h11m Filesystem # View file stores in nfs [root@nfs ~]# more /root/data/pv1/out.txt node1 node1 [root@nfs ~]# more /root/data/pv2/out.txt node2 node2
8.2.3 life cycle
PVC and PV are one-to-one correspondence, and the interaction between PV and PVC follows the following life cycle:
-
Provisioning: the administrator manually creates the underlying storage and PV
-
Resource binding: users create PVC, and kubernetes is responsible for finding PV according to the declaration of PVC and binding it
After the user defines the PVC, the system will select one of the existing PVS that meets the conditions according to the PVC's request for storage resources
- Once found, bind the PV with the user-defined PVC, and the user's application can use the PVC
- If it cannot be found, the PVC will be in the Pending state indefinitely until the system administrator creates a PV that meets its requirements
Once PV is bound to a PVC, it will be monopolized by this PVC and can no longer be bound with other PVC
-
Resource usage: users can use pvc in pod like volume
Pod uses the definition of Volume to mount PVC to a path in the container for use.
-
Resource release: users delete pvc to release pv
After the storage resource is used, the user can delete the PVC, and the PV bound to the PVC will be marked as "released", but it cannot be bound to other PVC immediately. Data written through the previous PVC may still be left on the storage device, and the PV can be used again only after it is cleared.
-
Resource recycling: kubernetes recycles resources according to the recycling policy set by pv
For PV, the administrator can set the recycling policy to set how to deal with the legacy data after the bound PVC releases resources. Only after the storage space of PV is recycled can it be bound and used by new PVC
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ce2zi2ut-163354912850) (kubenetes. Assets / image-20200515002806726. PNG)]
8.3 configuration storage
8.3.1 ConfigMap
ConfigMap is a special storage volume, which is mainly used to store configuration information.
Create configmap.yaml as follows:
apiVersion: v1
kind: ConfigMap
metadata:
name: configmap
namespace: dev
data:
info: |
username:admin
password:123456
Next, use this configuration file to create a configmap
# Create configmap [root@k8s-master01 ~]# kubectl create -f configmap.yaml configmap/configmap created # View configmap details [root@k8s-master01 ~]# kubectl describe cm configmap -n dev Name: configmap Namespace: dev Labels: <none> Annotations: <none> Data ==== info: ---- username:admin password:123456 Events: <none>
Next, create a pod-configmap.yaml and mount the configmap created above
apiVersion: v1
kind: Pod
metadata:
name: pod-configmap
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
volumeMounts: # Mount the configmap to the directory
- name: config
mountPath: /configmap/config
volumes: # Reference configmap
- name: config
configMap:
name: configmap
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-configmap.yaml pod/pod-configmap created # View pod [root@k8s-master01 ~]# kubectl get pod pod-configmap -n dev NAME READY STATUS RESTARTS AGE pod-configmap 1/1 Running 0 6s #Enter container [root@k8s-master01 ~]# kubectl exec -it pod-configmap -n dev /bin/sh # cd /configmap/config/ # ls info # more info username:admin password:123456 # You can see that the mapping has been successful, and each configmap has been mapped to a directory # Key --- > file value --- > contents in the file # At this time, if the contents of the configmap are updated, the values in the container will also be updated dynamically
8.3.2 Secret
In kubernetes, there is also an object very similar to ConfigMap, called Secret object. It is mainly used to store sensitive information, such as passwords, Secret keys, certificates, etc.
- First, use base64 to encode the data
[root@k8s-master01 ~]# echo -n 'admin' | base64 #Prepare username YWRtaW4= [root@k8s-master01 ~]# echo -n '123456' | base64 #Prepare password MTIzNDU2
- Next, write secret.yaml and create Secret
apiVersion: v1 kind: Secret metadata: name: secret namespace: dev type: Opaque data: username: YWRtaW4= password: MTIzNDU2
# Create secret [root@k8s-master01 ~]# kubectl create -f secret.yaml secret/secret created # View secret details [root@k8s-master01 ~]# kubectl describe secret secret -n dev Name: secret Namespace: dev Labels: <none> Annotations: <none> Type: Opaque Data ==== password: 6 bytes username: 5 bytes
- Create pod-secret.yaml and mount the secret created above:
apiVersion: v1
kind: Pod
metadata:
name: pod-secret
namespace: dev
spec:
containers:
- name: nginx
image: nginx:1.17.1
volumeMounts: # Mount secret to directory
- name: config
mountPath: /secret/config
volumes:
- name: config
secret:
secretName: secret
# Create pod [root@k8s-master01 ~]# kubectl create -f pod-secret.yaml pod/pod-secret created # View pod [root@k8s-master01 ~]# kubectl get pod pod-secret -n dev NAME READY STATUS RESTARTS AGE pod-secret 1/1 Running 0 2m28s # Enter the container, check the secret information, and find that it has been automatically decoded [root@k8s-master01 ~]# kubectl exec -it pod-secret /bin/sh -n dev / # ls /secret/config/ password username / # more /secret/config/username admin / # more /secret/config/password 123456
So far, the encoding of information using secret has been realized.
9. Safety certification
9.1 Access Control Overview
Kubernetes is a distributed cluster management tool, and it is an important task to ensure the security of the cluster. The so-called security is actually to ensure the authentication and authentication of various clients of kubernetes.
client
In Kubernetes clusters, there are usually two types of clients:
- User Account: it is generally a User Account managed independently of other services except kubernetes.
- Service Account: the account managed by kubernetes, which is used to provide identity for the service process in Pod when accessing kubernetes.
[external chain picture transfer failed, and the source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-cs0nychk-163354912851) (kubenetes. Assets / image-2020052012949189. PNG)]
Certification, authorization and access control
ApiServer is the only entrance to access and manage resource objects. Any request to access ApiServer must go through the following three processes:
- Authentication: identity authentication. Only the correct account can be authenticated
- Authorization: determines whether the user has permission to perform specific actions on the accessed resources
- Admission Control: it is used to supplement the authorization mechanism to achieve more detailed access control functions.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-x4rywwq7-163354912851) (kubenetes. Assets / image-20200520103942580. PNG)]
9.2 certification management
The key point of Kubernetes cluster security is how to identify and authenticate the client identity. It provides three client identity authentication methods:
-
HTTP Base authentication: authentication by user name + password
This authentication method is "user name":For "password" BASE64 The string encoded by the algorithm is placed in HTTP In request Header Authorization Send to the server in the domain. After receiving it, the server decodes it, obtains the user name and password, and then authenticates the user's identity.
-
HTTP Token authentication: identify legal users through a Token
This authentication method uses a long string that is difficult to imitate--Token A way to identify customers. each Token Corresponding to a user name, when the client initiates API When invoking a request, you need to HTTP Header Put in Token，API Server receive Token It will be followed by the information saved in the server token Comparison, and then the process of user identity authentication.
-
HTTPS certificate authentication: a two-way digital certificate authentication method based on CA root certificate signature
This authentication method is the most secure way, but it is also the most troublesome way to operate.
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-mgdrsnwc-163354912851) (kubenetes. Assets / image-20200518211037434. PNG)]
HTTPS authentication is generally divided into three processes:
-
Certificate application and issuance
HTTPS Servers on both sides of communication CA Application for certificate by the institution, CA The organization issues the root certificate, server certificate and private key to the applicant
-
Two way authentication between client and server
1> The client sends a request to the server, and the server issues its own certificate to the client, After receiving the certificate, the client decrypts the certificate through the private key and obtains the public key of the server in the certificate, The client uses the information in the server's public key authentication certificate. If it is consistent, the server will be recognized 2> The client sends its own certificate to the server. After receiving the certificate, the server decrypts the certificate through the private key, Obtain the public key of the client in the certificate, and use the public key to authenticate the certificate information to confirm whether the client is legal -
The server communicates with the client
After the server and the client negotiate the encryption scheme, the client will generate a random secret key and encrypt it, and then send it to the server. After the server receives the secret key, all the contents of the next communication between the two sides are encrypted by the random secret key
Note: Kubernetes allows you to configure multiple authentication methods at the same time, as long as any one of them passes the authentication
9.3 authorization management
Authorization occurs after successful authentication. You can know who the requesting user is through authentication, and then Kubernetes will decide whether the user has access according to the pre-defined authorization policy. This process is called authorization.
Each request sent to ApiServer carries user and resource information, such as the user sending the request, the path of the request, the action of the request, etc. authorization is compared with the authorization policy according to these information. If the policy is met, it is considered that the authorization is passed, otherwise an error will be returned.
API Server currently supports the following authorization policies:
- AlwaysDeny: it means to reject all requests. It is generally used for testing
- AlwaysAllow: all requests are allowed to be received, which is equivalent to that the cluster does not need authorization process (Kubernetes default policy)
- ABAC: attribute based access control, which means that user requests are matched and controlled using user configured authorization rules
- Webhook: authorize users by calling external REST services
- Node: a special mode used for access control of requests issued by kubelet
- RBAC: role based access control (default option under kubedm installation mode)
RBAC(Role-Based Access Control) mainly describes one thing: which objects are granted which permissions
The following concepts are involved:
- Objects: User, Groups, ServiceAccount
- Role: represents a set of actionable actions (permissions) defined on a resource
- Binding: bind the defined role to the user
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-lptcrlmi-163354912852) (kubenetes. Assets / image-20200519181209566. PNG)]
RBAC introduces four top-level resource objects:
- Role, ClusterRole: role, used to specify a group of permissions
- RoleBinding, ClusterRoleBinding: role binding, used to assign roles (permissions) to objects
Role,ClusterRole
A role is a collection of permissions. All permissions here are in the form of permission (white list).
# Role can only authorize resources in the namespace. nameapce needs to be specified kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: dev name: authorization-role rules: - apiGroups: [""] # A list of supported API groups. An empty string indicates the core API Group resources: ["pods"] # List of supported resource objects verbs: ["get", "watch", "list"] # List of allowed operation methods on resource objects
# ClusterRole can authorize resources within the cluster, resources across namespaces, and non resource types kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: authorization-clusterrole rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"]
The parameters in rules need to be explained in detail:
-
apiGroups: list of supported API groups
"","apps", "autoscaling", "batch"
-
resources: list of supported resource objects
"services", "endpoints", "pods","secrets","configmaps","crontabs","deployments","jobs", "nodes","rolebindings","clusterroles","daemonsets","replicasets","statefulsets", "horizontalpodautoscalers","replicationcontrollers","cronjobs"
-
verbs: list of operation methods on resource objects
"get", "list", "watch", "create", "update", "patch", "delete", "exec"
RoleBinding,ClusterRoleBinding
Role binding is used to bind a role to a target object. The binding target can be User, Group or ServiceAccount.
# RoleBinding can bind a subject in the same namespace to a Role, and the subject has the permissions defined by the Role kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: authorization-role-binding namespace: dev subjects: - kind: User name: heima apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: authorization-role apiGroup: rbac.authorization.k8s.io
# ClusterRoleBinding binds specific subject s to ClusterRole at the whole cluster level and all namespaces, and grants permissions kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: authorization-clusterrole-binding subjects: - kind: User name: heima apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: authorization-clusterrole apiGroup: rbac.authorization.k8s.io
RoleBinding refers to ClusterRole for authorization
RoleBinding can reference ClusterRole to authorize resource principals defined by ClusterRole in the same namespace.
A common practice is that the Cluster Administrator predefines a set of roles for the cluster scope( ClusterRole)，These are then reused in multiple namespaces ClusterRole. This can greatly improve the efficiency of authorization management and make the basic authorization rules under each namespace consistent with the use experience.
# Although authorization clusterrole is a cluster role, because RoleBinding is used # Therefore, heima can only read resources in the dev namespace kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: authorization-role-binding-ns namespace: dev subjects: - kind: User name: heima apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: authorization-clusterrole apiGroup: rbac.authorization.k8s.io
Actual combat: create an account that can only manage Pods resources in dev space
- Create account
# 1) Create certificate [root@k8s-master01 pki]# cd /etc/kubernetes/pki/ [root@k8s-master01 pki]# (umask 077;openssl genrsa -out devman.key 2048) # 2) Sign with apiserver's certificate # 2-1) sign the application. The user of the application is devman and the group is devgroup [root@k8s-master01 pki]# openssl req -new -key devman.key -out devman.csr -subj "/CN=devman/O=devgroup" # 2-2) signing certificate [root@k8s-master01 pki]# openssl x509 -req -in devman.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out devman.crt -days 3650 # 3) Set cluster, user and context information [root@k8s-master01 pki]# kubectl config set-cluster kubernetes --embed-certs=true --certificate-authority=/etc/kubernetes/pki/ca.crt --server=https://192.168.109.100:6443 [root@k8s-master01 pki]# kubectl config set-credentials devman --embed-certs=true --client-certificate=/etc/kubernetes/pki/devman.crt --client-key=/etc/kubernetes/pki/devman.key [root@k8s-master01 pki]# kubectl config set-context devman@kubernetes --cluster=kubernetes --user=devman # Switch account to devman [root@k8s-master01 pki]# kubectl config use-context devman@kubernetes Switched to context "devman@kubernetes". # Check the pod under dev and find that you do not have permission [root@k8s-master01 pki]# kubectl get pods -n dev Error from server (Forbidden): pods is forbidden: User "devman" cannot list resource "pods" in API group "" in the namespace "dev" # Switch to admin account [root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes Switched to context "kubernetes-admin@kubernetes".
2) Create Role and RoleBinding to authorize devman users
kind: Role apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: namespace: dev name: dev-role rules: - apiGroups: [""] resources: ["pods"] verbs: ["get", "watch", "list"] --- kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: authorization-role-binding namespace: dev subjects: - kind: User name: devman apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: dev-role apiGroup: rbac.authorization.k8s.io
[root@k8s-master01 pki]# kubectl create -f dev-role.yaml role.rbac.authorization.k8s.io/dev-role created rolebinding.rbac.authorization.k8s.io/authorization-role-binding created
- Switch accounts and verify again
# Switch account to devman [root@k8s-master01 pki]# kubectl config use-context devman@kubernetes Switched to context "devman@kubernetes". # View again [root@k8s-master01 pki]# kubectl get pods -n dev NAME READY STATUS RESTARTS AGE nginx-deployment-66cb59b984-8wp2k 1/1 Running 0 4d1h nginx-deployment-66cb59b984-dc46j 1/1 Running 0 4d1h nginx-deployment-66cb59b984-thfck 1/1 Running 0 4d1h # In order not to affect later learning, switch back to the admin account [root@k8s-master01 pki]# kubectl config use-context kubernetes-admin@kubernetes Switched to context "kubernetes-admin@kubernetes".
9.4 access control
After passing the previous authentication and authorization, the apiserver will not process the request until it passes the admission control process.
Admission control is a configurable controller list. You can select which admission controllers to execute through command line settings on API server:
--admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,
DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds
The apiserver will execute the request only after all admission controllers have passed the check. Otherwise, it will return reject.
Currently configurable Admission Control is as follows:
- AlwaysAdmit: allow all requests
- AlwaysDeny: all requests are prohibited, which is generally used for testing
- Always pullimages: always download images before starting the container
- DenyExecOnPrivileged: it blocks all requests to execute commands on the Privileged Container
- ImagePolicyWebhook: this plug-in will allow a Webhook program on the back end to complete the functions of the admission controller.
- Service Account: Service Account is automated
- SecurityContextDeny: this plug-in will invalidate all definitions in the Pod using SecurityContext
- ResourceQuota: for resource quota management purposes, observe all requests to ensure that the quota on the namespace does not exceed the limit
- LimitRanger: used for resource restriction management. It acts on the namespace to ensure resource restriction on the Pod
- InitialResources: set the Pod without resource request and limit according to the usage of its mirrored historical resources
- NamespaceLifecycle: if you try to create a resource object in a namespace that does not exist, the creation request will be rejected. When you delete a namespace, the system will delete all objects in the namespace.
- DefaultStorageClass: in order to realize the dynamic provisioning of shared storage, try to match the default StorageClass for PVC without specified StorageClass or PV, so as to minimize the back-end storage details that users need to know when applying for PVC
- DefaultTolerationSeconds: this plug-in sets the default "tolerance" time of 5min for those pods that do not have mandatory tolerances and have not ready: noexecute and unreachable:NoExecute tails
- PodSecurityPolicy: this plug-in is used to decide whether to control the security policy of a Pod according to its security context and available PodSecurityPolicy when creating or modifying a Pod
10. DashBoard
All operations previously done in kubernetes are done through the command-line tool kubectl. In fact, in order to provide a richer user experience, kubernetes has also developed a web-based user interface (Dashboard). Users can use Dashboard to deploy containerized applications, monitor the status of applications, perform troubleshooting, and manage various resources in kubernetes.
10.1 deploying Dashboard
- Download yaml and run Dashboard
# Download yaml
[root@k8s-master01 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0/aio/deploy/recommended.yaml
# Modify the Service type of kubernetes dashboard
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
type: NodePort # newly added
ports:
- port: 443
targetPort: 8443
nodePort: 30009 # newly added
selector:
k8s-app: kubernetes-dashboard
# deploy
[root@k8s-master01 ~]# kubectl create -f recommended.yaml
# View the resources under kubernetes dashboard in namespace
[root@k8s-master01 ~]# kubectl get pod,svc -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
pod/dashboard-metrics-scraper-c79c65bb7-zwfvw 1/1 Running 0 111s
pod/kubernetes-dashboard-56484d4c5-z95z5 1/1 Running 0 111s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/dashboard-metrics-scraper ClusterIP 10.96.89.218 <none> 8000/TCP 111s
service/kubernetes-dashboard NodePort 10.104.178.171 <none> 443:30009/TCP 111s
2) Create an access account and obtain a token
# Create account
[root@k8s-master01-1 ~]# kubectl create serviceaccount dashboard-admin -n kubernetes-dashboard
# to grant authorization
[root@k8s-master01-1 ~]# kubectl create clusterrolebinding dashboard-admin-rb --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:dashboard-admin
# Get account token
[root@k8s-master01 ~]# kubectl get secrets -n kubernetes-dashboard | grep dashboard-admin
dashboard-admin-token-xbqhh kubernetes.io/service-account-token 3 2m35s
[root@k8s-master01 ~]# kubectl describe secrets dashboard-admin-token-xbqhh -n kubernetes-dashboard
Name: dashboard-admin-token-xbqhh
Namespace: kubernetes-dashboard
Labels: <none>
Annotations: kubernetes.io/service-account.name: dashboard-admin
kubernetes.io/service-account.uid: 95d84d80-be7a-4d10-a2e0-68f90222d039
Type: kubernetes.io/service-account-token
Data
====
namespace: 20 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImJrYkF4bW5XcDhWcmNGUGJtek5NODFuSXl1aWptMmU2M3o4LTY5a2FKS2cifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4teGJxaGgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiOTVkODRkODAtYmU3YS00ZDEwLWEyZTAtNjhmOTAyMjJkMDM5Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.NAl7e8ZfWWdDoPxkqzJzTB46sK9E8iuJYnUI9vnBaY3Jts7T1g1msjsBnbxzQSYgAG--cV0WYxjndzJY_UWCwaGPrQrt_GunxmOK9AUnzURqm55GR2RXIZtjsWVP2EBatsDgHRmuUbQvTFOvdJB4x3nXcYLN2opAaMqg3rnU2rr-A8zCrIuX_eca12wIp_QiuP3SF-tzpdLpsyRfegTJZl6YnSGyaVkC9id-cxZRb307qdCfXPfCHR_2rt5FVfxARgg_C0e3eFHaaYQO7CitxsnIoIXpOFNAR8aUrmopJyODQIPqBWUehb7FhlU1DCduHnIIXVC_UICZ-MKYewBDLw
ca.crt: 1025 bytes
3) Access Dashboard UI through browser
Enter the above token on the login page
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-mufavuxg-163354912852) (kubenetes. Assets / image-20200520144548997. PNG)]
The following page represents success
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-xepe7ug9-163354912852) (kubenetes. Assets / image-20200520144959353. PNG)]
10.2 using DashBoard
This chapter takes Deployment as an example to demonstrate the use of DashBoard
see
Select the specified namespace dev, and then click Deployments to view all Deployments in the dev space
[external link picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-21kjj2qi-163354912853) (kubenetes. Assets / image-20200520154628679. PNG)]
Expansion and contraction capacity
Click scale on Deployment, specify the number of target replicas, and click OK
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-ysaiztph-163354912853) (kubenetes. Assets / image-20200520162605102. PNG)]
edit
Click edit on Deployment, then modify yaml file, and click OK
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-vm1o85gj-163354912853) (kubenetes. Assets / image-2020052016325644. PNG)]
View Pod
Click pods to view the pods list
[the external chain picture transfer fails. The source station may have an anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-whdjzcnk-163354912853) (kubenetes. Assets / image-20200520163552110. PNG)]
Operation Pod
Select a Pod to perform logs, exec, edit and delete operations
[external chain picture transfer failed, and the source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-afrsd62a-163354912854) (kubenetes. Assets / image-20200520163832827. PNG)]
Dashboard provides most functions of kubectl, which will not be demonstrated here