Data persistence of k8s
kubernetes storage volume:
We know that by default, the container's data is non persistent, and the data is lost after the container is destroyed, so docker provides a volume mechanism to store the data persistently. Similarly, k8s provides a more powerful volume mechanism and rich plug-ins to solve the problem of container data persistence and data sharing between containers.
volume:
We often say: containers and pods are transient.
The implication is that they may have a short life cycle and are frequently destroyed and created. When the container is destroyed, the data stored in the internal file system of the container will be cleared. In order to persist the data of the container, k8s volume can be used.
The life cycle of a Volume is independent of the container. The container in the Pod may be destroyed and rebuilt, but the Volume will be retained.
The volume types supported by k8s include emptydir, hostpath, persistentVolumeClaim, gcepersistent disk, awsexelasticblockstore, nfs, iscsi, gitRepo, secret, etc. for a complete list and detailed documents, please refer to http://docs.kubernetes.org.cn/429.html.
In this paper, we mainly practice the following volume types:
1. EmptyDir (temporary storage):
emptyDir is the most basic Volume type. As its name suggests, an emptyDir Volume is an empty directory on the Host. In other words, there is no specified directory or file on the Host, which is directly mapped from the pod to the Host. (similar to the way docker manager volume is mounted in docker)
Let's practice emptydir with the following example:
[root@master yaml]# vim emptydir.yaml
apiVersion: v1
kind: Pod
metadata:
name: read-write
spec:
containers:
- name: write
image: busybox
volumeMounts: #Define data persistence
- mountPath: /write #Define the mount directory, which is the directory inside the pod
name: share-volume
args:
- /bin/sh
- -c
- echo "hello volumes" > /write/hello; sleep 3000;
- name: read #Define a second container within the pod
image: busybox
volumeMounts:
- mountPath: /read
name: share-volume
args:
- /bin/sh
- -c
- cat /read/hello; sleep 30000;
volumes:
- name: share-volume
emptyDir: {} #Define a data persistence type empytdirWe simulate a pod running two containers, two containers share a volume, one is responsible for writing data, one is responsible for reading data.
//Run the pod and view: [root@master yaml]# kubectl apply -f emptydir.yaml pod/read-write created
[root@master yaml]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES read-write 2/2 Running 0 14s 10.244.2.2 node02 <none> <none>
//Let's look at the mount contents in two containers: [root@master yaml]# kubectl exec -it read-write -c read cat /read/hello hello volumes [root@master yaml]# kubectl exec -it read-write -c write cat /write/hello hello volumes
Parameter interpretation:
-c: for specifying a container, it is the abbreviation of -- container =, which can be viewed through -- help.
Because emptyDir is the directory in the Docker Host file system, its effect is equivalent to executing docker run -v /write and docker run -v /read. We're at node02
Check the detailed configuration information of the container through docker inspect ion. We find that both containers have the same directory mount ed:
"Mounts": [
{
"Type": "bind",
"Source": "/var/lib/kubelet/pods/756b4f4a-917a-414d-a7ee-523eecf05465/volumes/kubernetes.io~empty-dir/share-volume",
"Destination": "/read",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},
{
"Type": "bind",
"Source": "/var/lib/kubelet/pods/756b4f4a-917a-414d-a7ee-523eecf05465/volumes/kubernetes.io~empty-dir/share-volume",
"Destination": "/write",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
},Here, "/ var / lib / kubelet / pods / 756b4f4a-917a-414d-a7ee-523eecf05465 / volumes / kubernetes. IO ~ empty dir / share volume" is the real path for emptydir to mount to dockerhost.
So we can enter this path to view:
[root@node02 ~]# cd /var/lib/kubelet/pods/756b4f4a-917a-414d-a7ee-523eecf05465/volumes/kubernetes.io~empty-dir/share-volume/ [root@node02 share-volume]# cat hello hello volumes
Summary emptydir:
Different containers in the same pod share the same persistent directory. When the pod node is deleted, the contents of the volume will also be deleted, but if only the container is destroyed and the pod is still there, the volume will not be affected. That is to say, the data persistence life cycle of emptydir is consistent with the pod used. Generally used as temporary storage, as well as the temporary storage directory of checkpoint, which is the intermediate process of long-time tasks, and multi container shared directory.
2，hostPath:
- 1) mount the existing directory or file on the host to the container.
- 2) there are not many scenarios for this persistence method, because the core of virtualization technology is to isolate the host, but this method increases the coupling between pod and nodes.
- 3) generally, this method is used for data persistence of k8s cluster and docker.
For example, Kube API server and Kube controller manager are such applications.
We use the command "kubectl Edit - n Kube system pod Kube apserver master" to view the configuration of Kube apserver pod. The following is the relevant part of Volume:
volumeMounts: - mountPath: /etc/ssl/certs name: ca-certs readOnly: true - mountPath: /etc/pki name: etc-pki readOnly: true - mountPath: /etc/kubernetes/pki name: k8s-certs readOnly: true
volumes:
- hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
- hostPath:
path: /etc/pki
type: DirectoryOrCreate
name: etc-pki
- hostPath:
path: /etc/kubernetes/pki
type: DirectoryOrCreate
name: k8s-certsHere, three hostpath volumes are defined as k8s certs, CA certs and etc PKI, corresponding to the Host directory / etc/kubernetes/pki, / etc/ssl/certs and / etc/pki respectively.
If the Pod is destroyed, the directory corresponding to the hostPath will also be preserved. From this point of view, the persistence of hostPath is better than emptyDir. However, once the Host crashes, the hostPath cannot be accessed.
3,pv & pvc
- Persistent volume (pv): a unified data persistence directory refers to a section of space on a storage system provided by the Cluster Administrator configuration. It is an abstraction of the underlying shared storage, which takes shared storage as a resource that can be applied for by users and realizes the "storage consumption" mechanism.
- Persistent volume claim (PVC): a claim for pv persistent space, declaration. Specify the required minimum capacity requirements and access patterns, and then the user submits the list of persistent volume declarations to the kubernetes api server, which will find a matching persistent volume and bind it to the persistent volume declaration.
NFS PersistentVolume
Practice PV and PVC through NFS.
1) we deploy the nfs service on the master node:
[root@master ~]# yum -y install nfs-utils [root@master ~]# mkdir /nfsdata [root@master ~]# vim /etc/exports #Writing an nfs configuration file /nfsdata 172.16.1.0/24(rw,sync,no_root_squash) [root@master ~]# systemctl enable rpcbind [root@master ~]# systemctl start rpcbind [root@master ~]# systemctl enable nfs-server [root@master ~]# systemctl start nfs-server
[root@master ~]# showmount -e #Check whether the mount is successful Export list for master: /nfsdata 172.16.1.0/24
2) create pv:
[root@master yaml]# vim nfs-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nfs
nfs:
path: /nfsdata #Specify nfs shared directory
server: 172.16.1.30 #Specifies the ip address of the nfs server//Run pv with the following command: [root@master yaml]# kubectl apply -f nfs-pv.yaml persistentvolume/nfs-pv created
Field explanation:
Capacity: Specifies the capacity of pv. At present, capacity only supports space setting. In the future, IOPS and throughput should also be specified.
Access modes: access modes, including the following modes:
ReadWriteOnce: mount to a single node in a read-write manner, abbreviated as RWO in the command line.
ReadOnlyMany: mount to multiple nodes in read-only mode, abbreviated as ROX in the command line.
ReadWriteMany: mounts to multiple nodes in a read-write manner, abbreviated to RWX on the command line.
Persistentvolumerreclaimpolicy: the recovery policy when pv space is released. There are several policies as follows:
Recycle: clear the data in pv and recycle it automatically. (the automatic recycling policy is protected by the pvc protection mechanism. When the pvc is deleted, as long as the pvc is still in the data, it is still in the data)
Retain: keep it still, and the administrator will recycle it manually.
Delete: delete cloud storage resources, only supported by some cloud storage systems, if AWS, EBS, GCE PD, Azure Disk and Cinder.
Note: the recycling policy here refers to whether the stored source files are deleted after pv is deleted.
storageClassName: the basis of association between pv and pvc.
//Verify that pv is available: [root@master yaml]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE nfs-pv 1Gi (Capacity is 1. GB) RWO (Read and write) Recycle (Auto recycle) Available(Available, make sure it is in this state.) nfs(Be based on nfs To do it) Eighteen m(Time)
3) create a pvc:
[root@master yaml]# vim nfs-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nfs-pvc
spec:
accessModes:
- ReadWriteOnce #The access mode of pv and pvc must be consistent
resources: #Define the resources to request in the requests subfield under this field
requests:
storage: 1Gi
storageClassName: nfsRun this pvc: [root@master yaml]# kubectl apply -f nfs-pvc.yaml persistentvolumeclaim/nfs-pvc created
//Verify pvc is available: [root@master yaml]# kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE nfs-pvc Bound nfs-pv 1Gi RWO nfs 3m53s [root@master yaml]# kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE nfs-pv 1Gi RWO Recycle Bound default/nfs-pvc nfs 29m
Make sure that the status of both pv and pvc is Bound, which means the binding is successful.
Use of pv space.
Next, we practice the pv usage of mysql:
1) create a mysql pod:
[root@master yaml]# vim mysql-pod.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: mysql
spec:
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.7
env: #Define a variable to map the mysqlroot password in the container to the local
- name: MYSQL_ROOT_PASSWORD
value: 123.com #Password is 123.com
ports:
- containerPort: 3306
volumeMounts: #Define data persistence
- name: mysql-pv-storage
mountPath: /var/lib/mysql #This directory is the default mysql data persistence directory
volumes: #The volumes field is an explanation above
- name: mysql-pv-storage #Note that the name should be the same as above
persistentVolumeClaim: #Specify pvc, note that the pvc declared below should be the same as the pvc name created before
claimName: nfs-pvc
---
apiVersion: v1 #Create a service resource object
kind: Service
metadata:
name: mysql
spec:
type: NodePort
ports:
- port: 3306
targetPort: 3306
nodePort: 30000
selector:
app: mysqlRun pod with the following command: [root@master yaml]# kubectl apply -f mysql-pod.yaml deployment.extensions/mysql created service/mysql created
//To see if the pod is working properly: [root@master yaml]# kubectl get pod -o wide mysql-68d65b9dd9-hf2bf NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mysql-68d65b9dd9-hf2bf 1/1 Running 0 9m34s 10.244.1.3 node01 <none> <none>
2) log in to mysql database to write data:
[root@master yaml]# kubectl exec -it mysql-68d65b9dd9-hf2bf -- mysql -u root -p123.com Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
mysql> create database volumes_db; #Create Library
Query OK, 1 row affected (0.01 sec)
mysql> use volumes_db; #Entering the library
Database changed
mysql> create table my_id( #Create table
-> id int primary key,
-> name varchar(25)
-> );
Query OK, 0 rows affected (0.04 sec)
mysql> insert into my_id values(1,'zhangsan'); #Write data to table
Query OK, 1 row affected (0.01 sec)
mysql> select * from my_id; #View data
+----+----------+
| id | name |
+----+----------+
| 1 | zhangsan |
+----+----------+
1 row in set (0.00 sec)3) verification:
(1) delete the pod manually to verify whether the data in the database still exists
[root@master ~]# kubectl delete pod mysql-68d65b9dd9-hf2bf pod "mysql-68d65b9dd9-hf2bf" deleted
[root@master ~]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mysql-68d65b9dd9-bf9v8 1/1 Running 0 26s 10.244.1.4 node01 <none> <none>
After deleting the pod, kubernetes will generate a new pod. We log in to mysql to check
Whether the data will still exist.
[root@master ~]# kubectl exec -it mysql-68d65b9dd9-bf9v8 -- mysql -u root -p123.com Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select * from volumes_db.my_id; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | +----+----------+ 1 row in set (0.01 sec)
You can see that the data will still exist.
2) simulate whether the node where the pod is running goes down and whether the data returns to normal in the newly generated pod.
From the information above, we know that the pod is running on node01, so we shut down the node01 host in the cluster.
##[root@node01 ~]# systemctl poweroff
After a period of time, kubernetes will migrate the pod to the node02 host in the cluster:
[root@master ~]# kubectl get nodes #Know that node01 node is down NAME STATUS ROLES AGE VERSION master Ready master 39d v1.15.0 node01 NotReady <none> 39d v1.15.0 node02 Ready <none> 39d v1.15.0
[root@master ~]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mysql-68d65b9dd9-bf9v8 1/1 Terminating 0 15m 10.244.1.4 node01 <none> <none> mysql-68d65b9dd9-mvxdg 1/1 Running 0 83s 10.244.2.3 node02 <none> <none>
You can see that the pod has been migrated to node02.
Finally, we log in to mysql to verify whether the data is recovered:
[root@master ~]# kubectl exec -it mysql-68d65b9dd9-mvxdg -- mysql -u root -p123.com Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select * from volumes_db.my_id; +----+----------+ | id | name | +----+----------+ | 1 | zhangsan | +----+----------+ 1 row in set (0.09 sec)
It can be seen that after pod migration, mysql service is running normally, and data is not lost...
pv and pvc realize the persistence of mysql data, separate the responsibilities of administrators and ordinary users, and are more suitable for the production environment.