Explain:
master : 132， node1:11 ， node2:12
The main tests are as follows:
1. container is specified to the corresponding node
2. After Container's death
3. node restart
4. When node is unavailable
5. node recovery
The main thing is to test the above situation to see if k8s will guarantee our application as we want.
1. container is specified to the corresponding node
It can be implemented by nodename, nodeselector
nodeselector is used here for better extension and customization.
deployment yaml file
[root@yzb-centos72-3 imgcloud]# more dfs-data-deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: dfs-data-depl
namespace: imgcloud
spec:
replicas: 1
template:
metadata:
labels:
k8s-app: dfs-data
spec:
nodeSelector:
imgcloud/app: dfs-data
containers:
- name: dfs-data
image: registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data
imagePullPolicy: Always
ports:
- containerPort: 2101
hostPort: 2101
- containerPort: 3101
hostPort: 3101Before creating the deployment, label settings are performed, and node is labeled by kubectl label.
View the pod status after creating deployment
Visit 11[root@yzb-centos72-3 imgcloud]# kubectl get pod --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE imgcloud dfs-data-depl-2521169640-62m79 1/1 Running 0 19m 172.17.73.2 10.3.14.11
2. After Container's death
Stop container
[root@k8s-node-11 k8s]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS 1a0e1704331a registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 45 seconds ago Up 43 seconds [root@k8s-node-11 k8s]# [root@k8s-node-11 k8s]# docker stop 3fcf58bbd812 3fcf58bbd812 [root@k8s-node-11 k8s]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS fed8ef3177c6 registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data "/usr/sbin/init" 8 seconds ago Up 7 seconds 1a0e1704331a registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 18 minutes ago Up 18 minutes [root@k8s-node-11 k8s]#
You can see that a container was restarted soon to compose our requirements
3. node restart
After reboot node 1,
[root@k8s-node-11 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES [root@k8s-node-11 ~]# [root@k8s-node-11 ~]# [root@k8s-node-11 ~]# [root@k8s-node-11 ~]# netstat -lnpt Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 2074/sshd tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 2317/kubelet tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 751/kube-proxy tcp6 0 0 :::10250 :::* LISTEN 2317/kubelet tcp6 0 0 :::10255 :::* LISTEN 2317/kubelet tcp6 0 0 :::2101 :::* LISTEN 2523/docker-proxy-c tcp6 0 0 :::3101 :::* LISTEN 2514/docker-proxy-c tcp6 0 0 :::4194 :::* LISTEN 2317/kubelet [root@k8s-node-11 ~]# [root@k8s-node-11 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ed9be07a17f6 registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data "/usr/sbin/init" 13 seconds ago Up 12 seconds k8s_dfs-data.a55c69cf_dfs-data-depl-2521169640-62m79_imgcloud_dd8bf189-5af9-11e7-a7aa-0669a40010d2_368201ac 5407f90223f6 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 15 seconds ago Up 14 seconds 0.0.0.0:2101->2101/tcp, 0.0.0.0:3101->3101/tcp k8s_POD.92d92fce_dfs-data-depl-2521169640-62m79_imgcloud_dd8bf189-5af9-11e7-a7aa-0669a40010d2_b6a7a804 [root@k8s-node-11 ~]#
You can see that during node1, the reboot period, node2 did not create our new pod, that is to say, app is not accessible during this period. Noe1 started our service very quickly.
4. When node is unavailable
I turn off the flannel of node1 and look at nodes, demploy, pod
[root@yzb-centos72-3 imgcloud]# kubectl get node NAME STATUS AGE 10.3.14.11 Ready 1h 10.3.14.12 Ready 1h 172.20.4.133 Ready 3d [root@yzb-centos72-3 imgcloud]# [root@yzb-centos72-3 imgcloud]# kubectl get node NAME STATUS AGE 10.3.14.11 NotReady 1h 10.3.14.12 Ready 1h 172.20.4.133 Ready 3d [root@yzb-centos72-3 imgcloud]# kubectl get deployment --all-namespaces NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE imgcloud dfs-data-depl 1 1 1 0 51m [root@yzb-centos72-3 imgcloud]# kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE imgcloud dfs-data-depl-2521169640-62m79 1/1 Running 2 33m [root@yzb-centos72-3 imgcloud]#
node status becomes notready, deployment avaliable becomes 0. But pod does not change.
Looking at the pod status through description, we found that after a while (no timer, maybe 2 minutes or so), the controller manager worked.
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
36m 33m 14 {default-scheduler } Warning FailedScheduling pod (dfs-data-depl-2521169640-62m79) failed to fit in any node
fit failure summary on nodes : MatchNodeSelector (1)
33m 33m 1 {default-scheduler } Normal Scheduled Successfully assigned dfs-data-depl-2521169640-62m79 to 10.3.14.11
33m 33m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Created Created container with docker id 3fcf58bbd812; Security:[seccomp=unconfined]
33m 33m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Started Started container with docker id 3fcf58bbd812
33m 14m 2 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Pulling pulling image "registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data"
33m 14m 2 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Pulled Successfully pulled image "registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data"
14m 14m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Started Started container with docker id fed8ef3177c6
14m 14m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Created Created container with docker id fed8ef3177c6; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Pulling pulling image "registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data"
10m 10m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Pulled Successfully pulled image "registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data"
10m 10m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Created Created container with docker id ed9be07a17f6; Security:[seccomp=unconfined]
10m 10m 1 {kubelet 10.3.14.11} spec.containers{dfs-data} Normal Started Started container with docker id ed9be07a17f6
21s 21s 1 {controllermanager } Normal NodeControllerEviction Marking for deletion Pod dfs-data-depl-2521169640-62m79 from Node 10.3.14.11[root@iz2ze0fq2isg8vphkpos5tz ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 35701d1242b9 registry.cn-beijing.aliyuncs.com/zybros/env:imgcloud-dfs-data "/usr/sbin/init" 39 seconds ago Up 37 seconds k8s_dfs-data.a55c69cf_dfs-data-depl-2521169640-s62vc_imgcloud_e2d678e9-5afe-11e7-a7aa-0669a40010d2_6463e4cb 2b48e0f43df9 registry.access.redhat.com/rhel7/pod-infrastructure:latest "/pod" 40 seconds ago Up 38 seconds 0.0.0.0:2101->2101/tcp, 0.0.0.0:3101->3101/tcp k8s_POD.92d92fce_dfs-data-depl-2521169640-s62vc_imgcloud_e2d678e9-5afe-11e7-a7aa-0669a40010d2_dd97ae88
5. node recovery
app scale to 2[root@yzb-centos72-3 imgcloud]# kubectl get node NAME STATUS AGE 10.3.14.11 Ready 1h 10.3.14.12 Ready 1h
[root@yzb-centos72-3 imgcloud]# kubectl get deployment --all-namespaces NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE imgcloud dfs-data-depl 1 1 1 1 1h [root@yzb-centos72-3 imgcloud]# kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE imgcloud dfs-data-depl-2521169640-s62vc 1/1 Running 0 11m [root@yzb-centos72-3 imgcloud]# [root@yzb-centos72-3 imgcloud]# kubectl scale deployment dfs-data-depl -n imgcloud --replicas=2 deployment "dfs-data-depl" scaled [root@yzb-centos72-3 imgcloud]# kubectl get deployment --all-namespaces NAMESPACE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE imgcloud dfs-data-depl 2 2 2 2 1h [root@yzb-centos72-3 imgcloud]# kubectl get pod --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE imgcloud dfs-data-depl-2521169640-ggptp 1/1 Running 0 10s imgcloud dfs-data-depl-2521169640-s62vc 1/1 Running 0 12m [root@yzb-centos72-3 imgcloud]# kubectl get pod --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE imgcloud dfs-data-depl-2521169640-ggptp 1/1 Running 0 17s 172.17.73.2 10.3.14.11 imgcloud dfs-data-depl-2521169640-s62vc 1/1 Running 0 12m 172.17.97.2 10.3.14.12
On the premise of high availability of master etcd, under the normal condition of node power and network, it is basically unattended and repaired automatically.