kubernetes RBAC Actual Warfare

Environmental preparation

Firstly, the kubernetes cluster is installed with kubeadm. Packet address here Easy to use and convenient, considerate service, no deception of children

For the purposes of this article, let users named devuser have access only to pod s under a specific namespace

Command line kubectl access

Install cfssl

This tool is very convenient to generate certificates. pem certificates and crt certificates can be directly used with the same coding.

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
chmod +x cfssl_linux-amd64
mv cfssl_linux-amd64 /bin/cfssl

wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssljson_linux-amd64
mv cfssljson_linux-amd64 /bin/cfssljson

wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl-certinfo_linux-amd64
mv cfssl-certinfo_linux-amd64 /bin/cfssl-certinfo

Issuing Client Certificate

Issuing user certificate based on ca certificate and single key
The root certificate is already in the / etc/kubernetes/pki directory

[root@master1 ~]# ls /etc/kubernetes/pki/
apiserver.crt                 ca-config.json  devuser-csr.json    front-proxy-ca.key      sa.pub
apiserver.key                 ca.crt          devuser-key.pem     front-proxy-client.crt
apiserver-kubelet-client.crt  ca.key          devuser.pem         front-proxy-client.key
apiserver-kubelet-client.key  devuser.csr     front-proxy-ca.crt  sa.key

Note the following files: ca.crt ca.key ca-config.json devuser-csr.json

Create a ca-config.json file

cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "87600h"
      }
    }
  }
}
EOF

Create devuser-csr.json file:
The user name of k8s is obtained from CN s. Group is obtained from O. This user or group is used for later role binding

cat > devuser-csr.json <<EOF
{
  "CN": "devuser",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

Generate user certificates:

$ cfssl gencert -ca=ca.crt -ca-key=ca.key -config=ca-config.json -profile=kubernetes devuser-csr.json | cfssljson -bare devuser

The following files are generated:

devuser.csr devuser-key.pem devuser.pem

Verification certificate

# cfssl-certinfo -cert kubernetes.pem

Generate config file

kubeadm has generated admin.conf, which can be used directly to configure cluster parameters by itself.

$ cp /etc/kubernetes/admin.conf devuser.kubeconfig

Set client authentication parameters:

kubectl config set-credentials devuser \
--client-certificate=/etc/kubernetes/ssl/devuser.pem \
--client-key=/etc/kubernetes/ssl/devuser-key.pem \
--embed-certs=true \
--kubeconfig=devuser.kubeconfig

Set context parameters:

kubectl config set-context kubernetes \
--cluster=kubernetes \
--user=devuser \
--namespace=kube-system \
--kubeconfig=devuser.kubeconfig

Set no context:

kubectl config use-context kubernetes --kubeconfig=devuser.kubeconfig

Take a look at the devuser.kubeconfig changes in one step above. There are three main things in it.

• Cluster: Cluster information, containing cluster addresses and public keys
• user: user information, client certificates and private keys. True information is read from certificates. What people can see is only for people to see.
• context: Maintain a triple, namespace cluster and user

Create roles

Create a role called pod-reader

[root@master1 ~]# cat pod-reader.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: kube-system
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

kubectl create -f pod-reader.yaml

Bind user

Create a role binding that binds the pod-reader role to devuser

[root@master1 ~]# cat devuser-role-bind.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: kube-system
subjects:
- kind: User
  name: devuser   # Target user
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader  # Role information
  apiGroup: rbac.authorization.k8s.io

kubectl create -f devuser-role-bind.yaml

Use the new config file

$ rm .kube/config && cp devuser.kubeconfig .kube/config

As a result, there are no other namespace privileges and no access to node information.

[root@master1 ~]# kubectl get node
Error from server (Forbidden): nodes is forbidden: User "devuser" cannot list nodes at the cluster scope

[root@master1 ~]# kubectl get pod -n kube-system
NAME                                       READY     STATUS    RESTARTS   AGE
calico-kube-controllers-55449f8d88-74x8f   1/1       Running   0          8d
calico-node-clpqr                          2/2       Running   0          8d
kube-apiserver-master1                     1/1       Running   2          8d
kube-controller-manager-master1            1/1       Running   1          8d
kube-dns-545bc4bfd4-p6trj                  3/3       Running   0          8d
kube-proxy-tln54                           1/1       Running   0          8d
kube-scheduler-master1                     1/1       Running   1          8d

[root@master1 ~]# kubectl get pod -n default
Error from server (Forbidden): pods is forbidden: User "devuser" cannot list pods in the namespace "default": role.rbac.authorization.k8s.io "pod-reader" not found

dashboard access

service account principle

There are two kinds of users in k8s, one is User, the other is service account. User is used by people, and service account is used by process, so that process has relevant permissions.

If dasboard is a process, we can create a service account for it to access k8s.

Let's see how admin is granted to dashboard:

╰─➤  cat dashboard-admin.yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

Bind the kubernetes-dashboard Service Account to cluster-admin Cluster Role. This cluster role is very powerful and has all permissions.

[root@master1 ~]# kubectl describe clusterrole cluster-admin -n kube-system
Name:         cluster-admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate=true
PolicyRule:
  Resources  Non-Resource URLs  Resource Names  Verbs
  ---------  -----------------  --------------  -----
             [*]                []              [*]
  *.*        []                 []              [*]

The service account was created when dashboard was created:

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kube-system

Then specify service account in deployment

      volumes:
      - name: kubernetes-dashboard-certs
        secret:
          secretName: kubernetes-dashboard-certs
      - name: tmp-volume
        emptyDir: {}
      serviceAccountName: kubernetes-dashboard

Safer practices

[root@master1 ~]# cat admin-token.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: admin
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: admin
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin
  namespace: kube-system
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile

[root@master1 ~]# kubectl get secret -n kube-system|grep admin
admin-token-7rdhf                        kubernetes.io/service-account-token   3         14m

[root@master1 ~]# kubectl describe secret admin-token-7rdhf -n kube-system
Name:         admin-token-7rdhf
Namespace:    kube-system
Labels:       <none>
Annotations:  kubernetes.io/service-account.name=admin
              kubernetes.io/service-account.uid=affe82d4-d10b-11e7-ad03-00163e01d684

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1025 bytes
namespace:  11 bytes
token:      eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi10b2tlbi03cmRoZiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJhZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImFmZmU4MmQ0LWQxMGItMTFlNy1hZDAzLTAwMTYzZTAxZDY4NCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTphZG1pbiJ9.jSfQhFsY7V0ZmfqxM8lM_UUOoUhI86axDSeyVVtldSUY-BeP2Nw4q-ooKGJTBBsrOWvMiQePcQxJTKR1K4EIfnA2FOnVm4IjMa40pr7-oRVY37YnR_1LMalG9vrWmqFiqIsKe9hjkoFDuCaP7UIuv16RsV7hRlL4IToqmJMyJ1xj2qb1oW4P1pdaRr4Pw02XBz9yBpD1fs-lbwheu1UKcEnbHS_0S3zlmAgCrpwDFl2UYOmgUKQVpJhX4wBRRQbwo1Sn4rEFVI1NIa9l_lM7Mf6YEquLHRu3BCZTdu9YfY9pevQz4OfHE0NOvDIqmGRL8Z9kPADAXbljWzcD1m1xCQ

Use this token to login on the interface