Hello, I'm Zhang Jintao.

Articles published before me Container image security in the cloud native Era In (Series), I mentioned Kube apiserver, the core component of Kubernetes cluster, which allows components from end users or clusters to communicate with it (for example, query, create, modify or delete Kubernetes resources).

In this article, we will focus on a very important part of Kube apiserver request processing - Admission Controller

What is the access controller of K8s

Request processing flow in K8s

Before talking about what the K8s admission controller is, let's review the process of processing specific requests in the Kubernetes API.

Figure 1. The process of Kubernetes API processing requests (from API Handler to etcd persistence)

As shown in the figure above, the request of each API is received by Kube apiserver from the beginning and finally persisted to ETCD, which is the request processing flow of Kubernetes API.

It mainly includes the following parts:

• API Handler -- mainly responsible for providing services and receiving requests.

For its internal implementation, the request will arrive first. FullHandlerChain (which is built by DefaultBuildHandlerChain) is a director object

type director struct {
    name               string
    goRestfulContainer *restful.Container
    nonGoRestfulMux    *mux.PathRecorderMux
}

The director initializes according to the configuration. If the RootPath of the WebServices of the goRestfulContainer is / apis, or the request prefix matches the RootPath, it enters the Restful processing link.

• Authentication -- authentication process.

After the TLS connection is established, authentication processing will be carried out. If the request for authentication fails, the request will be rejected and 401 error code will be returned; If the authentication is successful, it will proceed to the authentication part. At present, there are many supported client authentication methods, such as x509 client certificate, Bearer Token, user name and password based authentication, OpenID authentication, etc. Since these contents are not the focus of this article, we will skip them for the time being. Interested partners can leave a message in the comment area for discussion.

• Authorization -- authentication process.

Kubernetes supports a variety of authentication modes, such as ABAC mode, RBAC mode and Webhook mode. When creating a cluster, we can directly pass parameters to Kube apiserver for configuration, which will not be repeated here.

• Mutating Admission -- refers to the admission controller that can be used to change operations, which will be described in detail below.

• Object Schema Validation -- schema validation of resource objects.

• Validating Admission -- refers to the admission controller that can be used to perform validation operations, which will be described in detail below.

• ETCD -- ETCD implements persistent storage of resources.

The above is a request processing flow, in which changing permission and validating permission are our protagonists today. Let's take a detailed look.

What is the Admission Controller

Access controller refers to some codes or functions that can be used to change or verify it after the request passes authentication and authorization.

The process of admission control is divided into two stages:

• In the first stage, run the changing admission controller. It can modify the objects it accepts, which leads to another role of it, changing the relevant resources as part of the request processing;

• In the second stage, run the Validating Admission controller. It can only be verified and cannot modify any resource data;

It should be noted that some controllers can be both a change admission controller and a verification admission controller. If the admission controller of any stage rejects the request, the whole request will be rejected immediately and an error will be returned to the end user.

Why do I need Admission Controller

We mainly understand why we need access controller from two perspectives:

• From a safety perspective

• • We need to know whether the image source deployed in the Kubernetes cluster is trusted to avoid being attacked;
  • In general, try not to use the root user in the Pod, or try not to open the privilege container;

• From the perspective of governance

• • For example, to distinguish businesses / services through labels, you can verify whether the corresponding label already exists in the service through the admission controller;
  • For example, add resource quota restrictions to avoid oversold resources;

Admission controller

Considering that these requirements are useful indeed, Kubernetes has implemented many built-in permission controllers. Refer to the official documentation for a detailed list: https://kubernetes.io/docs/re...

These built-in admission controllers are built together with Kube apiserver in the form of plug-ins. You can enable and shut them down. For example, the following parameters are used for control:

➜  bin ./kube-apiserver --help |grep admission-plugins    
      --admission-control strings              Admission is divided into two phases. In the first phase, only mutating admission plugins run. In the second phase, only validating admission plugins run. The names in the below list may represent a validating plugin, a mutating plugin, or both. The order of plugins in which they are passed to this flag does not matter. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. (DEPRECATED: Use --enable-admission-plugins or --disable-admission-plugins instead. Will be removed in a future version.)
      --disable-admission-plugins strings      admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
      --enable-admission-plugins strings       admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

There are two special admission controllers, MutatingAdmissionWebhook and ValidatingAdmissionWebhook. They do not really implement the corresponding policies, but provide an extensible way for Kube apiserver. Users can use self built services by configuring MutatingAdmissionWebhook and ValidatingAdmissionWebhook. In this way, there is no need to compile or restart Kube spiserver. It is completely dynamic and very convenient.

Let's take a closer look.

Dynamic admission controller

When the MutatingAdmissionWebhook and ValidatingAdmissionWebhook mentioned above are used for runtime configuration, the advertisement controller called in the form of Webhook is the dynamic Admission Controller.

It is an HTTP callback mechanism used to receive and process admission requests, which is a Web service. Currently, there are two types of admission webhook s:

• validating admission webhook
• mutating admission webhook

The mutating admission webhook will be called preferentially. Resources can be modified during this process.

If we need to ensure the final state of the object to perform some operations, we should consider using the validating admission webhook, because the requests that reach this stage will not be modified.

Service conditions

• Ensure that the Kubernetes cluster version is at least v1.16 (to use the admissionregistration.k8s.io/v1 API) or v1.9 (to use the admissionregistration.k8s.io/v1beta1 API);

• Ensure that the MutatingAdmissionWebhook and ValidatingAdmissionWebhook admission controllers have been enabled;

• Ensure that the admissionregistration.k8s.io/v1beta1 or admissionregistration.k8s.io/v1 API is enabled;

What is an Admission Webhook

In fact, it is an ordinary HTTP Server, which needs to process resources of type AdmissionReview. Let's take an example, for example, to verify the admission of Ingress resources:

func (ia *IngressAdmission) HandleAdmission(obj runtime.Object) (runtime.Object, error) {

    review, isV1 := obj.(*admissionv1.AdmissionReview)
    if !isV1 {
        return nil, fmt.Errorf("request is not of type AdmissionReview v1")
    }

    if !apiequality.Semantic.DeepEqual(review.Request.Kind, ingressResource) {
        return nil, fmt.Errorf("rejecting admission review because the request does not contain an Ingress resource but %s with name %s in namespace %s",
            review.Request.Kind.String(), review.Request.Name, review.Request.Namespace)
    }

    status := &admissionv1.AdmissionResponse{}
    status.UID = review.Request.UID

    ingress := networking.Ingress{}

    codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
        Pretty: true,
    })
    codec.Decode(review.Request.Object.Raw, nil, nil)
    _, _, err := codec.Decode(review.Request.Object.Raw, nil, &ingress)
    if err != nil {
        klog.ErrorS(err, "failed to decode ingress")
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    if err := ia.Checker.CheckIngress(&ingress); err != nil {
        klog.ErrorS(err, "invalid ingress configuration", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
        status.Allowed = false
        status.Result = &metav1.Status{
            Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
            Message: err.Error(),
        }

        review.Response = status
        return review, nil
    }

    klog.InfoS("successfully validated configuration, accepting", "ingress", fmt.Sprintf("%v/%v", review.Request.Name, review.Request.Namespace))
    status.Allowed = true
    review.Response = status

    return review, nil
}

The core processing logic is actually the AdmissionReview sent when processing the request for Webhook, which will contain the resource objects to be verified. Then we verify or modify the resource object according to the actual needs.

Here are several points to note:

• The processing of Mutating Webhook is serial, while Validating Webhook is parallel;
• Although the processing of Mutating Webhook is serial, it does not guarantee the order;
• Note that the processing of Mutating Webhook should be idempotent to avoid the result not meeting the expectations;
• When processing a request, pay attention to processing all API versions of the resource object;

How to deploy administration webhook

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx-admission
webhooks:
  - name: validate.nginx.ingress.kubernetes.io
    matchPolicy: Equivalent
    rules:
      - apiGroups:
          - networking.k8s.io
        apiVersions:
          - v1
        operations:
          - CREATE
          - UPDATE
        resources:
          - ingresses
    failurePolicy: Fail
    sideEffects: None
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        namespace: ingress-nginx
        name: ingress-nginx-controller-admission
        path: /networking/v1/ingresses

Configure the specific connection information and trigger rules of webhooks in webhooks. In rules, you can specify which resources have specific actions in effect.

summary

This article mainly introduces the Admission Controller in Kubernetes. By default, some have been compiled with Kube apiserver in the form of plug-in. In addition, we can also write our own dynamic Admission Controller to complete the relevant requirements.

Of course, at present, there are many ready-made tools in K8s ecology to help us complete these corresponding things. In many cases, we don't need to develop corresponding services by ourselves. Later, I'll share some mainstream tools that can be used for Mutating and Validating access control. Welcome to pay attention.

Welcome to subscribe my official account number [MoeLove].