concept
In short, Sidecar injection adds the configuration of additional containers to the Pod template. This refers to the Pod in which the Envoy container is applied.
Currently, the containers required by Istio service grid are:
Istio init is used to set iptables rules so that inbound / outbound traffic passes through the Sidecar agent.
Initialization containers differ from application containers in the following ways:
- It runs before the application container is started, and it runs until it is finished.
- If there are multiple initialization containers, each container should complete successfully before starting the next one.
So you can see how perfect this container is for setup or initialization jobs that don't need to be part of the actual application container. In this case, istio init does and sets the iptables rule.
The container of istio proxy is the real Sidecar proxy (based on envy).
The following describes two ways to inject Istio Sidecar into a pod:
- Manual injection using istioctl
- Enable automatic injection of the Istio Sidecar injector for the namespace to which the pod belongs.
Manual injection directly modifies the configuration, such as deployment, and injects the agent configuration into it.
When auto injection is enabled for the namespace to which the pod belongs, the auto injector will use the admission controller to automatically inject the agent configuration when the pod is created.
Inject by applying the template defined in istio sidecar injector configmap.
Automatic injection
When you set the istio-injection=enabled tag in a namespace and the injection webhook is enabled, any new pod will automatically add Sidecar. please note that unlike manual injection, automatic injection occurs at the pod level. You will not see any changes in the deployment itself.
kubectl label namespace default istio-inhection=enabled kubectl get namespace -L istio-injection NAME STATUS AGE ISTIO-INJECTION default Active 1h enabled istio-system Active 1h kube-public Active 1h kube-system Active 1h
Injection occurs when a pod is created. Kill the running pod and verify that the newly created pod is injected into sidecar. The original pod has a container with a READY of 1 / 1, while the pod injected with sidecar has a container with a READY of 2 / 2.
Principle of automatic injection
Automatic injection is realized by using k8s permission webhook. Permission webhook is an HTTP callback mechanism used to receive and process access requests. It can change the objects sent to the API server to perform custom default setting operations. Details can be found Admission webhook file.
Istio sidecar injector webhook configuration corresponding to istio will call back the / inject address of istio sidecar injector service by default.
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
name: istio-sidecar-injector
webhooks:
- name: sidecar-injector.istio.io
clientConfig:
service:
name: istio-sidecar-injector
namespace: istio-system
path: "/inject"
caBundle: ${CA_BUNDLE}
rules:
- operations: [ "CREATE" ]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
namespaceSelector:
matchLabels:
istio-injection: enabled
The callback API entry code is in pkg/kube/inject/webhook.go in
// Create a new instance for automatic sidecar injection
func NewWebhook(p WebhookParameters) (*Webhook, error) {
// ... omit ten thousand words
wh := &Webhook{
Config: sidecarConfig,
sidecarTemplateVersion: sidecarTemplateVersionHash(sidecarConfig.Template),
meshConfig: p.Env.Mesh(),
configFile: p.ConfigFile,
valuesFile: p.ValuesFile,
valuesConfig: valuesConfig,
watcher: watcher,
healthCheckInterval: p.HealthCheckInterval,
healthCheckFile: p.HealthCheckFile,
env: p.Env,
revision: p.Revision,
}
//api server callback function, listening for / inject callback
p.Mux.HandleFunc("/inject", wh.serveInject)
p.Mux.HandleFunc("/inject/", wh.serveInject)
// ... omit ten thousand words
return wh, nil
}
serveInject logic
func (wh *Webhook) serveInject(w http.ResponseWriter, r *http.Request) {
// ... omit ten thousand words
var reviewResponse *v1beta1.AdmissionResponse
ar := v1beta1.AdmissionReview{}
if _, _, err := deserializer.Decode(body, nil, &ar); err != nil {
handleError(fmt.Sprintf("Could not decode body: %v", err))
reviewResponse = toAdmissionResponse(err)
} else {
//Execute specific inject logic
reviewResponse = wh.inject(&ar, path)
}
// Respond to the content after inject sidecar to k8s api server
response := v1beta1.AdmissionReview{}
if reviewResponse != nil {
response.Response = reviewResponse
if ar.Request != nil {
response.Response.UID = ar.Request.UID
}
}
// ... omit ten thousand words
}
// Injection logic implementation
func (wh *Webhook) inject(ar *v1beta1.AdmissionReview, path string) *v1beta1.AdmissionResponse {
// ... omit ten thousand words
// injectRequired determines whether automatic injection is set
if !injectRequired(ignoredNamespaces, wh.Config, &pod.Spec, &pod.ObjectMeta) {
log.Infof("Skipping %s/%s due to policy check", pod.ObjectMeta.Namespace, podName)
totalSkippedInjections.Increment()
return &v1beta1.AdmissionResponse{
Allowed: true,
}
}
// ... omit ten thousand words
// Return the object to be injected into Pod
spec, iStatus, err := InjectionData(wh.Config.Template, wh.valuesConfig, wh.sidecarTemplateVersion, typeMetadata, deployMeta, &pod.Spec, &pod.ObjectMeta, wh.meshConfig, path) // nolint: lll
if err != nil {
handleError(fmt.Sprintf("Injection data: err=%v spec=%vn", err, iStatus))
return toAdmissionResponse(err)
}
// Execute container injection logic
patchBytes, err := createPatch(&pod, injectionStatus(&pod), wh.revision, annotations, spec, deployMeta.Name, wh.meshConfig)
if err != nil {
handleError(fmt.Sprintf("AdmissionResponse: err=%v spec=%vn", err, spec))
return toAdmissionResponse(err)
}
reviewResponse := v1beta1.AdmissionResponse{
Allowed: true,
Patch: patchBytes,
PatchType: func() *v1beta1.PatchType {
pt := v1beta1.PatchTypeJSONPatch
return &pt
}(),
}
return &reviewResponse
}
injectRequired function
func injectRequired(ignored []string, config *Config, podSpec *corev1.PodSpec, metadata *metav1.ObjectMeta) bool {
// HostNetwork mode skip injection directly
if podSpec.HostNetwork {
return false
}
// k8s system namespace (Kube system / Kube public) skip injection
for _, namespace := range ignored {
if metadata.Namespace == namespace {
return false
}
}
annos := metadata.GetAnnotations()
if annos == nil {
annos = map[string]string{}
}
var useDefault bool
var inject bool
// Whether the priority judgment is stated` sidecar.istio.io/inject `Annotation, overwriting naming configuration
switch strings.ToLower(annos[annotation.SidecarInject.Name]) {
case "y", "yes", "true", "on":
inject = true
case "":
// Use namespace configuration
useDefault = true
}
// Specifies that Pod does not need a label selector injected into Sidecar
if useDefault {
for _, neverSelector := range config.NeverInjectSelector {
selector, err := metav1.LabelSelectorAsSelector(&neverSelector)
if err != nil {
} else if !selector.Empty() && selector.Matches(labels.Set(metadata.Labels))
// Setup does not require injection
inject = false
useDefault = false
break
}
}
}
// Always inject sidecar into the pod matching the tag selector, ignoring the global policy
if useDefault {
for _, alwaysSelector := range config.AlwaysInjectSelector {
selector, err := metav1.LabelSelectorAsSelector(&alwaysSelector)
if err != nil {
log.Warnf("Invalid selector for AlwaysInjectSelector: %v (%v)", alwaysSelector, err)
} else if !selector.Empty() && selector.Matches(labels.Set(metadata.Labels)){ // Setup requires injection
inject = true
useDefault = false
break
}
}
}
// Use default injection policy if none is configured
var required bool
switch config.Policy {
default: // InjectionPolicyOff
log.Errorf("Illegal value for autoInject:%s, must be one of [%s,%s]. Auto injection disabled!",
config.Policy, InjectionPolicyDisabled, InjectionPolicyEnabled)
required = false
case InjectionPolicyDisabled:
if useDefault {
required = false
} else {
required = inject
}
case InjectionPolicyEnabled:
if useDefault {
required = true
} else {
required = inject
}
}
return required
}
From the above we can see whether the priority of Sidecar injection is
Pod Annotations → NeverInjectSelector → AlwaysInjectSelector → Default Policy
createPath function
func createPatch(pod *corev1.Pod, prevStatus *SidecarInjectionStatus, revision string, annotations map[string]string,
sic *SidecarInjectionSpec, workloadName string, mesh *meshconfig.MeshConfig) ([]byte, error) {
var patch []rfc6902PatchOperation
// ... omit ten thousand words
// Injection initialization start container
patch = append(patch, addContainer(pod.Spec.InitContainers, sic.InitContainers, "/spec/initContainers")...)
// Inject Sidecar container
patch = append(patch, addContainer(pod.Spec.Containers, sic.Containers, "/spec/containers")...)
// Inject mount volume
patch = append(patch, addVolume(pod.Spec.Volumes, sic.Volumes, "/spec/volumes")...)
patch = append(patch, addImagePullSecrets(pod.Spec.ImagePullSecrets, sic.ImagePullSecrets, "/spec/imagePullSecrets")...)
// Inject new comments
patch = append(patch, updateAnnotation(pod.Annotations, annotations)...)
// ... omit ten thousand words
return json.Marshal(patch)
}
Summary: it can be seen that the whole injection process is actually the original Pod configuration is de parsed into a Pod object, the yaml content to be injected (such as Sidecar) is de sequenced into an object, and then it is append ed to the corresponding Pod (such as Container), and then the modified Pod is re parsed into yaml content and returned to the api server of k8s, and then k8s Take the modified content and schedule the two containers to the same machine for deployment, so that the corresponding Sidecar injection is completed.
Uninstall sidecar auto injector
kubectl delete mutatingwebhookconfiguration istio-sidecar-injector kubectl -n istio-system delete service istio-sidecar-injector kubectl -n istio-system delete deployment istio-sidecar-injector kubectl -n istio-system delete serviceaccount istio-sidecar-injector-service-account kubectl delete clusterrole istio-sidecar-injector-istio-system kubectl delete clusterrolebinding istio-sidecar-injector-admin-role-binding-istio-system
The above command does not remove the injected sidecar from the pod. You need to make a rolling update or delete the corresponding pod directly, and force deployment to recreate the new pod.
Manual injection of sidecar
Manual injection of deployment requires the use of istioctl Kube inject
istioctl kube-inject -f samples/sleep/sleep.yaml | kubectl apply -f -
By default, the configuration within the cluster will be used, or the local copy of the configuration will be used to complete the injection.
kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath='{.data.config}' > inject-config.yaml
kubectl -n istio-system get configmap istio-sidecar-injector -o=jsonpath='{.data.values}' > inject-values.yaml
kubectl -n istio-system get configmap istio -o=jsonpath='{.data.mesh}' > mesh-config.yaml
Specify the input file, run Kube inject and deploy
istioctl kube-inject
--injectConfigFile inject-config.yaml
--meshConfigFile mesh-config.yaml
--valuesFile inject-values.yaml
--filename samples/sleep/sleep.yaml
| kubectl apply -f -
Verify that sidecar has been injected into 2 / 2 of the sleep pod under the READY column
kubectl get pod -l app=sleep NAME READY STATUS RESTARTS AGE sleep-64c6f57bc8-f5n4x 2/2 Running 0 24s
Principle of manual injection
The code entry for manual injection is in istioctl/cmd/kubeinject.go
There are some differences between manual injection and automatic injection. Manual injection changes the Deployment. Let's see what it does:
Configuration before Deployment injection:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
spec:
replicas: 7
selector:
matchLabels:
app: hello
tier: backend
track: stable
template:
metadata:
labels:
app: hello
tier: backend
track: stable
spec:
containers:
- name: hello
image: "fake.docker.io/google-samples/hello-go-gke:1.0"
ports:
- name: http
containerPort: 80
Configuration after Deployment injection:
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
name: hello
spec:
replicas: 7
selector:
matchLabels:
app: hello
tier: backend
track: stable
strategy: {}
template:
metadata:
annotations:
sidecar.istio.io/status: '{"version":"2343d4598565fd00d328a3388421ee637d25d3f7068e7d5cadef374ee1a06b37","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":null,"imagePullSecrets":null}'
creationTimestamp: null
labels:
app: hello
istio.io/rev: ""
security.istio.io/tlsMode: istio
tier: backend
track: stable
spec:
containers:
- image: fake.docker.io/google-samples/hello-go-gke:1.0
name: hello
ports:
- containerPort: 80
name: http
resources: {}
- image: docker.io/istio/proxy_debug:unittest
name: istio-proxy
resources: {}
initContainers:
- image: docker.io/istio/proxy_init:unittest-test
name: istio-init
resources: {}
securityContext:
fsGroup: 1337
status: {}
---
You can add a container image
- image: docker.io/istio/proxy_debug:unittest
name: istio-proxy
resources: {}
There are two options for where to get the injected content template.
- - injectConfigFile specifies the corresponding injection file
- - injectConfigMapName the ConfigMap name of the injection configuration
If it is found that Sidecar is not injected successfully during the operation, you can check the injection process above to find out the problem according to the way of injection.
reference
https://kubernetes.io/zh/docs/reference/access-authn-authz/admission-controllers/
https://istio.io/zh/docs/reference/commands/istioctl/#istioctl-kube-inject