1 Introduction
If you often use Kubernetes, you must have the need to customize the resource list file. However, it seems that everyone likes to use helm. Helm is easy to use, but it also has many disadvantages. For example, you need a tiller server, which requires ultra-high permissions. Most importantly, if you want to make a Helm Chart package by yourself, it is not so easy, You need to know something about go template. It discards some logic we learned from Docker and Kubernetes. Today we will introduce another one called Kustomize ❤️ Alternative tools.
In fact, Kustomize is not a new tool, and now it has been integrated into the subcommand of kubectl version 1.14. Is it very convenient and avoids the trouble of installing third-party tools, because kubectl tools are basically used every day, so... You can throw away the Helm command 😉.
Like Kubernetes, Kustomize is completely declarative. The system will provide you with what you say you want. You don't need to follow the command to describe the object you want to build.
Secondly, it is similar to Docker. It consists of many layers. Each layer modifies the previous layer. Because of this concept, we can constantly write things to others without increasing the complexity of configuration. The final result of construction consists of the basic part and other layers you configure above.
Finally, like Git, you can use a remote basic configuration as the original configuration, and then add some custom configurations on this basis;
2 installation
For 🍎 For Mac users, you can use brew to install directly:
brew install kustomize
Of course, if you are using other operating systems, you can directly start from Release page Download the binary file above and add it to the PATH below. Of course, if you like, you can also build the code warehouse directly from the source code: https://github.com/Kubernetes-sigs/kustomize.
3 Foundation formwork
To use Kustomize, you need an original yaml file to describe any resources you want to deploy to the cluster. Here, we store these base files under the. / k8s/base / folder;
We will never access these files directly. We will add some custom configurations to them to create new resource definitions.
You can use the kubectl apply - F. / k8s / base / command to build the base template at any point in time.
In the following example, we will use Service and Deployment resource objects as examples. The following two resource manifest files are defined: service.yaml
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-appdeployment.yaml is defined as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
selector:
matchLabels:
app: sl-demo-app
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- name: app
image: foo/bar:latest
ports:
- name: http
containerPort: 8080
protocol: TCPThen add a file named kustomization.yaml in the current folder:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - service.yaml - deployment.yaml
This file will be your basic configuration file, which describes the resource file you use;
When you run the kubectl apply - F. / k8s / base / command, some errors may appear in the kustomization.yaml file. You can add the parameter -- validate=false for verification. Of course, you can not run the command for the entire folder.
To install the resources in the basic template into your cluster, just execute the following command:
$ kubectl apply -k k8s/base service/sl-demo-app created deployment.apps/sl-demo-app created
In order to understand what resources will be installed into the cluster, we mainly use the kustomize build command instead of the kubectl apply -k command in this article. Of course, it is also possible to use the kubectl kusomize command, because we said that kusomize has been integrated since kubectl version 1.14.
The results after running the kustomize build command are as follows. We will see that the two files are connected together:
$ kustomize build k8s/base
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
selector:
matchLabels:
app: sl-demo-app
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCP4 customization
Now we want to customize for some specific scenarios. For example, different configurations are required for production environment and test environment. We will not cover the entire feature set of Kustomize here, but as a standard example to show you the philosophy behind this tool.
First, we create a new folder k8s/overlays/prod, which contains a file named kustomzification.yaml. The contents of the file are as follows:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base
The directory structure under the current folder is as follows:
$ tree
.
└── k8s
├── base
│ ├── deployment.yaml
│ ├── kustomization.yaml
│ └── service.yaml
└── overlays
└── prod
└── kustomization.yamlIf we build this file now, we will see the same result as the base directory built before:
$ kustomzie build k8s/overlays/prod
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
selector:
matchLabels:
app: sl-demo-app
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPNext, let's customize our prod environment;
4.1 defining environment variables
In the base basic template, we define any environment variables. Now we need to add some environment variables to the previous basic template. In fact, it's very simple. We just need to create a code block we want to template on our basic template, and then reference it to the kustomization.yaml file;
For example, we define a configuration file containing environment variables: (custom-env.yaml);
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
template:
spec:
containers:
- name: app # (1)
env:
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️Note (1) the name defined here is very important. kustomize will find the container to be modified through this value.
The yaml file itself is invalid. It only describes what we want to add to the above basic template. We just need to add this file to the k8s/overlays/prod/kustomization.yaml file:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - custom-env.yaml
Now, if we build the following, we can see the following output results:
$ kustomize build k8s/overlays/prod
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
selector:
matchLabels:
app: sl-demo-app
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- env:
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️
image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPYou can see that our env block has been merged into our basic template, and the customized env variable appears in the deployment.yaml file.
4.2 number of modified copies
Like the above example, let's extend our basic template to define some undefined variables.
You can also overwrite some existing variables in the base file.
Here, let's add some information about the replica. As before, we only need to define the additional information blocks required for the replica in a YAML file, and create a new file named replica-and-rollout-strategy.yaml. The content is as follows:
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
replicas: 10
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdateAs before, add the data customized here under the patchesStrategicMerge in the kustomization.yaml file:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - custom-env.yaml - replica-and-rollout-strategy.yaml
Similarly, use the kustomize build command to build at this time, as shown below:
$ kustomize build k8s/overlays/prod
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
replicas: 10
selector:
matchLabels:
app: sl-demo-app
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- env:
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️
image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPWe can see that the number of copies and the rolling update strategy have been added to the basic template:
4.3 define secret from the command line
We often add a secret object through the command line. kustomize has an edit subcommand that can be used to edit the kustomization.yaml file and create a secret object. For example, we add a secret object as follows:
$ cd k8s/overlays/prod $ kustomize edit add secret sl-demo-app --from-literal=db-password=12345
The above command will modify the kustomization.yaml file and add a secret generator field in it.
Of course, you can also create secret objects through files (such as -- from file = file / path or -- from evn file = env / path. Env).
After creating the secret object through the above command, the contents of the kustomization.yaml file are as follows:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - custom-env.yaml - replica-and-rollout-strategy.yaml secretGenerator: - literals: - db-password=12345 name: sl-demo-app type: Opaque
Then, similarly, we go back to the root directory and execute the kustomize build command to build the template. The output is as follows:
$ kustomize build k8s/overlays/prod
apiVersion: v1
data:
db-password: MTIzNDU=
kind: Secret
metadata:
name: sl-demo-app-6ft88t2625
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
replicas: 10
selector:
matchLabels:
app: sl-demo-app
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- env:
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️
image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPWe can see that the name of the secret object is sl-demo-app-6ft88t2625 instead of the sl-demo-app we defined, which is normal, because if the secret content is changed, the rolling update can be triggered;
Similarly, if we want to use this Secret object in Deployment, we can add a new layer definition using Secret as before.
For example, if we inject the value of DB password into Deployment through environment variables, we can define the following new layer information: (database secret. Yaml)
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
template:
spec:
containers:
- name: app
env:
- name: "DB_PASSWORD"
valueFrom:
secretKeyRef:
name: sl-demo-app
key: db.passwordThen, similarly, we add the layers defined here to the k8s/overlays/prod/kustomization.yaml file:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - custom-env.yaml - replica-and-rollout-strategy.yaml - database-secret.yaml secretGenerator: - literals: - db-password=12345 name: sl-demo-app type: Opaque
Now let's build the whole prod directory, and we will get the following information:
$ kustomize build k8s/overlays/prod
apiVersion: v1
data:
db-password: MTIzNDU=
kind: Secret
metadata:
name: sl-demo-app-6ft88t2625
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
replicas: 10
selector:
matchLabels:
app: sl-demo-app
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
key: db.password
name: sl-demo-app-6ft88t2625
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️
image: foo/bar:latest
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPWe can see that the value of secretKeyRef.name has also been modified to the name of the secret object generated above.
Since secret is some private information, it is best to add the above secret object in a secure environment, rather than submit it to the code warehouse together with other code.
The same logic applies to ConfigMap. Finally, a hash value name will be generated, so that redeployment can be triggered when ConfigMap is changed;
4.4 modifying images
Like the secret resource object, we can directly change the image or tag from the command line. This is very useful if you need to deploy the image marked through the CI/CD system.
For example, let's modify the image tag:
$ cd k8s/overlays/prod $ TAG_VERSION=3.4.5 $ kustomize edit set image foo/bar=foo/bar:$TAG_VERSION
In general, TAG_VERSION is often defined in CI/CD systems.
The contents of the kustomization.yaml file are as follows:
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../base patchesStrategicMerge: - custom-env.yaml - replica-and-rollout-strategy.yaml - database-secret.yaml secretGenerator: - literals: - db-password=12345 name: sl-demo-app type: Opaque images: - name: foo/bar newName: foo/bar newTag: 3.4.5
Similarly, when you go back to the root directory to build the template, you will get the following information:
$ kustomize build k8s/overlays/prod
apiVersion: v1
data:
db-password: MTIzNDU=
kind: Secret
metadata:
name: sl-demo-app-6ft88t2625
type: Opaque
---
apiVersion: v1
kind: Service
metadata:
name: sl-demo-app
spec:
ports:
- name: http
port: 8080
selector:
app: sl-demo-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: sl-demo-app
spec:
replicas: 10
selector:
matchLabels:
app: sl-demo-app
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: sl-demo-app
spec:
containers:
- env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
key: db.password
name: sl-demo-app-6ft88t2625
- name: CUSTOM_ENV_VARIABLE
value: Value defined by Kustomize ❤️
image: foo/bar:3.4.5
name: app
ports:
- containerPort: 8080
name: http
protocol: TCPWe can see that the first container.image of the Deployment has been modified to version 3.4.5.
Finally, our customized template file directory structure is as follows:
$ tree .
.
└── k8s
├── base
│ ├── deployment.yaml
│ ├── kustomization.yaml
│ └── service.yaml
└── overlays
└── prod
├── custom-env.yaml
├── database-secret.yaml
├── kustomization.yaml
└── replica-and-rollout-strategy.yaml
4 directories, 7 filesIt is also easy to install into a cluster:
kustomize build k8s/overlays/prod | kubectl apply -f -
5 Summary
In the above example, we learned how to use the powerful function of Kustomize to define your Kuberentes resource manifest file without using any additional template system. All modified block files created will be applied to the original basic template file without using modifications such as curly braces to change it (seems to despise Helm 😄).
There are many other advanced uses in Kustomize, such as mixins and inheritance, or allowing you to define a name, label or namespace for each created object. You can use it in the official Kustomize GitHub code warehouse View advanced samples and documentation in.