k8s ~ add sidecar sidebucket for service

Keywords: Java ElasticSearch Kubernetes Nginx JSON

The word sidecar generally refers to motorcycles with straddles. In the Second World War, small Japan drove many of these motorcycles. It added a straddle on the original basis, and then it could carry an extra person. It had no impact on the original two wheeled motorcycles. It was also OK to dismantle the straddle. It had no essential damage to the original things, just expanded the new functions. This Similar to the COP principle in software development, there is also this concept in istio of service grid. It calls this component "sidecar". In istio, sidecar is just a concept, specifically implemented by envy.

sidecar of specific fluent function

Our containers are deployed in k8s, and we manage our containers through k8s to realize life cycle management, service discovery management, multi copy management, etc. we can make these containers ideal for one by one micro service, and the logs of these services are generally recorded locally, and then pushed to elastic search, and we can choose fluent and fil as the log collection tool Ebeat, Logstash, etc.

Add the sidecar of fluent

Add fluent.config configuration

 	<source>
type tail
format json
path /var/log/*.log
pos_file /var/log/log.pos
tag saas
</source>

<match **>
@id elasticsearch
@type elasticsearch
@log_level debug
index_name fluentd
type_name _doc
host elasticsearch.elk
port 9200
include_tag_key true
tag_key @log_name
logstash_format true
flush_interval 10s
</match>

Add sidecar to the deployment file of the service

kind: Service
apiVersion: v1
metadata:
  name: hello-world
  namespace: saas
spec:
  selector:
    app: hello-world
  type: ClusterIP
  ports:
    - protocol: TCP
      targetPort: 9001
      port: 80
---
# Building a reflection agent
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: hello-world-ingress
  namespace: saas
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
    - hosts:
        - www.abc.com
      secretName: saas-tls
  rules:
    - host: www.abc.com
      http:
        paths:
          - backend:
              serviceName: hello-world
              servicePort: 9001
          - path: /dotnet
            backend:
              serviceName: dotnet-hello
              servicePort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: hello-world-deployment
  namespace: saas
  labels:
    app: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello-world
  template:
    metadata:
      labels:
        app: hello-world
    spec:
      containers:
        - name: hello-world
          image: 172.17.0.22:8888/saas/hello-world:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 9001
          env:
            - name: spring.profiles.active
              value: prod
          volumeMounts:
            - name: varlog
              mountPath: /var/log
        - name: fluent-sidecar
          image: registry.cn-beijing.aliyuncs.com/k8s-mqm/fluentd-elasticsearch:v2.1.0
          env:
            - name: FLUENTD_ARGS
              value: -c /etc/fluentd-config/fluentd.conf
          volumeMounts:
            - name: varlog
              mountPath: /var/log
            - name: config-volume
              mountPath: /etc/fluentd-config
      volumes:
        - name: varlog
          emptyDir: {}
        - name: config-volume
          configMap:
            name: fluentd-config

When your Hello world is deployed to k8s, it will write to the / var/logs directory when there is a log record, and the side card of fluent can also read the contents of the log because it is a disk used for container consumption, and then send the log to elastic search.

Posted by Azarian on Tue, 24 Mar 2020 00:49:01 -0700