There are two common solutions for pod log collection in k8s.

• Solution 1: Use fluentd as daemonset to collect all logs in stdout and / var/lib/containers directories (because fluentd is not familiar with, so it is troublesome);

• Solution 2: Use filebeat as sidecar (which is too cumbersome and requires adding this container to each pod)

Accidentally found that Aliyun open source log-pilot s collection of k8s logs is really super-convenient, simple configuration;

Official introduction:

github address: https://github.com/AliyunContainerService/log-pilot

Official introduction of log-pilot s: https://yq.aliyun.com/articles/674327

Log-pilots officially built: https://yq.aliyun.com/articles/674361?spm=a2c4e.11153940.0.0.21ae21c3mTKwWS

The daemonset file of log-pilot:

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: log-pilot
  labels:
    app: log-pilot
  #Setting up namespace for expected deployment
  namespace: kube-system
spec:
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: log-pilot
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      #Allow deployment to Master node
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      containers:
      - name: log-pilot
        #Refer to https://github.com/AliyunContainerService/log-pilot/releases for the version
        image: registry.cn-hangzhou.aliyuncs.com/acs/log-pilot:0.9.7-filebeat
        resources:
          limits:
            memory: 500Mi
          requests:
            cpu: 200m
            memory: 200Mi
        env:
          - name: "NODE_NAME"
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: "LOGGING_OUTPUT"
            value: "elasticsearch"
          #Ensure cluster access to ES network
          - name: "ELASTICSEARCH_HOSTS"
            value: "10.10.5.78:9200"
          #Configuring ES access rights
          #- name: "ELASTICSEARCH_USER"
          #  value: "{es_username}"
          #- name: "ELASTICSEARCH_PASSWORD"
          #  value: "{es_password}"
        volumeMounts:
        - name: sock
          mountPath: /var/run/docker.sock
        - name: root
          mountPath: /host
          readOnly: true
        - name: varlib
          mountPath: /var/lib/filebeat
        - name: varlog
          mountPath: /var/log/filebeat
        - name: localtime
          mountPath: /etc/localtime
          readOnly: true
        livenessProbe:
          failureThreshold: 3
          exec:
            command:
            - /pilot/healthz
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 2
        securityContext:
          capabilities:
            add:
            - SYS_ADMIN
      terminationGracePeriodSeconds: 30
      volumes:
      - name: sock
        hostPath:
          path: /var/run/docker.sock
      - name: root
        hostPath:
          path: /
      - name: varlib
        hostPath:
          path: /var/lib/filebeat
          type: DirectoryOrCreate
      - name: varlog
        hostPath:
          path: /var/log/filebeat
          type: DirectoryOrCreate
      - name: localtime
        hostPath:
          path: /etc/localtime

Create an nginx test pod Collection log example:

apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: node-affinity
spec:
  selector:
    matchLabels:
      app: node-affinity
  replicas: 3
  template:
    metadata:
      labels:
        app: node-affinity
    spec:
      containers:
      - name: nginx
        image: nginx
        imagePullPolicy: IfNotPresent
        env:
        - name: aliyun_logs_nginx
          value: "stdout"
---
apiVersion: v1
kind: Service
metadata:
  name: node-affinity
spec:
  selector:
    app: node-affinity
  ports:
  - port: 80
    targetPort: 80
  type: NodePort

Create an example of tomcat test pod Collection log:

apiVersion: v1
kind: Pod
metadata:
  name: tomcat
spec:
  containers:
  - name: tomcat
    image: "tomcat:8.0"
    env:
    #1. stdout is the agreed key word, representing the collection of standard output logs
    #2. Configure standard output log to catalina index of ES
    - name: aliyun_logs_catalina
      value: "stdout"
    #1. Configure file logs in collection containers to support wildcards
    #2. Configure the log to be collected under the access index of ES
    - name: aliyun_logs_access
      value: "/usr/local/tomcat/logs/catalina.*.log"
    #File log paths in containers need to be configured with emptyDir
    volumeMounts:
      - name: tomcat-log
        mountPath: /usr/local/tomcat/logs
  volumes:
    - name: tomcat-log
      emptyDir: {}