docker installs elk log analysis system

Install docker environment on win10

tip: win7/8

win7, win8 systems
 win7, win8, etc. need to be installed by docker toolbox. In China, you can download Ali cloud mirror at http://mirrors.aliyun.com/docker-toolbox/windows/docker-toolbox/

win10

Docker now has a dedicated installation package for the Win10 Professional Edition system and needs to turn on Hyper-V.

Programs and Functions - > Enable or Close Windows Functions - > Select Hyper-V

1. Install Toolbox

Download address for the latest version of Toolbox: https://www.docker.com/get-docker

click Download Desktop and Take a Tutorial And download the version of Windows. If you are not already logged in, you will be asked to sign in.

2. Run installation files

Double-click the downloaded Docker for Windows Installer installation file, Next all the way, and Finish to complete the installation.

installation is complete

Docker will start automatically.An icon for a small whale appears on the notification bar, which means the Docker is running.

We can execute docker version from the command line to see the version number, and docker run hello-world to load the test mirror test.

If not, you can search for Docker s on Windows to start.

Mirror Acceleration

Due to the domestic network problems, it is very slow to pull the Docker image in the future. We can configure the accelerator to solve this problem. I use NetEase's mirror address: http://hub-mirror.c.163.com.

1. Win10 can be configured by clicking the docker icon Daemon in Settings.
2. The new version of Docker configures Daemon using / etc/docker/daemon.json (Linux) or%programdata%\dockerconfig\daemon.json (Windows).

Add it to the profile (if you don't have one first):

{
  "registry-mirrors": ["http://hub-mirror.c.163.com"]
}

Install elk

Open the cmd\powershell and enter the docker command:

docker run --ulimit nofile=65536:65536 -p 5601:5601 -p 9200:9200 -p 5044:5044 -p 5045:5045 -p 5046:5046 -d --restart=always --name elk sebp/elk

The logstash configuration needs to be modified:

docker exec -it elk /bin/bash #Enter Container

cd etc/logstash/conf.d/

vim 02-beats-input.conf #Modify input configuration
--------------------------------------------cover
input {    
    tcp {         
        port => 5044         
        codec => json_lines     
    } 
} 
output{  
    elasticsearch { 
    hosts => ["localhost:9200"] 
    }  
}

Exit the container and restart elk

docker restart elk

Access localhost:5601 to enter kibana interface

Zipkin

The latest version of zipkin needs to be installed and can be accessed github of zipkin Get

Maven repository link: https://search.maven.org/remote_content?g=io.zipkin&a=zipkin-server&v=LATEST&c=exec

Download the jar package and run it

Download the jar package (version may change) and execute the command in the jar's directory:

java -jar zipkin-server-2.16.2-exec.jar --STORAGE_TYPE=elasticsearch --DES_HOSTS=http://ip:9200 (localhost is available if local)

--STORAGE_TYPE Indicate data service
--DES_HOSTS specified address

appendix

pom.xml dependency:

<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>5.2</version>
</dependency>

logback.xml configuration:

<?xml version="1.0" encoding="UTF-8"?>
<!--This log will have different log levels log Save the information in a different file -->
<configuration>
    <include resource="org/springframework/boot/logging/logback/defaults.xml" />
 
    <springProperty scope="context" name="springAppName"
                    source="spring.application.name" />
 
    <!-- Output location of log in project -->
    <property name="LOG_FILE" value="${BUILD_FOLDER:-build}/${springAppName}" />
 
    <!-- Log Output Style for Console -->
    <property name="CONSOLE_LOG_PATTERN"
              value="%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}" />
 
    <!-- console output -->
    <appender name="console" class="ch.qos.logback.core.ConsoleAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>INFO</level>
        </filter>
        <!-- Log Output Encoding -->
        <encoder>
            <pattern>${CONSOLE_LOG_PATTERN}</pattern>
            <charset>utf8</charset>
        </encoder>
    </appender>
 
    <!-- by logstash Output JSON Formatted Appender -->
    <appender name="logstash"
              class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>127.0.0.1:5044</destination>
        <!-- Log Output Encoding -->
        <encoder
                class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <timestamp>
                    <timeZone>UTC</timeZone>
                </timestamp>
                <pattern>
                    <pattern>
                        {
                        "severity": "%level",
                        "service": "${springAppName:-}",
                        "trace": "%X{X-B3-TraceId:-}",
                        "span": "%X{X-B3-SpanId:-}",
                        "exportable": "%X{X-Span-Export:-}",
                        "pid": "${PID:-}",
                        "thread": "%thread",
                        "class": "%logger{40}",
                        "rest": "%message"
                        }
                    </pattern>
                </pattern>
            </providers>
        </encoder>
    </appender>
 
    <!-- Log Output Level -->
    <root level="INFO">
        <appender-ref ref="console" />
        <appender-ref ref="logstash" />
    </root>
</configuration>

From blog.koreyoshi.work