Distributed ELK platform

What is ELK? ELK is a complete solution, which is the acronym of three software products ELK represents Elasticsearch: responsible for log retrieval and storage Logstash: responsible for the collection, analysis and processing of logs Kibana: responsible for the visualization of logs These three k ...

Posted by arjuna on Sat, 22 Feb 2020 03:03:17 -0800

Using Python script to clean up es data

Demand background The business system stores all kinds of reports and statistical data in ES. Due to historical reasons, the system makes statistics in full amount every day. As time goes on, the data storage space of ES is under great pressure. At the same time, because the use of ES index is not well planned, some indexes even exceed the max ...

Posted by beefy23 on Fri, 21 Feb 2020 03:37:14 -0800

Docker installs Logstash and connects to Elasticsearch

1. Pull the Logstash image (keep consistent with ES version) docker pull logstash:7.5.1 2. Start the container docker run --name logstash -d -p 5044:5044 --net esnet 8b94897b4254 The network set by -- net in the command should be consistent with ES and kibana 3. Modify Logstash configuration file // 0.0.0.0: allow any IP access http.host: "0.0 ...

Posted by olivarespablo on Wed, 19 Feb 2020 08:42:59 -0800

Graylog building configuration details (centos7)

Graylog build configuration (centos7) 1, Basic environment 1. jdk environment required: [root@iz2zee3zwuvnmai605c99vz ~]# java -version openjdk version "1.8.0_171" OpenJDK Runtime Environment (build 1.8.0_171-b10) OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode) //If not, you need to install it. He ...

Posted by thepreacher on Sat, 15 Feb 2020 07:29:27 -0800

Install the Elastic Stack component (elastic search, logstash, kibana, filebeat) error report brocade set

Error 1: kibana login error, check the log and find: crypt_r() failed $ cat /var/log/nginx/error.log 2020/01/28 12:11:38 [crit] 8863#8863: *3 crypt_r() failed (22: Invalid argument), client: 10.20.44.113, server: xx.xx.xx.xx, request: "GET /status HTTP/1.1", host: "xx.xx.xx.xx" Cause analysis # Whe ...

Posted by jenniferG on Mon, 03 Feb 2020 06:54:32 -0800

Operation and maintenance | Elastic Stack ELK installation and deployment

Preface ELK is the abbreviation of three kinds of software, which are composed of Elasticsearch, Logstash and Kibana. In the process of development, there are new members, Beats, which form Elastic Stack. So, ELK is the old name, and Elastic Stack is the new name. The entire Elastic Stack technology ...

Posted by deezerd on Tue, 21 Jan 2020 08:57:35 -0800

Logstash and filebeat configuration

    The mutate plug-in can modify the data in the event, including rename, update, replace, convert, split, gsub, uppercase, lowercase, strip, remove field, join, merge and other functions. 1,rename For a field that already exists, rename its field name.   filter { mutate { rename = ...

Posted by fiorefrank on Thu, 16 Jan 2020 03:45:15 -0800

How does Elasticsearch handle data that has an association?

Three main paradigms of relational databases What is a paradigm? A paradigm is the rule of data modeling. First paradigm: Ensure that each column maintains atomicity. All fields in a database table are indivisible atomic values. Second paradigm: Ensure that each column in the table is related to the ...

Posted by brmcdani on Tue, 14 Jan 2020 16:43:11 -0800

Using the Java API of elastic search to query

1. Preface elsaticsearch version is 6.8.3. The Java API used is based on Java High Level REST Client 2. data 3.  InitClient Used to initialize clients package com.htkj.elasticsearch; import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestHighLevelClient; public class Ini ...

Posted by christa on Mon, 13 Jan 2020 02:04:58 -0800

Chapter 16 nine analysis takes you through helm3 efk - elastic search

This series: Chapter 1: Nine analysis takes you easy to finish the installation of helm3 The second chapter: Nine analysis takes you to the public warehouse of helm3 The third chapter: Nine analysis takes you through the explosion of helm3 private warehouse easily The fourth chapter: Nine analysis takes you easy The fifth chapter: Nine analy ...

Posted by bakigkgz on Sat, 11 Jan 2020 00:15:20 -0800