Distributed ELK platform
What is ELK?
ELK is a complete solution, which is the acronym of three software products
ELK represents
Elasticsearch: responsible for log retrieval and storage
Logstash: responsible for the collection, analysis and processing of logs
Kibana: responsible for the visualization of logs
These three k ...
Posted by arjuna on Sat, 22 Feb 2020 03:03:17 -0800
Using Python script to clean up es data
Demand background
The business system stores all kinds of reports and statistical data in ES. Due to historical reasons, the system makes statistics in full amount every day. As time goes on, the data storage space of ES is under great pressure. At the same time, because the use of ES index is not well planned, some indexes even exceed the max ...
Posted by beefy23 on Fri, 21 Feb 2020 03:37:14 -0800
Docker installs Logstash and connects to Elasticsearch
1. Pull the Logstash image (keep consistent with ES version)
docker pull logstash:7.5.1
2. Start the container
docker run --name logstash -d -p 5044:5044 --net esnet 8b94897b4254
The network set by -- net in the command should be consistent with ES and kibana
3. Modify Logstash configuration file
// 0.0.0.0: allow any IP access
http.host: "0.0 ...
Posted by olivarespablo on Wed, 19 Feb 2020 08:42:59 -0800
Graylog building configuration details (centos7)
Graylog build configuration (centos7)
1, Basic environment
1. jdk environment required:
[root@iz2zee3zwuvnmai605c99vz ~]# java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)
//If not, you need to install it. He ...
Posted by thepreacher on Sat, 15 Feb 2020 07:29:27 -0800
Install the Elastic Stack component (elastic search, logstash, kibana, filebeat) error report brocade set
Error 1: kibana login error, check the log and find: crypt_r() failed
$ cat /var/log/nginx/error.log
2020/01/28 12:11:38 [crit] 8863#8863: *3 crypt_r() failed (22: Invalid argument), client: 10.20.44.113, server: xx.xx.xx.xx, request: "GET /status HTTP/1.1", host: "xx.xx.xx.xx"
Cause analysis
# Whe ...
Posted by jenniferG on Mon, 03 Feb 2020 06:54:32 -0800
Operation and maintenance | Elastic Stack ELK installation and deployment
Preface
ELK is the abbreviation of three kinds of software, which are composed of Elasticsearch, Logstash and Kibana. In the process of development, there are new members, Beats, which form Elastic Stack. So, ELK is the old name, and Elastic Stack is the new name.
The entire Elastic Stack technology ...
Posted by deezerd on Tue, 21 Jan 2020 08:57:35 -0800
Logstash and filebeat configuration
The mutate plug-in can modify the data in the event, including rename, update, replace, convert, split, gsub, uppercase, lowercase, strip, remove field, join, merge and other functions.
1,rename
For a field that already exists, rename its field name.
filter {
mutate {
rename = ...
Posted by fiorefrank on Thu, 16 Jan 2020 03:45:15 -0800
How does Elasticsearch handle data that has an association?
Three main paradigms of relational databases
What is a paradigm? A paradigm is the rule of data modeling.
First paradigm: Ensure that each column maintains atomicity.
All fields in a database table are indivisible atomic values.
Second paradigm: Ensure that each column in the table is related to the ...
Posted by brmcdani on Tue, 14 Jan 2020 16:43:11 -0800
Using the Java API of elastic search to query
1. Preface
elsaticsearch version is 6.8.3. The Java API used is based on Java High Level REST Client
2. data
3. InitClient
Used to initialize clients
package com.htkj.elasticsearch;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
public class Ini ...
Posted by christa on Mon, 13 Jan 2020 02:04:58 -0800
Chapter 16 nine analysis takes you through helm3 efk - elastic search
This series:
Chapter 1: Nine analysis takes you easy to finish the installation of helm3
The second chapter: Nine analysis takes you to the public warehouse of helm3
The third chapter: Nine analysis takes you through the explosion of helm3 private warehouse easily
The fourth chapter: Nine analysis takes you easy
The fifth chapter: Nine analy ...
Posted by bakigkgz on Sat, 11 Jan 2020 00:15:20 -0800