Automatically clean up elasticsearch index on a regular basis

Keywords: Linux ElasticSearch curl crontab

Catalog

1.1 how to view index

1.2 manual index deletion method

1.3 script cleanup index

1.4 scheduled tasks

1.1 how to view index

Explain:

01: I have installed elasticsearch, kibana and logstash on one machine

02:192.168.10.138 is the intranet IP address monitored by elasticsearch

03:9200 is one of the ports of elasticsearch


#How to view indexes on elastic search server

1.png

#Sort index

2.png

1.2 manual index deletion method

#Manually delete15_tomcat_18080_catalina_log-2018.12.28Indexes

curl -XDELETE "http://192.168.10.138:9200/15_tomcat_18080_catalina_log-2018.12.28"


#Manually delete15_tomcat_18080_access_log-2018.12.28Indexes

curl -XDELETE "http://192.168.10.138:9200/15_tomcat_18080_access_log-2018.12.28"

1.3 script cleanup index

#Be careful

01: After the script, it needs to be added to the scheduled task to automatically clean the index

02: If elasticsearch After running for some time, the data 7 days ago cannot be deleted

03: The script is to delete the index on the 7th day before the current time (there is no need to have some rules for the name of the index)

04: So it needs to be deployed elasticsearch After that, add the script to the scheduled task.

[root@node-1 scripts]# cat delete_elk_indexs.sh

#!/bin/bash
#
# Define variables
RETVAL=0
Port=9200
Ip=192.168.10.138
Time1=$(date +%Y.%m.%d)
Time2=$(date +%Y.%m.%d -d "-7day")

# Determine the user to execute
if [ "$UID" -ne "$RETVAL" ];then
  echo "Must be root to run scripts"
  exit 1
fi

# Load local functions
[ -f /etc/init.d/functions ] && source /etc/init.d/functions

# Delete index 7 days ago
curl -XDELETE "http://$Ip:$Port/*${Time2}" >/dev/null 2>&1
RETVAL=$?
if [ "$RETVAL" -eq 0 ];then
  action "delete elk 7 day ago index" /bin/true
 else
  action "delete elk 7 day ago index" /bin/false
fi

# Scripts return values
exit $RETVAL

#Script execution results

[root@ node-1 scripts]# sh delete_elk_indexs.sh

delete elk 7 day ago index [ OK ]

1.4 scheduled tasks

[root@ node-1 scripts]# crontab -l|tail -2

# Crond delete elk 7 day ago index. USER:chenliang TIME:2019-01-03

00 12 * * * /bin/sh /server/scripts/delete_elk_indexs.sh >/dev/null 2>&1

Posted by Dan400007 on Sun, 01 Dec 2019 07:58:36 -0800