inotify asynchronous monitoring mechanism

Keywords: inotify RPM EPEL Linux

1. Introduction to inotify

  • Inotify is a powerful, fine-grained, asynchronous file system event monitoring mechanism (software), linux kernel from 2.6.13, joined the Inotify support, through Inotify can monitor the file system to add, delete, modify, move and other events.
  • Inotify is actually an event-driven mechanism, which provides a real-time response mechanism for applications to monitor file system events without the need for polling mechanisms such as cron to obtain events. Cron and other mechanisms not only can not achieve real-time, but also consume a lot of system resources. In contrast, inotify is event-driven, which can achieve real-time response to event processing, and does not consume system resources caused by polling. It is a very natural event notification interface, which is also consistent with the event mechanism of the natural world.
  • There are several kinds of software to implement inotify: inotify-tools, sersync, lsyncd

Note: sersync software is actually developed on the basis of inotify software, with more powerful functions, more timing retransmit mechanism, filtering mechanism providing interface to do CDN, and supporting multi-threaded fabrication. It was developed by Zhou Yang in Jinshan Company.

2. Deployment of inotify software

Do you support:
~]# uname -r		#2.6.13The above kernel version
3.10.0-957.el7.x86_64

~]# ll /proc/sys/fs/inotify/	#Before you install inotify software, you should have these three files
total 0
-rw-r--r-- 1 root root 0 Sep 10 22:38 max_queued_events
-rw-r--r-- 1 root root 0 Sep 10 22:38 max_user_instances
-rw-r--r-- 1 root root 0 Sep 10 22:38 max_user_watches
Document description:
file Default value Function description
max_queued_events 8192 Setting the inotifywait or inotifywatch command can monitor the number of files (single process)
max_user_instances 128 Set the number of processes that each user can run with the inotifywait or inotifywatch command
max_user_watches 16384 Set the number of events that the inotify instance event queue can hold
Install inotify-tools:
~]# rpm -ivh https://mirrors.aliyun.com/epel/epel-release-latest-7.noarch.rpm
~]# yum install -y inotify-tools	#Dependent on epel source
inotify-tools provides two command-line tools:
~]# rpm -ql inotify-tools | grep /bin
/usr/bin/inotifywait
/usr/bin/inotifywatch

inotifywait: Waiting for a specific file system event (open, close, delete, etc.) to occur on the monitored file or directory, which is blocked after execution and suitable for shell scripts.
inotifywatch: Statistical data used to collect file systems, such as how many inotify events have occurred, how many times a file has been accessed, etc., are generally not used.

2.1 Intifywait command parameter parsing

Basic parameters:
parameter Meaning
- m,--monitor (important parameter) Always listen for events.
-d, --daemon Running as a daemon
- r, --recursive (important parameter) Recursive Monitoring of Catalog Data Information Change
-o, --outfile Print events to files, equivalent to standard correct output
-s, --syslog Sending errors to syslog is equivalent to standard error output
- q,--quiet (important parameter) Less output information (print event information only)
–exclude <pattern> Exclude files or directories
–excludei <pattern> When excluding files or directories, case-insensitive
Time FMT < FMT > (important parameters) Specified time output format
Format < FMT > (important parameters) Printing uses a specified output similar to a format string; that is, the actual monitoring of the output content
- e, --event (important parameter) (Important parameter) Specifies that the specified event is monitored, if omitted, indicating that all events are monitored
Inotifywait-e [parameter] specifies the event type:
Event name Event description
access The contents of a file or directory are read
modify File or directory contents are written
attrib Change of file or directory attributes
close_write (important parameter) The file or directory is closed after the write mode is opened.
close_nowrite Closed when the file or directory is opened in read-only mode
close Close files or directories, whether in read or write mode
open Files or directories are opened
moved_to Files or directories are moved to monitored directories
moved_from Files or directories are moved from monitored directories
move (important parameter) Files or directories trigger events whether they are moved to or removed from the monitoring directory
create (important parameters) Files or directories are created in the monitoring directory
delete (important parameter) Files or directories are deleted in the monitoring directory
delete_self Files or directories are deleted, and directories themselves are deleted
unmount Unmount the file or directory contained in the file system
isdir Monitor directory related operations
Inotifywait -- format < FMT > event format parameter:
Command parameter Parameter description
% w (important parameters) Monitor file or directory name information when an event occurs
% f (Important parameters) When an event occurs, the file or directory information triggering the event in the monitoring directory will be displayed, otherwise it will be empty.
% e (Important parameters) Display event information, separate event information by commas
%Xe Display the event information that happened. Different event information is separated by X. X can be modified to specify the separator.
%T Output the time format information defined in the time format, specify the time information through the -- time FMT option grammar format
inotifywait parameter -- timefmt < FMT > event format parameter:
Command parameter Parameter description
% d (Important parameters) On the day of each month, the multiplier is shown as decimal (range 01-31)
% m (Important parameters) Display month, display information is decimal (range 01-12)
%M Display minutes, display information in decimal (range 00-59)
% y (important parameters) Annual information, showing information in decimal, and no century information
%Y Annual information, showing information in decimal and containing century information
%H Hour information, display information is decimal, use 24-hour system (range 00-23)

2.2 inotifywait test

Monitoring Directory:
~]# mkdir /work		#Monitor/work Catalog
# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move	
# --timefmt Specify output time format
# --format Specify the output string,%T output--timefmt Specified time format content
# -e,This parameter does not apply. By default, it listens to all events. We only listen to create, delete, modify, move and so on.
Create common files:
~]# touch test.file		#Operate the new meeting

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:34 /work/test.file CREATE
19/09/10 23:34 /work/test.file CLOSE_WRITE,CLOSE
Create a directory:
~]# mkdir testdir

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:34 /work/testdir CREATE,ISDIR
Write data:
~]# echo "hello world" > test.file 

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:34 /work/test.file CLOSE_WRITE,CLOSE
Modify data:
~]# vim test.file 

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:52 /work/.test.file.swp CREATE
19/09/10 23:52 /work/.test.file.swx CREATE
19/09/10 23:52 /work/.test.file.swx CLOSE_WRITE,CLOSE
19/09/10 23:52 /work/.test.file.swx DELETE
19/09/10 23:52 /work/.test.file.swp CLOSE_WRITE,CLOSE
19/09/10 23:52 /work/.test.file.swp DELETE
19/09/10 23:52 /work/.test.file.swp CREATE
19/09/10 23:52 /work/4913 CREATE
19/09/10 23:52 /work/4913 CLOSE_WRITE,CLOSE
19/09/10 23:52 /work/4913 DELETE
19/09/10 23:52 /work/test.file MOVED_FROM
19/09/10 23:52 /work/test.file~ MOVED_TO
19/09/10 23:52 /work/test.file CREATE
19/09/10 23:52 /work/test.file CLOSE_WRITE,CLOSE
19/09/10 23:52 /work/test.file~ DELETE
19/09/10 23:52 /work/.test.file.swp CLOSE_WRITE,CLOSE
19/09/10 23:52 /work/.test.file.swp DELETE
Move in the file:
]# cp /etc/hosts testdir/

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:35 /work/testdir/hosts CREATE
19/09/10 23:35 /work/testdir/hosts CLOSE_WRITE,CLOSE
Remove files:
~]# mv test.file /tmp/ 

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:49 /work/test.file MOVED_FROM
Delete files:
~]# rm -f testdir/hosts 

~]# inotifywait -mrq /work --timefmt "%y/%m/%d %H:%M" --format "%T %w%f %e" -e create,delete,close_write,move
19/09/10 23:51 /work/testdir/hosts DELETE

More parameters can be tested and used by ourselves.

2.3 Intifywatch command parameter parsing

Although this command is not used much, this article still makes a simple analysis of it.

parameter Meaning
-v, --verbose Output details
-r, --recursive Monitor all subdirectories in a directory
-t, --timeout Setting timeout time
-a, --ascending Arrange in ascending order of specified events
-d, --descending In descending order of specified events
–exclude Regular matching excluded files, case sensitive
–excludei Regular matching excludes files, ignoring case
@<file> Exclude files that do not need to be monitored, either relative or absolute
–formfile <file> Read files to be monitored or excluded from files, one file line, excluded files begin with @.
-e, --event Listen only for specified events
Simple use:
~]# mkdir hello{1..10}
~]# touch hello1/file{1..5}
~]# rm -f hello1/file{1..2}

~]# inotifywatch -v -e create,delete,close_write,move -t 60 -r /work  	#Monitor/work Directory60s Corresponding events occur within
Establishing watches...
Setting up watch(es) on /work
OK, /work is now being watched.
Total of 1 watches.
Finished establishing watches, now collecting statistics.
Will listen for events for 60 seconds.
total  close_write  create  delete  filename
12     5            5       2       /work/hello1/
10     0            10      0       /work/

2.4 Simple use of inotifywait in scripts

#!/bin/bash
#author by chuan

inotifywait -mrq --timefmt '%y/%m/%d %H:%M' --format '%T %w%f %e' -e create,delete,close_write,move,isdir /work |\
while read line
do
    if echo $line | grep -i -E "CREATE|CLOSE_WRITE|DELETE|MOVED_TO|MOVED_FROM";then
        echo $line >> /tmp/inotify.log
    fi
done

2.5 inotify system parameter modification

According to the actual needs, the value of the three files can be increased to monitor a wider range.

Temporary amendments:
~]# echo "16385" >> max_queued_events 
~]# echo "129" >> max_user_instances
~]# echo "8193" >> max_user_watches
Permanent effect:
~]# sysctl -w fs.inotify.max_queued_events="16385"
~]# sysctl -w fs.inotify.max_user_instances="129"
~]# sysctl -w fs.inotify.max_user_watches="8193"

Posted by urb on Thu, 12 Sep 2019 01:13:02 -0700