An introduction to rsync

rsync is a common Linux application for file synchronization. It can synchronize files between local computers and remote computers, or between two local directories (but it does not support synchronization between two remote computers). It can also be used as a file copying tool instead of cp and mv commands. r refers to remote. rsync actually means "remote sync". Different from other file transfer tools (such as FTP or scp), rsync is characterized by checking the existing files of the sender and receiver, and only transmitting the changed parts (the default rule is that the file size or modification time changes).

II. rsync features

You can mirror and save the entire directory tree and file system. It is easy to maintain the permissions, time, soft and hard links of the original files and install them without special permissions.

Fast: rsync will copy all the contents during the first synchronization, but only the modified files will be transferred next time. rsync can compress and decompress data, so it can use less bandwidth

Security: you can use scp, ssh and other methods to transfer files. Of course, you can also connect directly through socket. Support anonymous transmission to facilitate website mirroring

Three rsynx ssh authentication protocols

In the ssh authentication mode, rsync can be authenticated by the system user, that is, it can be transmitted through the ssh tunnel on rsync. Similar to the scp tool, the synchronization operation is not limited to the synchronization folder defined in rsync, and the rsync server does not need to start the daemon process of rsync, As long as you obtain the user name and password of remote host, you can directly rsync synchronize the file rsync server side. Because you do not need to start the daemon process, you do not need the configuration file / etc/rsyncd.conf

//This method omits - e ssh by default, which is equivalent to the following:
rsync -avz /SRC -e ssh root@192.168.143.106:/DEST 
    -a  //The file host changes and the timestamp remains unchanged
    -V  //Process for displaying details
    -z  //Compressed data transmission

[root@103 ~]# rsync -avz anaconda-ks.cfg -e ssh root@192.168.143.106:/opt/
[root@106 ~]# ls /opt/
anaconda-ks.cfg  data

//When the port needs to be modified, we can:
#The ssh protocol port is modified. The default is 22
rsync -avz /SRC -e "ssh -p2222" root@192.168.143.106:/DEST  

IV. inotify+rsync

Compared with the traditional cp and tar backup methods, rsync has the advantages of high security, fast backup and supporting incremental backup. rsync can solve the data backup requirements with low real-time requirements, such as regularly backing up the file server data to the remote server, regularly mirroring the local disk, etc.

Environmental description:

Target server operation

[root@106 ~]# yum install -y rsync
. . . . . . . . . . . slightly
[root@106 ~]# yum install -y rsync-daemon
. . . . . . .. . .. . slightly
[root@106 ~]# vim /etc/rsyncd.conf 
[root@106 ~]# cat /etc/rsyncd.conf 
log file = /var/log/rsyncd.log    # Log file location. This file will be generated automatically after rsync is started. There is no need to create it in advance
pidfile = /var/run/rsyncd.pid     # Storage location of pid files
lock file = /var/run/rsync.lock   # Lock files that support the max connections parameter
secrets file = /etc/rsync.pass    # User authentication profile, which stores user name and password, must be created manually

[etc_from_client]     # Custom sync name
path = /tmp/          # rsync server data storage path, and client data will be synchronized to this directory
comment = sync etc from client
uid = root        # Set rsync running permission to root
gid = root        # Set rsync running permission to root
port = 873        # Default port
ignore errors     # Indicates that an error has occurred. Ignore the error
use chroot = no       # The default value is true and modified to no. the soft connection backup of directory files is added
read only = no    # Set the rsync server to read / write permission
list = no     # The rsync server resource list is not displayed
max connections = 200     # maximum connection
timeout = 600     # Set timeout
auth users = admin        # Multiple user names for data synchronization can be set, separated by commas in English
[root@106 ~]#  tr -dc A-Za-z0-9 < /dev/urandom | head -c 8 |xargs
MvfcaV2C
[root@106 ~]# echo 'admin:MvfcaV2C' > /etc/rsync.pass
[root@106 ~]# echo 'MvfcaV2C' > .pw_rsy
[root@106 ~]# chmod 600 /etc/rsync*
[root@106 ~]# systemctl enable --now rsyncd
Created symlink /etc/systemd/system/multi-user.target.wants/rsyncd.service → /usr/lib/systemd/system/rsyncd.service.
[root@106 ~]# ss -atnl
State      Recv-Q      Send-Q           Local Address:Port           Peer Address:Port     Process     
LISTEN     0           128                    0.0.0.0:22                  0.0.0.0:*                    
LISTEN     0           5                      0.0.0.0:873                 0.0.0.0:*  

Source server operation:

to configure elerl source
yum -y install epel-release
 Slightly....... . . .. ... . .. . .. . . . . . 

install inotify-tools
yum -y install inotify-tools
 Slightly............ . . . .. .. ..

Create password file
[root@103 ~]# echo 'MvfcaV2C' > /etc/rsync.pass
[root@server ~]# chmod 600 /etc/rsync.pass

Create directory
[root@103 ~]# mkdir /opt/txt/ 
root@103 ~]# rsync -avH --port 873 --progress --delete /opt/txt/ admin@192.168.143.106::etc_from_client --password-file=/etc/rsync.pass
[root@server ~]# 

Use script

[root@103 /]# mkdir scripts
[root@103 /]# ls
bin   dev  home  lib64  mnt  proc  run   scripts  sys  usr
boot  etc  lib   media  opt  root  sbin  srv      tmp  var
[root@103 /]# cd scripts/
[root@103 scripts]# ls
[root@103 scripts]# vim inotify.sh

host=192.168.168.106      # IP of the target server (backup server)
src=/etc        # The backup directory to be monitored on the source server (you can customize it here, but make sure it exists)
des=etc_from_client     # The customized module name must be consistent with the synchronization name defined on the target server
password=/etc/rsync.pass        # Password file to perform data synchronization
user=admin          # User name to perform data synchronization
inotifywait=/usr/bin/inotifywait

$inotifywait -mrq --timefmt '%Y%m%d %H:%M' --format '%T %w%f%e' -e modify,delete,create,attrib $src \
        | while read files;do
    rsync -avzP --delete  --timeout=100 --password-file=${password} $src $user@$host::$des
        echo "${files} was rsynced" >>/tmp/rsync.log 2>&1
done

[root@103 scripts]# chmod +x inotify.sh 
[root@103 scripts]# ls
inotify.sh

Start the script and hang it in the background
[root@103 scripts]# nohup bash /scripts/inotify.sh &
[2] 140039

View process
[root@103 scripts]# ps -ef|grep inotify
root      141852    5139  0 19:47 pts/0    00:00:00 bash /scripts/inotify.sh
root      141853  141852  0 19:47 pts/0    00:00:00 /usr/bin/inotifywait -mrq --timefmt %Y%m%d %H:%M --format %T %w%f%e -e modify,delete,create,attrib /etc
root      141854  141852  0 19:47 pts/0    00:00:00 bash /scripts/inotify.sh
root      144190    5139  0 19:48 pts/0    00:00:00 grep --color=auto inotify

Trigger view effect
[root@103 etc]# touch test.txt
[root@103 etc]# echo 'google' > test.txt 
[root@103 etc]# cat /etc/test.txt
google

Startup and self start
[root@103~ ]# chmod +x /etc/rc.d/rc.local
[root@103~ # cat /etc/rc.d/rc.local 
#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

nohup /scripts/inotify.sh &
touch /var/lock/subsys/local

View target host

[root@106 etc]# ls
adjtime                  GREP_COLORS               motd.d                  sasl2
aliases                  groff                     mtab                    security
alternatives             group                     nanorc                  selinux
anacrontab               group-                    netconfig               services
dconf                    libreport                 protocols                test.txt