About rsync

A fast incremental backup tool

Configure rsync source server

rsync sync source

Refers to the remote server of the backup operation, also known as the backup source

Configure rsync source

Basic thinking

Set up rsync.conf configuration file and independent account file
 Enable -- daemon mode of rsync

Application example

User backuper, allowing downlink synchronization
 The directory of the operation is / var/www/html

Configuration file rsyncd.conf

It needs to be established manually, and the syntax is similar to Samba configuration
 Authentication configuration: auth users, secrets file, anonymous if not added

rsync account file

Use the record format of "user name: password", one user record per line
 Independent account data, independent of system account

Enable rsync service

Provide services alone through -- daemon
 Execute kill $(cat /var/run/rsync.pid) to shut down the rsync service

Using the rsync backup tool

Usage of rsync command

rsync [options] original location target location

Common options

-a: Archive mode, recursion and retention of object properties, etc. for - rlptgoD
 -v: Show details of the synchronization process
 -z: Compress when transferring files
 -H: Keep hard connection files
 -A: Keep ACL attribute information
 --Delete: delete files that exist in the target location but not in the original location
 --checksum: decide whether to skip files based on the checksums of objects

Two representations of configuration source

Format 1: user name @ host address:: share module name
 Format 2: rsync: / / username @ host address / shared module name

rsync real time synchronization

Lack of periodic synchronization

The backup time is fixed, the delay is obvious, and the real-time performance is poor
 When the synchronous source does not change for a long time, intensive periodic tasks are unnecessary

Advantages of real-time synchronization

Start backup as soon as synchronization source changes
 Do not perform backup as long as the synchronization source is unchanged

About inotify (installed on the initiator)

Inotify is a Linux feature that monitors file system operations such as read, write, and create.
Inotify is sensitive, easy to use, and much more efficient than busy polling for cron tasks.
It can monitor the change of file system and make notification response;
Auxiliary software: inotify tools

Experimental environment

rsyncd: 192.168.52.134
client: 192.168.52.148

1. Modifying the configuration file on the rsyncd server

[root@rsyncd ~]# rpm -q rsync   ##Check whether rsync is installed. It is not installed using yum
rsync-3.0.9-18.el7.x86_64
[root@rsyncd ~]# vim /etc/rsyncd.conf
uid = nobody     ##Anonymous user
gid = nobody
use chroot = yes   ##Home detention directory
pid file = /var/run/rsyncd.pid  ##pid file
address = 192.168.13.128    ##Monitor address
port = 873    ##port number 
log file = /var/log/rsyncd.log   ##log file path
hosts allow = 192.168.13.0/24   ##Allow address segment access
dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2   ##Types that do not require compression

[wwwroot]      ##Shared module name
path = /var/www/html    ##Route
comment = www.kgc.com  ##Definition name
read only = yes    ##Open read-only
auth users = backuper   ##Authentication user name
secrets file = /etc/rsyncd_users.db    ##Password file

[root@rsyncd ~]# vim /etc/rsyncd_users.db  ##Create password file
backuper:abc123  ##User name: password
[root@rsyncd ~]# chmod 600 /etc/rsyncd_users.db   ##Give root access to read and write
[root@rsyncd ~]# rsync --daemon   ##Start rsync service
[root@rsyncd ~]# netstat -ntap | grep rsync   ##View port
tcp        0      0 192.168.52.134:873      0.0.0.0:*               LISTEN      15471/rsync      
[root@rsyncd ~]# systemctl stop firewalld.service   ##Turn off firewall
[root@rsyncd ~]# setenforce 0
[root@rsyncd ~]# yum install httpd -y   ##Install httpd service
[root@rsyncd ~]# cd /var/www/html/
[root@rsyncd html]# echo "this is test web" > index.html   ##Create web page information
[root@rsyncd html]# cd ../
[root@rsyncd www]# chmod 777 html/   ##Give maximum permission for any user
[root@rsyncd www]# ll   ##View permission
//Total dosage 0
drwxr-xr-x. 2 root root  6 8 Month 819:42 cgi-bin
drwxrwxrwx. 2 root root 24 12 Month 1315:11 html
[root@rsyncd www]# 

2. On the client server, pull the synchronization source rsyncd

[root@client ~]# rpm -q rsync  ##Check if rsync service is installed
rsync-3.0.9-18.el7.x86_64
[root@client ~]# systemctl stop firewalld.service  ##Turn off firewall
[root@client ~]# setenforce 0
[root@client ~]# yum install httpd -y  ##Install httpd service
[root@client ~]# cd /var/www/
[root@client www]# chmod 777 html/  ##Give maximum permission
[root@client www]# ls -l   ##Check where to go first
//Total dosage 0
drwxr-xr-x. 2 root root 6 8 Month 819:42 cgi-bin
drwxrwxrwx. 2 root root 6 8 Month 819:42 html

##Synchronization format 1:
[root@client www]# rsync -avz backuper@192.168.52.134::wwwroot /var/www/html/
##Pull sharing module
Password:   ##Input password  
./
index.html

sent 83 bytes  received 172 bytes  46.36 bytes/sec
total size is 17  speedup is 0.07
[root@client www]# ls
cgi-bin  html
[root@client www]# cd html/
[root@client html]# ls
index.html
[root@client html]# cat index.html     ##View synchronization
this is test web
[root@client html]# 
[root@client www]# cat html/index.html
this is test web

##Synchronization format 2:
[root@client html]# rm -rf index.html    ##Delete synchronized files
[root@client html]# ls
[root@client html]# rsync -avz rsync://backuper@192.168.52.134/wwwroot /var/www/html/
##Pull sharing module
Password:    ##Input password  
receiving incremental file list
./
index.html

sent 83 bytes  received 172 bytes  72.86 bytes/sec
total size is 17  speedup is 0.07
[root@client html]# ls
index.html
[root@client html]# cat index.html     ##View synchronization
this is test web
[root@client html]# 

##No interactive synchronization:
[root@client html]# rm -rf index.html     ##Delete synchronized files
[root@client html]# touch abc.html   ##Create an abc.html file in the directory
[root@client html]# ls
abc.html
[root@client html]# 
[root@client html]# vim /etc/server.pass  ##Create a local password file
abc123
[root@client html]# chmod 600 /etc/server.pass    ##Grant authority
[root@client html]# 
[root@client html]# rsync -avz --delete --password-file=/etc/server.pass backuper@192.168.52.134::wwwroot /var/www/html/
##Specify the local password file, delete the files in the target location but not in the original location, so as to realize interaction free
receiving incremental file list
deleting abc.html
./
index.html

sent 83 bytes  received 172 bytes  170.00 bytes/sec
total size is 17  speedup is 0.07
[root@client html]# ls   ##As you can see, abc.html was deleted because of the -- delete option
index.html
[root@client html]# cat index.html 
this is test web
[root@client html]# 

3. Install inotify monitoring on the client

[root@client html]# cd ../
[root@client www]# vim /etc/sysctl.conf   ##Modify kernel parameter file
fs.inotify.max_queued_events = 16384   ##queue
fs.inotify.max_user_instances = 1024     ##Number of instances per queue
fs.inotify.max_user_watches = 1048576  ##Number of files per instance
[root@client www]# sysctl -p  ##Load
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 1024
fs.inotify.max_user_watches = 1048576
[root@client www]# mount.cifs //192.168.100.100/tools /mnt/tools / ා񖓿mount
Password for root@//192.168.100.100/tools:  
[root@client www]# cd /mnt/tools/inotify/
[root@client inotify]# tar xf inotify-tools-3.14.tar.gz -C /opt/   ##Unzip inotify to / opt
[root@client inotify]# cd /opt/inotify-tools-3.14/
[root@client inotify-tools-3.14]# yum install gcc gcc-c++ make -y   ##Components necessary for the installation environment
[root@client inotify-tools-3.14]# ./configure    ##To configure
[root@client inotify-tools-3.14]# make && make install  ##Compilation and installation
[root@client inotify-tools-3.14]# inotifywait -mrq -e modify,create,move,delete /var/www/html/    
##Monitoring

##Restart the terminal of a client
[root@client ~]# cd /var/www/html/
[root@client html]# ls
index.html
[root@client html]# touch abc
[root@client html]# rm -rf abc 
[root@client html]# 

##View on client on monitor
/var/www/html/ CREATE abc
/var/www/html/ DELETE abc

4. Create script in client, trigger rsync synchronous operation script through inotifywait

[root@client inotify-tools-3.14]# cd /opt/
[root@client opt]# vim inotify.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e modify,create,move,delete /var/www/html/"
RSYNC_CMD="rsync -azH --delete --password-file=/etc/server.pass /var/www/html/ backuper@192.168.52.134::wwwroot/"
$INOTIFY_CMD | while read DIRECTORY EVENT FILE  
 do
   if [ $(pgrep rsync | wc -l) -le 0 ]; then    
      $RSYNC_CMD
   fi
done
[root@client opt]# chmod +x inotify.sh  ##Give execution permission
##Ensure that both the server and the client have the maximum permissions

5. Modifying the configuration file on the rsyncd server

[root@rsyncd www]# vim /etc/rsyncd.conf
read only = no  ##Close read only
[root@rsyncd www]# pkill -9 rsync   ##Close
[root@rsyncd www]# netstat -ntap | grep rsync
[root@rsyncd www]# 
[root@rsyncd www]# rm -rf /var/run/rsyncd.pid   ##Delete pid file
[root@rsyncd www]# rsync --daemon    ##Start rsync service
[root@rsyncd www]# netstat -ntap | grep rsync
tcp        0      0 192.168.52.134:873      0.0.0.0:*               LISTEN      50571/rsync         
[root@rsyncd www]#

6. Executing inotify script files on the client

##Client execution script
[root@client opt]# ./inotify.sh
##Enter monitoring status

##Restart a client terminal
[root@client ~]# cd /var/www/html/
[root@client html]# ls
index.html
[root@client html]# echo "this is test" > test.txt  ##Add text

##View monitoring service information
[root@client opt]# ./inotify.sh 
rsync: failed to set times on "/." (in wwwroot): Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]

##View on rsync server
[root@rsyncd www]# cd html/
[root@rsyncd html]# ls
index.html  test.txt   ##Synchronous completion

##In the newly opened client terminal
[root@client html]# rm -rf test.txt 
[root@client html]# ls
index.html
[root@client html]# 

##View on rsync server
[root@rsyncd html]# ls
index.html
[root@rsyncd html]# 
##Deletion is also synchronized