About rsync

A fast incremental backup tool

• Remote Sync, remote synchronization
• Support local replication or synchronization with other SsH, rsync hosts
• Official website: http://rsync.samba.org

Configure rsync Source Server

rsync Synchronization Source

• Refers to a remote server for backup operations, also known as a backup source

Configure rsync source

• Basic ideas
  • Set up rsyncd.conf configuration file, separate account file
  • Enable -daemon mode for rsync
• Application examples
  • User backuper, allowing downlink synchronization
  • The directory of operations is/var/www/html/
• Configuration file rsyncd.conf
  • Manually built, syntax similar to Samba configuration
  • Authentication configures auth users, secrets file s, or anonymous if not added
• rsync account file
  • User name: password record format with one user record per line
  • Independent account data, independent of system accounts
• Enable rsync service
  • Serve alone through--daemon
  • Execute kill $(cat/var/run/rsyncd.pid) to shut down rsync service

Use rsync backup tool

rsync command usage

• rsync [options] original location target location
• Common Options
  • -a: Archive mode, recursive and preserves object attributes, equivalent to -rlptgoD
  • -v: Verbose information showing the synchronization process
  • -z: compress when transferring files
  • -H: Keep hard-connected files
  • -A: Keep ACL attribute information
  • --delete: Delete files that have the destination location but not the original location
  • --checksum: Determines whether to skip files based on the object's checksum
• Two representations of the configuration source
  • Format 1:User name@Host address:Shared module name
  • Format 2: rsync://username@host address/shared module name

rsync Real-Time Synchronization

Insufficient periodic synchronization

• Performing backups for a fixed amount of time with significant delays and poor real-time performance
• Dense periodic tasks are unnecessary when the synchronization source remains constant for a long time

Advantages of real-time synchronization

• Start backup as soon as the synchronization source changes
• Backup will not be performed as long as the synchronization source has not changed

About inotify

inotify mechanism of Linux kernel

• Beginning with version 2.6.144
• You can monitor changes to the file system and respond with notifications
• Auxiliary software: inotify-tools

rsync+inotify real-time synchronization

• Adjust inotify kernel parameters
  • max_queue_events: Monitor queue size
  • max_user_instances: Maximum number of monitoring instances
  • max_user_watches: Maximum number of monitoring files per instance, which should be configured larger than the total number of files for the monitoring target
• Install the inotify-tools assistant
  • inotifywait: for continuous monitoring, real-time output of results
  • Common Options
  • -m: Continuous monitoring
  • -r: Monitor all child objects recursively
  • -q: simplify output information
  • -e: Specify which event types to monitor
  • inotifwatch: Used for short-term monitoring with results after the task is completed
• Triggering rsync synchronization via inotifywait
  • Use while, read to continuously obtain monitoring results
  • Based on the results, you can make further judgments and decide what to do

Practice Configuration

Experimental environment

• rsyncd server IP address: 192.168.144.128
• Client client IP address: 192.168.144.129

Modify configuration file on rsyncd server

[root@rsyncd ~]# rpm -q rsync
rsync-3.0.9-18.el7.x86_64
[root@rsyncd ~]# vim /etc/rsyncd.conf

uid = nobody           //anonymous
gid = nobody
use chroot = yes         //Prohibited Home Directory
pid file = /var/run/rsyncd.pid        //pid file path
address = 192.168.144.128                //Configure Listening Address
port = 873                        //Port number
log file = /var/log/rsyncd.log      //log file path
hosts allow = 192.168.144.0/24         //Allow Address Segment Access
dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2   //Types that do not require compression

[wwwroot]           //Shared Module Name
path = /var/www/html        //Shared File Path
comment = www.kgc.com      //Define Name
read only = yes            //Read-only rights
auth users = backuper       //Authentication User Name
secrets file = /etc/rsyncd_users.db        //Password file
:wq
[root@rsyncd ~]# Vim/etc/rsyncd_users.db //Create password file
backuper:123123       //Edit User Name: Password
:wq
[root@rsyncd ~]# Chmod 600/etc/rsyncd_users.db //Change permissions
[root@rsyncd ~]# rsync --daemon //Open rsync service
[root@rsyncd ~]# Netstat-ntap | grep Rsync //view port
tcp     0     0 192.168.144.128:873      0.0.0.0:*    LISTEN    36346/rsync 
[root@rsyncd ~]# systemctl stop firewalld.service //close firewall
[root@rsyncd ~]# setenforce 0
[root@rsyncd ~]# Yum install httpd-y //install httpd service
[root@rsyncd ~]# cd /var/www/html/
[root@rsyncd html]# Echo "this is test web" > index.html //Edit web page information
[root@rsyncd html]# cd ../
[root@rsyncd www]# chmod 777 html/ //Release directory permissions for user convenience

On the client server, pull the synchronization source rsyncd

[root@client ~]# systemctl stop firewalld.service //close firewall
[root@client ~]# setenforce 0 //close selinux
[root@client ~]# Rpm-q rsync //Check whether rsync service is installed
rsync-3.0.9-18.el7.x86_64
[root@client ~]# Yum install httpd-y //install httpd service
[root@client ~]# cd /var/www/
[root@client www]# chmod 777 html/ //Release directory permissions
[root@client www]# Rsync-avz backuper@192.168.144.128:: wwwroot/var/www/html //pull shared module
Password:                        //Input password  
[root@client www]# cat html/index.html //View synchronization information
this is test web
[root@client www]# rm -rf html/index.html 
[root@client www]# Vim/etc/server.pass//Create a local password file
123123
[root@client www]# Chmod 600/etc/server.pass //Change permissions
[root@client www]# Rsync-avz --delete --password-file=/etc/server.pass backuper@192.168.144.128:: wwwroot/var/www/html///Specify a local password file, delete files in the target location that are not in the original location, and avoid interaction

Install inotify on client client

[root@client www]# Vim/etc/sysctl.conf //Modify Kernel Parameter File
fs.inotify.max_queued_events = 16384       //queue
fs.inotify.max_user_instances = 1024        //Number of instances in each queue
fs.inotify.max_user_watches = 1048576      //Number of files in each instance
[root@client www]# sysctl -p  ##Load
[root@client www]# Mount.cifs //192.168.100.8/LNMP-C7/mnt/ //mount
Password for root@//192.168.100.3/LNMP-C7:  
[root@client www]# cd /mnt/
[root@client mnt]# Tar zxvf inotify-tools-3.14.tar.gz-C/opt/ //Unzip inotify to/opt
[root@client mnt]# cd /opt/
[root@client opt]# cd inotify-tools-3.14/
[root@client inotify-tools-3.14]# Yum install GCC gcc-c++ make-y //installation environment
[root@client inotify-tools-3.14]# . /configure //configure
[root@client inotify-tools-3.14]# Make & & make install //compile installation
[root@client inotify-tools-3.14]# Inotifywait-mrq-e modify, create, move, delete/var/www/html/ //Start monitoring

Reopen a client terminal

[root@client ~]# cd /var/www/html/
[root@client html]# touch abc
[root@client html]# rm -rf abc

Back to the terminal where monitoring is turned on

/var/www/html/ CREATE abc
/var/www/html/ DELETE abc    //Display monitoring information

Create a script on the client to trigger the rsync synchronization action script via inotifywait

[root@client inotify-tools-3.14]# cd /opt/
[root@client opt]# vim inotify.sh
#!/bin/bash
INOTIFY_CMD="inotifywait -mrq -e modify,create,move,delete /var/www/html/"
RSYNC_CMD="rsync -avz --delete --password-file=/etc/server.pass /var/www/html/ backuper@192.168.144.128::wwwroot/"
$INOTIFY_CMD | while read DIRECTORY EVENT FILE
 do
     if [ $(pgrep rsync | wc -l) -le 0 ]; then
            $RSYNC_CMD
     fi
done
[root@client opt]# chmod +x inotify.sh //Add Execution Rights

Modify configuration file on rsyncd server

[root@rsyncd www]# vim /etc/rsyncd.conf
read only = no         //Turn off read-only permissions
[root@rsyncd www]# netstat -natp | grep rsync
tcp     0    0 192.168.144.128:873    0.0.0.0:*      LISTEN      36346/rsync         
[root@rsyncd www]# Kill-9 36346 //Shut down service
[root@rsyncd www]# netstat -natp | grep rsync
[root@rsyncd www]# Rm-rf/var/run/rsyncd.pid //Delete PID file
[root@rsyncd www]# rsync --daemon //Restart rsync service

Execute inotify script file on client client

[root@client opt]# ./inotify.sh

Reopen a client client client terminal

[root@client html]# Echo "this is test" > test.txt //Add text

View Open Monitoring Service Terminal Information

[root@client opt]# ./inotify.sh 
sending incremental file list
./
rsync: failed to set times on "/." (in wwwroot): Operation not permitted (1)
test.txt

sent 121 bytes  received 30 bytes  302.00 bytes/sec
total size is 30  speedup is 0.20
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
sending incremental file list

sent 66 bytes  received 8 bytes  148.00 bytes/sec
total size is 30  speedup is 0.41

View on rsync server

[root@rsyncd www]# cd html/
[root@rsyncd html]# ls
index.html  test.txt       //Synchronize