Synchronized server-side configuration
mkdri /data/filedir chown -R rsync:rsync /data/filedir yum install rsync -y //create profile [root@node1 filedir]# vim /etc/rsyncd.conf ##rsync.conf config start uid = rsync gid = rsync use chroot = no max connetctions = 200 timeout = 100 pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsyncd.log [backup] path = /data/filedir ignore errors read only = false list = false hosts allow = 192.168.193.129 auth users = rsync_backup secrets file = /etc/rsync.password ##rsync config end //Create a password file: echo "rsync_backup:123456">/etc/rsync.password chmod 600 /etc/rsync.password //Start up service: rsync --daemon //See: [root@node1 filedir]# netstat -ntulp | grep rsync tcp 0 0 0.0.0.0:873 0.0.0.0:* LISTEN 1631/rsync tcp 0 0 :::873 :::* LISTEN 1631/rsync //Set up boot-up self-starting rsync service: vim /etc/rc.local //Add to # rsync server progress /usr/bin/rsync --daemon
Synchronized client configuration:
Configuration password file: echo "123456">/etc/rsync.password chmod 600 /etc/rsync.password //Manual test data push: [root@node2 ~]# rsync -avzP /etc/hosts rsync_backup@192.168.193.128::backup --password-file=/etc/rsync.password sending incremental file list sent 26 bytes received 8 bytes 22.67 bytes/sec total size is 180 speedup is 5.29 //View on the synchronous server side: [root@node1 filedir]# pwd /data/filedir [root@node1 filedir]# cat hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.193.129 node2 //The client's / etc/hosts file has been synchronized to the backup server.
Deployment of sersync Services on Synchronized Client Ends
tar zxvf sersync2.5.4_64bit_binary_stable_final.tar.gz -C /usr/local/
cd /usr/local/
mv GNU-Linux-x86 sersync
mkdir -p conf bin logs
mv confxml.xml conf/
mv sersync2 bin/sersync
echo 'export PATH=$PATH:/usr/local/sersync/bin'>>/etc/profile
source /etc/profile
cd conf/
cp -a confxml.xml{,.`date +%F`} #Copy configuration files
//The software catalog structure is as follows:
[root@node2 sersync]# tree
.
├── bin
│ └── sersync
├── conf
│ ├── confxml.xml
│ └── confxml.xml.2017-06-19
└── logs
└── rsync_fail_log.sh
#Modify configuration files
#Part I: Setting up synchronous directory, remote server and backup module for local monitoring (backup module is defined in remote server/etc/rsyncd.conf)
<localpath watch="/data/tongbu">
<remote ip="192.168.193.128" name="backup"/>
#<!--<remote ip="192.168.8.39" name="tongbu"/>-->
#<!--<remote ip="192.168.8.40" name="tongbu"/>-->
</localpath>
#Part II: Setting up the authentication part, the server-side authentication user and password file storage location (defined in remote server/etc/rsyncd.conf)
<rsync>
<commonParams params="-artuz"/>
<auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/>
<userDefinedPort start="false" port="874"/><!-- port=874 -->
<timeout start="true" time="100"/><!-- timeout=100 -->
<ssh start="false"/>
</rsync>
//Similar to rsync-avzP/etc/hosts, rsync_backup@192.168.193.128:: backup -- password-file=/etc/rsync.password
#Part 3: Set up the storage location of synchronization failure log, record when synchronization failure occurs, and resynchronize the failed log every 60 minutes.
<failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute
once-->After the modification, the complete configuration file is as follows:
[root@node2 conf]# more confxml.xml <?xml version="1.0" encoding="ISO-8859-1"?> <head version="2.5"> <host hostip="localhost" port="8008"></host> <debug start="false"/> <fileSystem xfs="false"/> <filter start="false"> <exclude expression="(.*)\.svn"></exclude> <exclude expression="(.*)\.gz"></exclude> <exclude expression="^info/*"></exclude> <exclude expression="^static/*"></exclude> </filter> <inotify> <delete start="true"/> <createFolder start="true"/> <createFile start="false"/> <closeWrite start="true"/> <moveFrom start="true"/> <moveTo start="true"/> <attrib start="false"/> <modify start="false"/> </inotify> <sersync> <localpath watch="/data/tongbu"> <remote ip="192.168.193.128" name="backup"/> <!--<remote ip="192.168.8.39" name="tongbu"/>--> <!--<remote ip="192.168.8.40" name="tongbu"/>--> </localpath> <rsync> <commonParams params="-artuz"/> <auth start="true" users="rsync_backup" passwordfile="/etc/rsync.password"/> <userDefinedPort start="false" port="874"/><!-- port=874 --> <timeout start="true" time="100"/><!-- timeout=100 --> <ssh start="false"/> </rsync> <failLog path="/usr/local/sersync/logs/rsync_fail_log.sh" timeToExecute="60"/><!--default every 60mins execute once--> <crontab start="false" schedule="600"><!--600mins--> <crontabfilter start="false"> <exclude expression="*.php"></exclude> <exclude expression="info/*"></exclude> </crontabfilter> </crontab> <plugin start="false" name="command"/> </sersync> <plugin name="command"> <param prefix="/bin/sh" suffix="" ignoreError="true"/> <!--prefix /opt/tongbu/mmm.sh suffix--> <filter start="false"> <include expression="(.*)\.php"/> <include expression="(.*)\.sh"/> </filter> </plugin> <plugin name="socket"> <localpath watch="/opt/tongbu"> <deshost ip="192.168.138.20" port="8009"/> </localpath> </plugin> <plugin name="refreshCDN"> <localpath watch="/data0/htdocs/cms.xoyo.com/site/"> <cdninfo domainname="ccms.chinacache.com" port="80" username="xxxx" passwd="xxxx"/> <sendurl base="http://pic.xoyo.com/cms"/> <regexurl regex="false" match="cms.xoyo.com/site([/a-zA-Z0-9]*).xoyo.com/images"/> </localpath> </plugin> </head>
The client starts the sersync service:
sersync -r -d -o /usr/local/sersync/conf/confxml.xml
Client validates:
cd /data/tongbu/ for i in `seq 100`;do mkdir $i;done [root@node2 tongbu]# ls 1 12 16 2 23 27 30 34 38 41 45 49 52 56 6 63 67 70 74 78 81 85 89 92 96 10 13 17 20 24 28 31 35 39 42 46 5 53 57 60 64 68 71 75 79 82 86 9 93 97 100 14 18 21 25 29 32 36 4 43 47 50 54 58 61 65 69 72 76 8 83 87 90 94 98 11 15 19 22 26 3 33 37 40 44 48 51 55 59 62 66 7 73 77 80 84 88 91 95 99
View on a synchronous server:
[root@node1 filedir]# cd /data/filedir [root@node1 filedir]# ls 1 12 16 2 23 27 30 34 38 41 45 49 52 56 6 63 67 70 74 78 81 85 89 92 96 hosts 10 13 17 20 24 28 31 35 39 42 46 5 53 57 60 64 68 71 75 79 82 86 9 93 97 100 14 18 21 25 29 32 36 4 43 47 50 54 58 61 65 69 72 76 8 83 87 90 94 98 11 15 19 22 26 3 33 37 40 44 48 51 55 59 62 66 7 73 77 80 84 88 91 95 99 //Verification success!!!
Command parameter description
| Sersync parameter | Explain |
| ./sersync -r | - The function of R parameter is to synchronize the main server directory with the remote target machine directory before real-time monitoring is started; if it is necessary to synchronize all existing files or directories in the main server directory to the remote end before running sersync, it is necessary to run sersync with the-r parameter to synchronize the local and remote whole once. Note: If the filter is set, that is, in the xml file, the filter is true, then the - r parameter can not be used for overall synchronization for the time being. |
| ./sersync -o xx.xml | No - o parameter specified: sersync uses the default configuration file confxml.xml under the sersync executable directory Specify - o parameters: You can specify multiple different configuration files to achieve data synchronization for multiple processes and multiple instances of sersync |
| ./sersync -n num | - n parameter: specify the total number of threads in the default thread pool; For example:. / sersync-n 5 specifies the total number of threads to be 5. If not specified, the default number of thread pools to start is 10. If the cpu is used too high, the parameter can be lowered. If the machine configuration is high, the default total number of threads can be increased to improve synchronization efficiency. |
| ./sersync -d | - D parameter is: background service, usually use-r parameter to synchronize the local to remote whole, then run this parameter in the background to start daemon real-time synchronization; in the first global synchronization, the-d and-r parameters are often used jointly; |
./sersync -m pluginName | - M parameter: no synchronization, only plug-in. / sersync-m plugin Name For example:. / sersync-m command, instead of synchronizing the remote target server after monitoring the event, runs the command plug-in directly. |
| Instructions for the use of composite commands: | |
| -n 8 -o liubl.xml -r -d | Several parameters can be used together. For example,. / sersync-n 16-o config. xml-r-d indicates that 16 thread pool threads are set, and liubl.xml is designated as configuration file. Before real-time monitoring, an overall synchronization is made to run in the background in a daemon mode. |
| ./sersync --help | Unfortunately, it didn't check for help (2 paths if needed, either source code or self-test for validation) |
sersync service profile parameters are detailed:
1. The annotation of the XML configuration file does not use "#", but <! -- the annotation content is in the middle - >" 2.Debug Open Switch: <debug start="false"/> Setting it to true means opening debug mode, and printing inotify time with rsync synchronization command on the console where sersync is running. 3.XFS File System Switch: <fileSystem xfs="false"/> For users of xfs file system, this option needs to be turned on in order to work properly with sersync. 4. File filtering function of filter <filter start="false"> <exclude expression="(.*)\.svn"></exclude> <exclude expression="(.*)\.gz"></exclude> <exclude expression="^info/*"></exclude> <exclude expression="^static/*"></exclude> </filter> Exclude some files, no need 5. The state of inotify <inotify> <delete start="true"/> <createFolder start="true"/> <createFile start="false"/> <closeWrite start="true"/> <moveFrom start="true"/> <moveTo start="true"/> <attrib start="false"/> <modify start="false"/> </inotify> For most applications, we can try to set createFile as false to improve performance and reduce rsync communication. Because the creation event and close_write event will occur when the file is copied to the monitor directory, if the create event is closed, only the close_write time at the end of the file copy can be monitored, the complete synchronization of the file can also be achieved. Note: If creatFolder is set to false, the generated directory will not be monitored, nor will the subfiles and subdirectories in the directory be monitored; so unless special needs are required, open it; by default, both the creation (directory) event and the deletion (directory) event will be monitored, if there is no need to delete far from the project. The file (directory) of the target server can set the delete parameter to false, and the delete event can not be monitored.