I haven't written a blog for a long time. Although I haven't written a few articles, I'm still active in the Internet industry... Needless to say, it's useless. Let's share how to synchronize Git services.
Git service we usually use gitlab rake tool to do regular backup. When there is a problem, we can use backup recovery. Is there a need for another environment to deploy a set of GIT service, and need to synchronize with the previous set of GIT service? Even if we do cold backup, we can save recovery time and achieve the purpose of regular recovery exercise.
At present, we meet such a demand in another environment. Of course, we can study and learn the Git API to realize it. However, because the current demand does not require high synchronization and real-time performance, I plan to use a direct, simple and crude way to realize it. In fact, I will take Git backup to another environment for automatic recovery, and we will give the expect tool to solve the interaction problem.
Here are Python and Shell scripts to share as reference learning. Synchronization time can adjust the timing backup time and recovery time. The content is similar and the ideas are basically the same:
#!/usr/bin/python # -*- coding: utf-8 -*- import os import sys import pexpect import paramiko # IP address and user name of backup server git_src_host = "xxx.xxx.xxx.xxx" git_src_user = "xxx" git_src_port = "22" # Address changes according to the environment private_key = paramiko.RSAKey.from_private_key_file('/xxx/xxx/xxx/.ssh/id_rsa') git_bak_dir = "/var/opt/gitlab/backups" # Copy backup to xxx.xxx.xxx.xxx try: ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect( hostname=git_src_host, username=git_src_user, port=git_src_port, password=private_key) cmd=("find " + git_bak_dir + " -type f -ctime -1") stdin, stdout, stderr = ssh.exec_command(cmd) for item in stdout.readlines(): chown_cmd="chown git:git " + git_bak_dir + "/*;chmod 700 " + git_bak_dir rsync_cmd=('rsync -avzP ' + git_src_user + '@' + git_src_host + ':' + item.strip('\n') + ' ' + git_bak_dir) #print(rsync_cmd) os.system(rsync_cmd) os.system(chown_cmd) ssh.close() except Exception, e: print e # Restore backup ls_cmd="ls -t " + git_bak_dir + " | \ head -n 1 | \ sed -r 's/^([0-9]+_[0-9]+_[0-9]+_[0-9]+_.*)_gitlab_backup.tar/\\1/g'" rs=os.popen(ls_cmd) restore_file=rs.read() # print(restore_file) os.system("gitlab-ctl stop unicorn;gitlab-ctl stop sidekiq") expectations=["Do you want to continue (yes/no)?","Do you want to continue (yes/no)?",] child = pexpect.spawn('gitlab-rake gitlab:backup:restore BACKUP=' + restore_file,timeout=1800) child.logfile = sys.stdout while True: try: i = child.expect(expectations) if i == 0: child.sendline('yes') elif i == 1: child.sendline('yes') except pexpect.EOF: print('Exiting fexpect for EOF.') break os.system("gitlab-ctl restart")
#!/bin/bash # git_src_host="xxx.xxx.xxx.xxx" git_src_user="xxx" git_bak_dir="/var/opt/gitlab/backups" git_bak_file=`ssh ${git_src_user}@${git_src_host} "find ${git_bak_dir} -type f -ctime -1"` # Copy backup to xxx.xxx.xxx.xxx for bakfiles in $git_bak_file ; do echo "------------------------------------" rsync -avzP ${git_src_user}@${git_src_host}:${bakfiles} ${git_bak_dir} echo $bakfiles echo "------------------------------------" done chown git:root $git_bak_dir chmod 700 $git_bak_dir # Restore backup restore_file=`ls -t $git_bak_dir | head -n 1 | sed -r 's/^([0-9]+_[0-9]+_[0-9]+_[0-9]+_.*)_gitlab_backup.tar/\1/g'` gitlab-ctl stop unicorn gitlab-ctl stop sidekiq /bin/expect <<EOF set time 30 spawn gitlab-rake gitlab:backup:restore BACKUP=$restore_file expect "Do you want to continue (yes/no)?" { send "yes\r"; } expect "Do you want to continue (yes/no)?" { send "yes\r"; } expect "Do you want to continue (yes/no)?" { send "yes\r"; } expect "Do you want to continue (yes/no)?" { send "yes\r"; } expect "Do you want to continue (yes/no)?" { send "yes\r"; } expect eof EOF #gitlab-ctl reconfigure gitlab-ctl restart