Introduction and deployment of keepalived

Keywords: Linux keepalived

1. Introduction to keepalived

Keepalived software was originally designed for LVS load balancing software to manage and monitor the status of each service node in LVS cluster system. Later, it added VRRP function that can realize high availability. Therefore, in addition to managing LVS software, keepalived can also be used as high availability solution software for other services (such as Nginx, Haproxy, MySQL, etc.).

Keepalived software mainly realizes high availability through VRRP protocol. VRRP is the abbreviation of virtual router redundancy protocol. The purpose of VRRP is to solve the problem of single point of failure of static routing. It can ensure that the whole network can run continuously when individual nodes are down.

Therefore, on the one hand, kept has the function of configuring and managing LVS, and also has the function of health inspection for the nodes under LVS. On the other hand, it can also realize the high availability of system network services.

Official website: http://www.keepalived.org/

1.2 functions of keepalived

  • Manage LVS load balancing software
  • Realize the health check of LVS cluster nodes
  • High availability as a system network service (failover)

1.3 principle of keepalived high availability failover

Failover between Keepalived high availability services is realized through VRRP.

When the Keepalived service works normally, the primary Master node will continuously send messages to the standby node (multicast) The heartbeat message is used to tell the standby Backup node that it is still alive. When the primary Master node fails, it cannot send the heartbeat message, so the standby node cannot continue to detect the heartbeat from the primary Master node, so it calls its own takeover program to take over the IP resources and services of the primary Master node. When the primary Master node recovers, the standby Backup node returns The IP resources and services taken over by the primary node in case of failure will be released and restored to the original standby role.

  1. VRRP: its full name is Virtual Router Redundancy Protocol, and its Chinese name is virtual routing redundancy protocol. The emergence of VRRP is to solve the problem of static single point of failure. VRRP gives the routing task to a VRRP router through a campaign mechanism.

  2. VRRP uses IP Multicast (default multicast address (224.0_0.18)) to realize the communication between high availability pairs.

  3. During operation, the master node contracts and the standby node receives the packets. When the standby node cannot receive the data packets sent by the master node, it starts the takeover program to take over the open source of the master node. There can be multiple standby nodes competing through priority, but generally there are one pair in the operation and maintenance of the Keepalived system.

  4. VRRP uses encryption protocol to encrypt data, but Keepalived officials still recommend configuring authentication type and password in plaintext.

1.4 working principle of kept

Keepalived high availability communicates through VRRP. VRRP determines the active and standby through the election mechanism. The primary node has higher priority than the standby. Therefore, when working, the primary node will give priority to obtaining all resources, and the standby node is in the waiting state. When the primary node hangs up, the standby node will take over the resources of the primary node, and then provide services on behalf of the primary node.

Between Keepalived services, only the master server will always send VRRP broadcast packets and tell the standby server that it is still alive. At this time, the standby server will not occupy the master. When the master is unavailable, that is, the standby server cannot monitor the broadcast packets sent by the master, it will start relevant services to take over resources to ensure business continuity. The fastest takeover speed can be less than 1 second.

keepalived deploy high availability httpd

Environmental description:

system information host nameIP
centos7master192.168.216.204
centos7slave192.168.216.179

The VIP (virtual ip) address is tentatively 192.168.216.250. For details, see previous articles LVS

  • Install master keepalived
//Turn off the firewall, selinux
[root@master ~]# systemctl disable --now firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@master ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:firewalld(1)
[root@master ~]# cat /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted 


[root@master ~]# setenforce 0
[root@master ~]# getenforce 
Permissive

//Configure network source
[root@master ~]# curl -o /etc/yum.repos.d/CentOS7-Base-163.repo http://mirrors.163.com/.help/CentOS7-Base-163.repo
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1572  100  1572    0     0   9684      0 --:--:-- --:--:-- --:--:--  9703
[root@master ~]# sed -i 's/\$releasever/7/g' /etc/yum.repos.d/CentOS7-Base-163.repo
[root@master ~]# sed -i 's/^enabled=.*/enabled=1/g' /etc/yum.repos.d/CentOS7-Base-163.repo

//Install related components
[root@master ~]#  yum -y install epel-release vim wget gcc gcc-c++

//Install keepalived
[root@master ~]# yum -y install keepalived
[root@master ~]# rpm -ql keepalived
/etc/keepalived      //configure directory
/etc/keepalived/keepalived.conf  //Master profile
/etc/sysconfig/keepalived
/usr/bin/genhash
/usr/lib/systemd/system/keepalived.service  //Service control document
/usr/libexec/keepalived
/usr/sbin/keepalived   //Command storage location

Use the same method to install keepalived on the standby server

  • Installed standby keepalived
//Turn off the firewall, selinux
[root@slave ~]# systemctl disable --now firewalld.service 
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@slave ~]# sed -ri 's/^(SELINUX=).*/\1disabled/g' /etc/selinux/config
[root@slave ~]# setenforce 0

//Configure network source
[root@slave ~]# curl -o /etc/yum.repos.d/CentOS7-Base-163.repo http://mirrors.163.com/.help/CentOS7-Base-163.repo
[root@slave ~]# sed -i 's/\$releasever/7/g' /etc/yum.repos.d/CentOS7-Base-163.repo
[root@slave ~]# sed -i 's/^enabled=.*/enabled=1/g' /etc/yum.repos.d/CentOS7-Base-163.repo

[root@slave ~]# yum -y install epel-release vim wget gcc gcc-c++

//Install keepalived
[root@slave ~]# yum -y install keepalived

Install httpd on the active and standby machines respectively

  • Installing httpd on the master
//Install httpd
[root@master ~]# yum -y install httpd

//Start and set the startup self start
[root@master ~]# systemctl enable --now httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@master ~]# ss -antl
State       Recv-Q Send-Q                                 Local Address:Port                                                Peer Address:Port              
LISTEN      0      128                                                *:22                                                             *:*                  
LISTEN      0      100                                        127.0.0.1:25                                                             *:*                  
LISTEN      0      128                                               :::80                                                            :::*                  
LISTEN      0      128                                               :::22                                                            :::*                  
LISTEN      0      100                                              ::1:25                                                            :::*     

//Configure web site
[root@master ~]# echo 'this is master' > /var/www/html/index.html
[root@master ~]# curl 192.168.216.204
this is master
  • Installing httpd on slave
[root@slave ~]# yum -y install httpd
[root@slave ~]# systemctl enable --now httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@slave ~]# echo 'this is slave' > /var/www/html/index.html
[root@slave ~]# curl 192.168.216.179
this is slave
[root@slave ~]# curl 192.168.216.204
this is master
  • Configure master keepalived
[root@master ~]# vim /etc/keepalived/keepalived.conf
[root@master ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass lll
    }
    virtual_ipaddress {
       192.168.216.250
    }
}

virtual_server 192.168.216.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.216.204 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.216.179 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}


//Restart for configuration to take effect
[root@master ~]# systemctl restart keepalived
[root@master ~]# systemctl enable keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.
  • Configure standby keepalived
[root@slave ~]# vim /etc/keepalived/keepalived.conf
[root@slave ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb02
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass lll
    }
    virtual_ipaddress {
        192.168.216.250
    }
}

virtual_server 192.168.216.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.216.204 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.216.179 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}


//Make configuration effective
[root@slave ~]# systemctl restart keepalived
[root@slave ~]# systemctl enable keepalived
Created symlink from /etc/systemd/system/multi-user.target.wants/keepalived.service to /usr/lib/systemd/system/keepalived.service.


//View VIP
[root@master ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:82:b6:d0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.216.204/24 brd 192.168.216.255 scope global noprefixroute dynamic ens33
       valid_lft 1486sec preferred_lft 1486sec
    inet 192.168.216.250/32 scope global ens33 //VIP here
       valid_lft forever preferred_lft forever
    inet6 fe80::78af:cdcb:c379:1773/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@slave ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:b8:1e:94 brd ff:ff:ff:ff:ff:ff
    inet 192.168.216.179/24 brd 192.168.216.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feb8:1e94/64 scope link 
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:b6:a9:b9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:b6:a9:b9 brd ff:ff:ff:ff:ff:ff

Modify kernel parameters and enable listening VIP function

//Modify kernel parameters on master
[root@master ~]# echo 'net.ipv4.ip_nonlocal_bind = 1' >>/etc/sysctl.conf 
[root@master ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
[root@master ~]# cat /proc/sys/net/ipv4/ip_nonlocal_bind
1

//Modify kernel parameters on slave
[root@slave ~]# echo 'net.ipv4.ip_nonlocal_bind = 1' >>/etc/sysctl.conf
[root@slave ~]# sysctl -p
net.ipv4.ip_nonlocal_bind = 1
[root@slave ~]# cat /proc/sys/net/ipv4/ip_nonlocal_bind
1

Let keepalived monitor the status of httpd load balancer through script

//Script on master
[root@master ~]# mkdir /scripts
[root@master scripts]# vim check_m.sh
[root@master scripts]# cat check_m.sh 
#!/bin/bash
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
if [ $httpd_status -lt 1 ];then
    systemctl stop keepalived
fi

[root@master scripts]# chmod +x check_m.sh 
[root@master scripts]# vim notify.sh
[root@master scripts]# cat notify.sh 
#!/bin/bash
VIP=$2
sendmail (){
        subject="${VIP}'s server keepalived state is translate"
        content="`date +'%F %T'`: `hostname`'s state change to master"
        echo $content | mail -s "$subject" 593582553@qq.com
}
case "$1" in
  master)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
        if [ $httpd_status -lt 1 ];then
            systemctl start httpd
        fi
        sendmail
  ;;
  backup)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
        if [ $httpd_status -gt 0 ];then
            systemctl stop httpd
        fi
  ;;
  *)
        echo "Usage:$0 master|backup VIP"
  ;;
esac

[root@master scripts]# chmod +x notify.sh
[root@master scripts]# ll
 Total consumption 8
-rwxr-xr-x. 1 root root 142 10 June 22 00:27 check_m.sh
-rwxr-xr-x. 1 root root 662 10 June 22 00:28 notify.sh


//Script on slave
[root@slave ~]# mkdir /scripts
[root@slave ~]# cd /scripts/
[root@slave scripts]# vim notify.sh
[root@slave scripts]# cat notify.sh 
#!/bin/bash
VIP=$2
sendmail (){
        subject="${VIP}'s server keepalived state is translate"
        content="`date +'%F %T'`: `hostname`'s state change to master"
        echo $content | mail -s "$subject" 1344933566@qq.com
}
case "$1" in
  master)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
        if [ $httpd_status -lt 1 ];then
            systemctl start httpd
        fi
        sendmail
  ;;
  backup)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep '\bhttpd\b'|wc -l)
        if [ $httpd_status -gt 0 ];then
            systemctl stop httpd
        fi
  ;;
  *)
        echo "Usage:$0 master|backup VIP"
  ;;
esac

[root@slave scripts]# chmod +x notify.sh
[root@slave scripts]# ll
 Total consumption 4
-rwxr-xr-x. 1 root root 663 10 September 21:32 notify.sh
  • Configure the configuration of keepalived to join the monitoring script
//Configure master keepalived
[root@master scripts]# vim /etc/keepalived/keepalived.conf
[root@master scripts]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01    ## It is very important to identify the hostname of this machine
}

vrrp_script httpd_check {
    script "/scripts/check_m.sh"  ## Script location to execute
    interval 1   ## Detection interval
    weight -20    ## If the condition holds, the weight is reduced by 20
}

vrrp_instance VI_1 {
    state MASTER    ## It is very important to identify the host and BACKUP the standby
    interface ens33    ## Very important, network card name
    virtual_router_id 52    ## Very important. Virtual route ID number (primary and standby nodes should be the same)
    priority 100   ## Priority (0-254). Generally, the priority of the host is greater than that of the standby
    advert_int 1   ## The sending interval of active and standby information must be the same between the two nodes. The default is 1 second
    authentication {
        auth_type PASS
        auth_pass lll  Certificate matching, setting authentication type and password, MASTER and BACKUP You must use the same password to communicate properly
    }
    virtual_ipaddress {  
        192.168.216.250   ##Virtual ip, multiple can be specified
    }
    track_script {    ## Script to check the health of haproxy
        httpd_check
    }
    notify_master "/scripts/notify.sh master 192.168.216.250"
    notify_backup "/scripts/notify.sh backup 192.168.216.250"
}

virtual_server 192.168.216.250 80 {   ####Detailed configuration of virtual ip
    delay_loop 6  
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.216.179 80 {  ## ## Real ip address of the machine
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.216.204 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

[root@master scripts]# systemctl restart keepalived
  • Configure standby keepalived
    Backup does not need to check whether httpd is normal. Start httpd when upgrading to MASTER and close it when demoting to backup
[root@slave scripts]# vim /etc/keepalived/keepalived.conf
[root@slave scripts]# cat /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   router_id lb02
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 52
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass lll
    }
    virtual_ipaddress {
        192.168.216.250
    }
    notify_master "/scripts/notify.sh master 192.168.216.250"
    notify_backup "/scripts/notify.sh backup 192.168.216.250"
}

virtual_server 192.168.216.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.216.204 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.216.179 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

[root@slave scripts]# systemctl restart keepalived

Visit the web page to see

Simulate primary server downtime

[root@master scripts]# systemctl stop  keepalived
[root@master scripts]# systemctl stop httpd



//The backup server receives the mail and the vip is transferred
[root@slave scripts]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:b8:1e:94 brd ff:ff:ff:ff:ff:ff
    inet 192.168.216.179/24 brd 192.168.216.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.216.250/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feb8:1e94/64 scope link 
       valid_lft forever preferred_lft forever
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:b6:a9:b9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:b6:a9:b9 brd ff:ff:ff:ff:ff:ff
 You are /var/spool/mail/root Mail in

[root@slave scripts]# curl 192.168.216.250
this is slave
 You are /var/spool/mail/root New messages in

Posted by mike97gt on Thu, 21 Oct 2021 12:43:16 -0700

Hot Keywords

©2024 Programmer Group