1. Briefly describe the four cluster characteristics and usage scenarios of lvs
LVS has three load balancing modes, VS/NAT (nat mode), VS/DR (routing mode), VS/TUN (tunnel mode), VS/FULLNAT 1. NAT mode (VS-NAT) Principle: The destination address of the IP header of a packet sent by a client is replaced by the IP address of one of the RSs on the load balancer and sent to the RS for processing. After the RS is processed, the data is handed over to the load balancer. The load balancer changes the original IP address of the packet to its own IP address and changes the destination address to the client IP address during this period, whether it is incoming traffic or outgoing traffic.All must pass through the load balancer Benefits: The physical servers in the cluster can use any TCP/IP operating system, and only the load balancer needs a legitimate IP address Disadvantages: Limited scalability.When server nodes (ordinary PC servers) grow too large, the load balancer becomes a bottleneck for the whole system Because the flow direction of all request and answer packages passes through the load balancer.When there are too many server nodes A large number of packets converge on the load balancer, and the speed will be slower! II. IP Tunneling Mode (VS-TUN) Principle: First of all, it is important to know that most Internet services on the Internet have very short request packets, while reply packets are usually very large. Tunneling mode is to unpack the packet from the client, encapsulate a new IP header tag (destination IP only) and send it to RSRS, then unpack the packet header, restore the packet, and return it to the client directly after processing, without passing through the load balancer.Note that since RS needs to restore the packets sent by the load balancer, it must support the IPTUNNEL protocol, so the option to support IPTUNNEL must be compiled in the RS kernel Advantages: Load balancer is only responsible for distributing request packages to back-end node servers, and RS sends reply packages directly to users. Therefore, it reduces the large data flow of the load balancer. Load balancer is no longer the bottleneck of the system and can handle a large number of requests. A load balancer can distribute for many RS.And you can distribute in different regions by running on the public network. Disadvantages: Tunnel-mode RS nodes require legitimate IP, which requires all servers to support the IP Tunneling protocol. Servers may be limited to some Linux systems 3. Direct Routing Mode (VS-DR) Principle: Load Balancer and RS both use the same IP to serve external services, but only DR responds to ARP requests. All RSs remain silent to ARP requests for their own IP. In other words, gateways will direct all requests for this service IP to DR, and after DR receives a packet, it will find out the corresponding RS according to the scheduling algorithm.Change the destination MAC address to the MAC of RS (because IP is consistent) and distribute the request to this RS When RS receives the packet, after processing, because IP is consistent, returning the data directly to the client is equivalent to receiving the packet directly from the client and returning it directly to the client after processing Since the load balancer has to change the two-tier headers, it is also easy to understand that the load balancer and RS must be in a broadcast domain and on the same switch. Advantages: Like TUN (Tunnel Mode), load balancers only distribute requests, and reply packages are returned to clients via a separate routing method. VS-DR implementation does not require a tunneling structure, so most operating systems can be used as physical servers. Disadvantages: (Not to mention a disadvantage, but an insufficiency) Requires that the network card of the load balancer must be on a physical segment with the physical network card. 4. fullnat mode lvs-fullnat: Forward by simultaneously modifying the source and destination IP addresses of the request message (1) VIP is a public network address, RIP and DIP are private network addresses and are usually not on the same IP network; therefore, RIP gateways generally do not point to DIP (2) The request message source address that RS receives is DIP, so it only needs to respond to DIP; however, Director sends it to Client (3) Request and response messages are sent via Director (4) Support port mapping Note: This type of kernel s is not supported by default
2. Describe how LVS-DR works and configure the implementation
Principle: Load Balancer and RS both use the same IP to serve external services, but only DR responds to ARP requests. All RSs remain silent to ARP requests for their own IP. In other words, gateways will direct all requests for this service IP to DR, and after DR receives a packet, it will find out the corresponding RS according to the scheduling algorithm.Change the destination MAC address to the MAC of RS (because IP is consistent) and distribute the request to this RS When RS receives the packet, after processing, because IP is consistent, returning the data directly to the client is equivalent to receiving the packet directly from the client and returning it directly to the client after processing Since the load balancer has to change the two-tier headers, it is also easy to understand that the load balancer and RS must be in a broadcast domain and on the same switch.
Planning: c1,c2,c3,c4 are centos7.6 c1 client c2 vs c3 web1 c4 web2
2.1 install web The server
[root@c3 ~]# yum install httpd -y
[root@c3 ~]# echo rs1 > /var/www/html/index.html
[root@c3 ~]# systemctl start httpd
[root@c3 ~]# curl c3
rs1
[root@c4 ~]# yum install httpd -y
[root@c4 ~]# echo rs2 > /var/www/html/index.html
[root@c4 ~]# systemctl start httpd
[root@c4 ~]# curl c4
rs2
2.2 To configure rs The server
[root@c3 ~]# yum install net-tools -y ###The net-tools package needs to be installed to use the ifconfig command
[root@c3 ~]# cat rs.sh
#!/bin/bash
vip=10.0.0.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask #broadcast $vip up
#route add -host $vip dev $dev
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
;;
*)
echo "Usage: $(basename $0) start|stop"
exit 1
;;
esac
[root@c3 ~]# sh rs.sh start
[root@c3 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:f1:37:a8 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.244/24 brd 10.1.1.255 scope global noprefixroute dynamic eth0
valid_lft 14582sec preferred_lft 14582sec
inet6 fe80::5025:c937:77d0:2b28/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@c4 ~]# yum install net-tools -y
[root@c4 ~]# sh rs.sh start
[root@c4 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 00:0c:29:05:32:f0 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.245/24 brd 10.1.1.255 scope global noprefixroute dynamic eth0
valid_lft 16671sec preferred_lft 16671sec
inet6 fe80::96c3:3cc3:b39e:dee3/64 scope link noprefixroute
valid_lft forever preferred_lft forever
2.3 To configure vs The server
[root@c2 ~]# yum install -y ipvsadm ###Install lvs package
[root@c2 ~]# cat vs.sh
#!/bin/bash
vip='10.0.0.100'
iface='lo:1'
mask='255.255.255.255'
port='80'
rs1='10.1.1.244'
rs2='10.1.1.245'
scheduler='rr' ###To test for easy results, rr polling algorithm is used
type='-g'
case $1 in
start)
ifconfig $iface $vip netmask $mask #broadcast $vip up
iptables -F
ipvsadm -A -t ${vip}:${port} -s $scheduler
ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1
ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1
;;
stop)
ipvsadm -C
ifconfig $iface down
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
esa
[root@c2 ~]# sh vs.sh start
[root@c2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 rr
-> 10.1.1.244:80 Route 1 0 0
-> 10.1.1.245:80 Route 1 0 0
2.4 Test:
[root@c1 ~]# route -n ###No route to 10.0.0.0 segment
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.1.1.254 0.0.0.0 UG 100 0 0 eth0
10.1.1.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
[root@c1 ~]# curl 10.0.0.100
^C
[root@c1 ~]# route add -host 10.0.0.100 dev eth0 ###Add routing to 10.0.0.0 segment
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs23. Implement high availability of LVS+Keepalived
keepalive uses vrrp (Virtual Routing Protocol) for load redundancy.A router claims to be the master router through periodic multicast announcements, and then compares its priority with that of routers in the network to select the master and standby routers.The primary router provides the corresponding routing function, the standby router selects a new primary router to provide services compared with the priority again when the primary router fails, and the rest become backup routers.This scenario is based on subsection 2, where c2 is the primary router and c5 is the standby router
3.1 Realization c2 and c5 Mutually Secret Login
[root@c2 ~]# ssh-keygen -t rsa -P "" ###Generate Key
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:C1wDPspULjsjJQs/hSUjac50V4BLQXCQFkwxbViT/DM root@c2
The key's randomart image is:
+---[RSA 2048]----+
|+O#*=.=. |
|.Oo&.= . |
|B * O + o |
| = O E o . |
| = * = S |
| o o . . |
| . |
| |
| |
+----[SHA256]-----+
[root@c2 ~]# ssh-copy-id c5 ###Transfer Key
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'c5 (10.1.1.246)' can't be established.
ECDSA key fingerprint is SHA256:ilZ46J85JC8Xhr2dVvYsUxMGyj17SDhD6/JrhmNy6GY.
ECDSA key fingerprint is MD5:2f:c5:a9:d6:d7:5f:5e:4e:c3:94:7c:92:3a:d2:55:63.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@c5's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'c5'"
and check to make sure that only the key(s) you wanted were added.
[root@c2 ~]# ssh c5 ###Test Secret Login
Last login: Mon May 25 21:40:03 2020 from 192.168.10.45
[root@c5 ~]#
[root@c5 ~]# ssh-keygen -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:abVEGoN7+mbpGU0aZY4VssjdndMC+cjLZYK5Icy+S/U root@c5
The key's randomart image is:
+---[RSA 2048]----+
| .+ +. |
| ..o O.+ o |
| oo.++Bo= . |
| = =X+.+o |
| . +S++= |
| oo.*o |
| .oo.E |
| .. =o |
| .=o |
+----[SHA256]-----+
[root@c5 ~]# ssh-copy-id c2
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'c2 (10.1.1.243)' can't be established.
ECDSA key fingerprint is SHA256:dldJTKtxApZyQT/FT6WKQsqKgtf4cPuAxBTiLMFdxSk.
ECDSA key fingerprint is MD5:1a:07:07:69:3f:0e:94:b3:f3:c5:04:dc:73:6b:ba:3e.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@c2's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'c2'"
and check to make sure that only the key(s) you wanted were added.
[root@c5 ~]# ssh c2
Last login: Mon May 25 21:40:01 2020 from 192.168.10.45
[root@c2 ~]#
3.2 Installation and Configuration keepalived
3.2.1 Clean up first c2 On ipvsadm strategy
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 rr
-> 10.1.1.244:80 Route 1 0 0
-> 10.1.1.245:80 Route 1 0 0
[root@c2 ~]# ls
anaconda-ks.cfg original-ks.cfg vs.sh
[root@c2 ~]# sh vs.sh stop
[root@c2 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
3.2.2 install keepalived Services, ipvsadm tool
[root@c2 ~]# yum install keepalived.x86_64 -y
[root@c5 ~]# yum install keepalived -y
[root@c5 ~]# yum install ipvsadm -y
[root@c2 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4 224.0.100.100
}
vrrp_instance VI_1 {
state MASTER
interface bond0
virtual_router_id 5
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.100/24 dev bond0 label bond0:0
}
}
virtual_server 10.0.0.100 80 {
delay_loop 1
lb_algo rr
lb_kind DR
protocol TCP
real_server 10.1.1.244 80 {
weight 1 ###Backend Service Detection
HTTP_GET {
url {
path /
status_code 200
}
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
real_server 10.1.1.245 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@c2 keepalived]# systemctl start keepalived
[root@c2 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 00:0c:29:ba:03:94 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 00:0c:29:ba:03:9e brd ff:ff:ff:ff:ff:ff
7: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:0c:29:ba:03:94 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.243/24 brd 10.1.1.255 scope global noprefixroute bond0
valid_lft forever preferred_lft forever
inet 10.0.0.100/24 scope global bond0:0
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:feba:394/64 scope link
valid_lft forever preferred_lft forever
[root@c5 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node2
vrrp_mcast_group4 224.0.100.100
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 5
priority 80
advert_int 1
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
10.0.0.100/24 dev eth0 label eth0:0
}
}
virtual_server 10.0.0.100 80 {
delay_loop 1
lb_algo rr
lb_kind DR
protocol TCP
real_server 10.1.1.244 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
real_server 10.1.1.245 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
}
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
[root@c5 keepalived]# systemctl start keepalived.service
3.3 test
3.3.1 Test first lvs Is it working
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]#
3.3.2 Stop c2 On keepalived Service, then test if it is dispatched properly
[root@c2 keepalived]# systemctl stop keepalived
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
rs1
[root@c1 ~]# curl 10.0.0.100
rs2
[root@c1 ~]# curl 10.0.0.100
3.3.3 Test Backend Server Health Check
[root@c3 ~]# systemctl stop httpd
[root@c1 ~]# while true;do curl 10.0.0.100;sleep 1;done
rs2
rs1
rs2
rs1
rs2
curl: (7) Failed connect to 10.0.0.100:80; Connection refused
rs2
curl: (7) Failed connect to 10.0.0.100:80; Connection refused
rs2
rs2
rs2
rs2
rs2
rs2
rs2
[root@c3 ~]# systemctl start httpd
[root@c1 ~]# while true;do curl 10.0.0.100;sleep 1;done
rs2
rs2
rs2
rs2
rs1
rs2
rs1
rs2
rs1
rs2
rs1
rs2
rs1