1. NAT mode topology and working principle
There is a scheduler in front of a group of servers, which are connected through Switch/HUB. These servers provide the same network service and content, that is, no matter which server the request is sent to, the execution result is the same. The content of the service can be copied to the local hard disk of each server, shared through a network file system (such as NFS), or provided through a distributed file system.
When the client accesses the network service through the Virtual IP Address, the request message arrives at the dispatcher. The dispatcher selects a server from a group of real servers according to the connection scheduling algorithm, and the destination address of the message is virtual IP The address is rewritten as the address of the selected server, the destination port of the message is rewritten as the corresponding port of the selected server, and finally the modified message is sent to the selected server. At the same time, the dispatcher records the connection in the connection Hash table. When the next message of the connection arrives, the address and port of the original selected server can be obtained from the connection Hash table, and the same rewriting operation can be carried out, and the message will be transmitted to the original selected server. When the response message from the real server passes through the dispatcher, the dispatcher changes the source address and source port of the message to Virtual IP Address and corresponding port, and then sends the message to the user. We introduce a state machine into the connection, different messages will make the connection in different states, different states have different timeout values. In TCP connection, state migration is performed according to the standard TCP finite state machine; in UDP, we only set one UDP state. The timeout values of different states can be set. By default, the timeout of SYN state is 1 minute, that of ESTABLISHED state is 15 minutes, that of FIN state is 1 minute, and that of UDP state is 5 minutes. When the connection terminates or times out, the scheduler removes the connection from the connection Hash table.
In this way, what the customer sees is only the services provided on the Virtual IP Address, and the structure of the server cluster is transparent to the user. For the rewritten message, the incremental adjustment Checksum algorithm is applied to adjust the value of TCP Checksum, avoiding the cost of scanning the whole message to calculate Checksum.
2. Types of scheduling algorithms
Weighted Round Robin
Weighted Least Connections
Locality based least connections
Locality based least connections with replication
Shortest expected delay
3. Case implementation
All the traffic with the IP address of 188.8.131.52 and the port of 80 for client access is on the real servers 172.16.0.2:80 and 172.16.0.3:80, which are scheduled by load balancing
Round Robin, polling algorithm, the first to 2, the second to 3, the third to 2, the fourth to 3
This test machine: Centos Linux 8.1.1911
1. Address configuration (install software first)
The network card of 172.16.0.0/24 is host only;
184.108.40.206/24 is NAT;
The scheduler requires a dual network card.
Note: in real server NAT mode, the gateway should point to the internal address of the scheduler, which is 172.16.0.1 in the figure.
2. Turn off firewall and selinux
All virtual machines are executed, and the real environment needs to be treated in a real way. When doing experiments, they will be shut down.
Turn off firewall
systemctl stop firewalld
systemctl disable firewalld
3. Clock synchronization
Time for all real server s to synchronize the scheduler.
4. Software installation
View kernel modules
lsmod | grep ip_vs
Installation management tools
yum install ipvsadm
real server installs httpd service
real server 1
[root@node1 ~]# echo "this is real server node1" >> /var/www/html/index.html [root@node1 ~]# systemctl start httpd [root@node1 ~]# systemctl enable httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. [root@node1 ~]#
real server 2
[root@node2 ~]# echo "this is real server node2" >> /etc/httpd/index.html [root@node2 ~]# systemctl start httpd [root@node2 ~]# systemctl enable httpd Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service. [root@node2 ~]#
4. Configure scheduler
SYNOPSIS ipvsadm -A|E virtual-service [-s scheduler] [-p [timeout]] [-M netmask] [-b sched-flags] ipvsadm -D virtual-service ipvsadm -C ipvsadm -R ipvsadm -S [-n] ipvsadm -a|e virtual-service -r server-address [-g|i|m] [-w weight] [-x upper] [-y lower] ipvsadm -d virtual-service -r server-address ipvsadm -L|l [virtual-service] [options] ipvsadm -Z [virtual-service] ipvsadm --set tcp tcpfin udp ipvsadm --start-daemon state [daemon-options] [--syncid syncid] ipvsadm --stop-daemon state ipvsadm -h
This parameter is missing in addition and modification. The old version has Oh.
-t: TCP protocol
-u: UDP protocol
Add a cluster service
ipvsadm -A -t 220.127.116.11:80 -s rr
Add real server
ipvsadm -a -t 18.104.22.168:80 -r 172.16.0.2:80 -m
ipvsadm -a -t 22.214.171.124:80 -r 172.16.0.3:80 -m
[root@filesystem ~]# ipvsadm -l -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 126.96.36.199:80 rr -> 172.16.0.2:80 Masq 1 0 0 -> 172.16.0.3:80 Masq 1 0 0 [root@filesystem ~]#
[root@filesystem ~]# ipvsadm-save > /root/ipvsadm.bak [root@filesystem ~]# cat ipvsadm.bak -A -t filesystem.server.com:http -s rr -a -t filesystem.server.com:http -r 172.16.0.2:http -m -w 1 -a -t filesystem.server.com:http -r 172.16.0.3:http -m -w 1 [root@filesystem ~]#
[root@filesystem ~]# ipvsadm -C # -C is the meaning of the empty rule [root@filesystem ~]# ipvsadm -R < ipvsadm.bak [root@filesystem ~]# ipvsadm IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP filesystem:http rr -> 172.16.0.2:http Masq 1 0 0 -> 172.16.0.3:http Masq 1 0 0 [root@filesystem ~]#
As we can see from the figure, access to our scheduler (that is, the address to provide external services, and switch back and forth on the internal real server). Load sharing)
In this way, the customer thinks that they get the correct response from the 188.8.131.52:80 service, without knowing whether the request is processed by server 172.16.0.2 or server 172.16.0.3.
There is always one on the way to study and keep fit