3.7 Load Balancing Setup
3.7.1 Environment Architecture
Figure 3.7.1 nginx environment architecture diagram
3.7.2 Install dependency packages
Command:
#yum -y install gcc pcre-devel zlib-devel openssl-devel
3.7.3 Upload the Nginx and Keepalived installation packages to the server
Upload nginx-1.6.2.tar.gz and keepalived-1.2.12.tar.gz to the two load balancing servers/app directories.
3.7.4 Install Nginx
1. Unzip the installation package, command:
#tar –zxvf nginx-1.6.2.tar.gz
2. Enter the nginx-1.6.2 folder and command:
#cd /app/nginx-1.6.2
3. Configure installation software, commands:
#./configure --with-http_stub_status_module
4. Compile and install, commands:
#make && make install
3.7.5 Install Keepalived
1. Unzip the installation package, command:
# tar –zxvf keepalived-1.2.12.tar.gz
2. Enter the nginx-1.6.2 folder and command:
# cd keepalived-1.2.12
3. Configure installation software, commands:
#./configure
4. Compile and install commands:
#make && make install
5. New keepalived service, command:
#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
#mkdir /etc/keepalived
#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
#cp /usr/local/sbin/keepalived /usr/sbin/
3.7.6 Configuring Nginx
The NginX configuration of the two access servers is exactly the same. Modify the following nginx.conf configuration file and replace it with the nginx.conf file in the usr/local/nginx/conf directory of the server.
Be careful:
Load balancing is configured as HTTP, Web Services protocol, which mainly includes Portal access IP and ports and system HTTP, DIP IP and ports used by Web Services protocol.
When DIP and Nginx deploy the same server, be aware to keep the Nginx proxy port and the DIP port inconsistent, that is, the ports in the upstream and server configuration sections of the Nginx configuration file are inconsistent.
3.7.7 Configuring Keepalived
The installation servers are divided into master and slave servers, and the keepalived configuration files for the master and slave servers differ.
1. Configure the configuration file for the master server
Modify the following keepalived-master.conf configuration file contents and change the file name to keepalived.conf, then replace keepalived.conf under the master server/etc/keepalived directory.
2. Configure the configuration file from the server
Modify the following keepalived-backup.conf configuration file contents and change the file name to keepalived.conf, then replace keepalived.conf in the master server/etc/keepalived directory.
3. Verification
(1) Start keepalived on primary and secondary servers: #/etc/init.d/keepalived start
(2) Check on the primary server if virtual IP is already bound: #ip addr, as shown in Figure 3.7.7-1, 192.168.0.63 is the bound virtual IP address.
Figure 3.7.7-1 Binding Virtual IP Address
(3) Stop keepalived: #/etc/init.d/keepalived stop on the primary server and check from the server if virtual IP is bound.
(4) Start keepalived on the master server to see if the master server can take over virtual IP again
3.7.8 Add monitoring scripts
Keepalived supports configuring monitoring scripts to monitor Nginx status.Here, nmap is used to check the nginx port to determine the state of nginx.
1. Install nmap software, command:
Upload nmap-4.76-1.x86_64.rpm to the / app directory of both servers
Installation Software: #rpm - ivh nmap-4.76-1.x86_64.rpm
2. Modify the following chk_nginx.sh monitoring script and upload it to the / opt directory of both servers.
3.7.9 Validation Test
1. Start the nginx and keepalived services of the master and slave servers, respectively.Note that the nginx service is started first, then the keepalivedservice, command:
Enter the / usr/local/nginx/sbin directory, #cd/usr/local/nginx/sbin
Start nginx service, #. /nginx
Start the keepalived service, #/etc/init.d/keepalived start
2. Host Server Test
(1) VIP Access Portal: Google Browser Address Bar Enter Address: http://VIP: Port/integrator/index.shtml .If the keepalived profile is configured with a VIP of 192.168.0.63 and the nginx configuration listening port is 8080, the access address is: http://192.168.0.63:8080/integrator/index.shtml To see if the Portal home page can be successfully loaded, and if it can be successfully loaded, nginx+keepalived is successfully configured;
(2) VIP access to DIP: The system configures a HTTP-LOCAL process and is successfully deployed.Push a message with the Firefox plugin httprequest or soapui.Push HTTP URL is http://VIP:Communications Gateway Configuration Port/ Channel configuration path.Observe message monitoring for successful processing and, if successful, the nginx+keepalived configuration dip load balancing portion is successfully configured.
3. Testing from the server
Stop the nginx service on the master server, same as 3.7.9 2.Test on the master server.If accessible, it means that when the nginx master server hangs up, it can automatically switch to nginx from the server successfully, and the monitoring script works.
Enclosure:
1. ngnix configuration file
#Start a process, usually set equal to the number of cpUs
worker_processes 1;
#Operating mode and maximum number of connections
events {
use epoll; #epoll is a way of multiplexing IO(I/O Multiplexing), but only for linux2.6 or above cores can greatly improve nginx performance
worker_connections 1024; #Maximum number of concurrent links for a single background worker process
}
#Set up an http server to leverage its reverse proxy capabilities to provide load balancing support
http {
#Set the mime type, defined by the mime.type file
include mime.types;
default_type application/octet-stream;
#The sendfile directive specifies whether nginx calls the sendfile function (zero copy mode) to output a file. For normal applications,
#Must be set to on, if used for applications such as downloads, disk IO overload applications can be set to off to balance disk and network I/O processing speed and reduce uptime of the system.
sendfile on;
#tcp_nopush on;
#Connection timeout
#keepalive_timeout 0;
keepalive_timeout 65;
#Set load balancing server list
upstream portal {
ip_hash; #Directs IP requests to the same backend.
#Native Open Port
server 192.168.80.2:8080;
server 192.168.80.3:8080;
}
server {
#Listen on port 8081
listen 8081;
#Define access using www.xx.com
server_name b2b.integrator.com;
#Default Request
location / {
proxy_set_header Host $host: 8081;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://Portal; #Request to go to the list of servers defined by the portal, name needs to be unique
}
}
}
global_defs {
notification_email { #Specify the objects to which keepalived needs to send email s when a switch occurs, one line at a time
ivan.li@sinoservices.com
}
notification_email_from sinovan.admin@sinoservices.com #Specify sender
smtp_server smtp.sinoservices.com #Specify smtp server address
smtp_connect_timeout 30 #Specify smtp connection timeout
router_id nginx_master #An identification of the machine running keepalived
}
#Backup load balancing server name defined earlier
vrrp_instance VI_1 {
#Only MASTER and BACKUP States must be capitalized.
state MASTER
#Port to communicate
interface eth0
#Unique identification of the same VRRP instance.That is, the virtual_router_id of the same vrrp_stance,MASTER and BACKUP are identical.It is also unique throughout vrrp.
virtual_router_id 51
#Weight, the larger the value, the larger the weight.MASTER is greater than SLAVE
priority 101
#Time interval for synchronization checks between MASTER and SLAVE load balancers.Unit is: seconds
advert_int 1
#Authentication methods for MASTER and SLAVE
authentication {
auth_type PASS
auth_pass 1111
}
#VIP
virtual_ipaddress {
192.168.80.6
}
}
virtual_server 192.168.80.6 8080 {
delay_loop 2 #Check the real_server status every 2 seconds
lb_algo wrr #LVS algorithm
lb_kind DR #LVS mode
persistence_timeout 60 #Session Hold Time
protocol TCP
real_server 192.168.0.75 8081 {
weight 3
notify_down /opt/chk_nginx.sh #Script executed after service down load detected
TCP_CHECK {
connect_timeout 10 #Connection timeout
nb_get_retry 3 #Number of reconnections
delay_before_retry 3 #Reconnection Interval Time
connect_port 8080 #Health Check Port
}
}
}3. keepalive configuration from server
global_defs {
notification_email { #Specify the objects to which keepalived needs to send email s when a switch occurs, one line at a time
ivan.li@sinoservices.com
}
notification_email_from sinovan.admin@sinoservices.com #Specify sender
smtp_server smtp.sinoservices.com #Specify smtp server address
smtp_connect_timeout 30 #Specify smtp connection timeout
router_id nginx_backup #An identification of the machine running keepalived
}
#Backup load balancing server name defined earlier
vrrp_instance VI_1 {
#Only MASTER and BACKUP States must be capitalized.
state BACKUP
#Port to communicate
interface eth0
#Unique identification of the same VRRP instance.That is, the virtual_router_id of the same vrrp_stance,MASTER and BACKUP are identical.It is also unique throughout vrrp.
virtual_router_id 51
#Weight, the larger the value, the larger the weight.MASTER is greater than SLAVE
priority 99
#Time interval for synchronization checks between MASTER and SLAVE load balancers.Unit is: seconds
advert_int 1
#Authentication methods for MASTER and SLAVE
authentication {
auth_type PASS
auth_pass 1111
}
#VIP
virtual_ipaddress {
192.168.80.6
}
}
virtual_server 192.168.80.6 8080 {
delay_loop 2 #Check the real_server status every 2 seconds
lb_algo wrr #LVS algorithm
lb_kind DR #LVS mode
persistence_timeout 60 #Session Hold Time
protocol TCP
real_server from nginx The server IP Corresponding service port {
weight 3
notify_down /opt/chk_nginx.sh #Script executed after service down load detected
TCP_CHECK {
connect_timeout 10 #Connection timeout
nb_get_retry 3 #Number of reconnections
delay_before_retry 3 #Reconnection Interval Time
connect_port from nginx Service Correspondence Port #Health Check Port
}
}
}