What is Load Balancing
Load Balance, which is called Load Balance in English, refers to a set of servers composed of multiple servers in a symmetrical manner. Each server has an equivalent status, and can provide services independently without the assistance of other servers. Through some load-sharing technology, requests sent from outside are evenly distributed to a server in a symmetrical structure, and the server receiving the request responds to the customer's requests independently. Load balancing can equally distribute client requests to server arrays to provide fast access to important data and solve a large number of concurrent access service problems. This clustering technology can achieve performance close to that of mainframe with minimal investment.
Load balancing is divided into software load balancing and hardware load balancing. The former represents LVS developed by Dr. Ali Zhangwensong, while the latter is a balancing server such as F5. Of course, this is just mentioned, not the focus.
This paper describes various algorithms of "distributing requests from outside evenly to a server in a symmetrical structure", and demonstrates the implementation of each algorithm with Java code. OK, let's get down to the main topic. Before entering the main topic, we first write a class to simulate the Ip list:
public class IpMap { // The list of Ips to be routed, Key for Ip, Value for the weight of the Ip public static HashMap<String, Integer> serverWeightMap = new HashMap<String, Integer>(); static { serverWeightMap.put("192.168.1.100", 1); serverWeightMap.put("192.168.1.101", 1); // Weight 4 serverWeightMap.put("192.168.1.102", 4); serverWeightMap.put("192.168.1.103", 1); serverWeightMap.put("192.168.1.104", 1); // Weight 3 serverWeightMap.put("192.168.1.105", 3); serverWeightMap.put("192.168.1.106", 1); // Weight 2 serverWeightMap.put("192.168.1.107", 2); serverWeightMap.put("192.168.1.108", 1); serverWeightMap.put("192.168.1.109", 1); serverWeightMap.put("192.168.1.110", 1); } }
Round Robin) method
Round Robin method is a polling method. Its code implementation is as follows:
public class RoundRobin { private static Integer pos = 0; public static String getServer() { // Reconstruct a Map to avoid concurrency problems caused by the server's upload and download. Map<String, Integer> serverMap = new HashMap<String, Integer>(); serverMap.putAll(IpMap.serverWeightMap); // Get the Ip address List Set<String> keySet = serverMap.keySet(); ArrayList<String> keyList = new ArrayList<String>(); keyList.addAll(keySet); String server = null; synchronized (pos) { if (pos > keySet.size()) pos = 0; server = keyList.get(pos); pos ++; } return server; } }
Because the address list in serverWeightMap is dynamic, there may be machine on-line, off-line or downtime at any time, so in order to avoid possible concurrency problems, we need to create a new local variable serverMap inside the method, and now copy the content of serverMap to the thread local, in order to avoid being modified by multiple threads. This may lead to new problems. After replication, the modification of serverWeightMap can not be reflected to serverMap. That is to say, in the process of selecting servers in this round, when new servers or offline servers are added, the load balancing algorithm will not be known. It doesn't matter if the server is offline or downtime, then it may access the address that does not exist. Therefore, the service caller needs fault-tolerant processing, such as re-launching a server selection and invoking.
For the position variable pos of the current polling, in order to ensure the order of server selection, it is necessary to lock it during operation, so that only one thread can modify the value of pos at the same time. Otherwise, when pos variables are modified concurrently, the order of server selection cannot be guaranteed, and even the keyList array may be crossed.
The advantage of polling method is that it tries to achieve absolute balance of request transfer.
The disadvantage of polling method is that in order to achieve absolute balance of request transfer, it must pay a considerable price, because in order to ensure the mutually exclusive modification of pos variables, it is necessary to introduce a heavy pessimistic lock synchronized, which will lead to a significant decline in the concurrent throughput of the polling code.
Random method
Through the system random function, one of the servers is selected randomly according to the size of the back-end server list for access. From the theory of probability and statistics, we can know that with the increase of call volume, its actual effect is more and more close to the average distribution of traffic to each back-end server, that is, the effect of polling.
The code implementation of the random method is roughly as follows:
public class Random { public static String getServer() { // Reconstruct a Map to avoid concurrency problems caused by the server's upload and download. Map<String, Integer> serverMap = new HashMap<String, Integer>(); serverMap.putAll(IpMap.serverWeightMap); // Get the Ip address List Set<String> keySet = serverMap.keySet(); ArrayList<String> keyList = new ArrayList<String>(); keyList.addAll(keySet); java.util.Random random = new java.util.Random(); int randomPos = random.nextInt(keyList.size()); return keyList.get(randomPos); } }
The whole code idea is consistent with polling method. First, rebuild serverMap, and then get the list of servers. When server is selected, Random's nextInt method takes a random value in the range of 0-keyList. size () to get a random server address from the list of servers for return. Based on the theory of probability and statistics, the greater the throughput, the closer the effect of random algorithm is to that of polling algorithm.
Source Address Hash Method
The idea of source address hashing is to get the IP address value accessed by the client, calculate a value by hash function, and use this value to modularize the size of the server list. The result is the serial number of the server to be accessed. The code implementation of the source address hashing algorithm is as follows:
public class Hash { public static String getServer() { // Reconstruct a Map to avoid concurrency problems caused by the server's upload and download. Map<String, Integer> serverMap = new HashMap<String, Integer>(); serverMap.putAll(IpMap.serverWeightMap); // Get the Ip address List Set<String> keySet = serverMap.keySet(); ArrayList<String> keyList = new ArrayList<String>(); keyList.addAll(keySet); // In Web applications, it can be obtained by the getRemoteIp method of HttpServlet. String remoteIp = "127.0.0.1"; int hashCode = remoteIp.hashCode(); int serverListSize = keyList.size(); int serverPos = hashCode % serverListSize; return keyList.get(serverPos); } }
The first two parts are the same as polling method and random method. The difference lies in the routing part. By remoteIp, the Hash value of the client is obtained, and the size of the server list is modeled. The result is the index value of the selected server in the server list.
The advantage of source address hashing is that it ensures that the same client IP address will be hashed to the same back-end server until the list of back-end servers changes. According to this feature, stateful session sessions can be established between service consumers and service providers.
The disadvantage of the source address hashing algorithm is that unless the servers in the cluster are very stable, they will not be offline at all. If there are servers online and offline, the probability that the servers routed by the source address hashing algorithm are servers online and offline is very low. If the session is not available, the session may not be retrieved. If the cache is used, the "avalanche" may be triggered. . If this explanation is not suitable for understanding, you can see my previous article MemCache's hyper-detailed interpretation of the consistency Hash algorithm section.
Weight ed polling Round Robin method
Different servers may have different machine configurations and loads of the current system, so their compressive capacity is also different. The machines with high configuration and low load are given higher weights to handle more requests, while the machines with low configuration and high load are given lower weights to reduce their system loads. The weighted polling method can deal with this problem well and assign the request order to the back end according to the weight. The code implementation of the weighted polling method is roughly as follows:
public class WeightRoundRobin { private static Integer pos; public static String getServer() { // Reconstruct a Map to avoid concurrency problems caused by the server's upload and download. Map<String, Integer> serverMap = new HashMap<String, Integer>(); serverMap.putAll(IpMap.serverWeightMap); // Get the Ip address List Set<String> keySet = serverMap.keySet(); Iterator<String> iterator = keySet.iterator(); List<String> serverList = new ArrayList<String>(); while (iterator.hasNext()) { String server = iterator.next(); int weight = serverMap.get(server); for (int i = 0; i < weight; i++) serverList.add(server); } String server = null; synchronized (pos) { if (pos > keySet.size()) pos = 0; server = serverList.get(pos); pos ++; } return server; } }
Similar to polling method, only a weight calculation code is added before the server address is obtained. According to the weight, the address is repeatedly added to the server address list. The larger the weight, the more requests the server obtains per round.
Weight ed Random Random) method
Similar to the weighted polling method, the weighted random method allocates different weights according to the different configuration and load of the back-end server. The difference is that it randomly selects servers by weight, not by order. The code implementation of the weighted random method is as follows:
public class WeightRandom { public static String getServer() { // Reconstruct a Map to avoid concurrency problems caused by the server's upload and download. Map<String, Integer> serverMap = new HashMap<String, Integer>(); serverMap.putAll(IpMap.serverWeightMap); // Get the Ip address List Set<String> keySet = serverMap.keySet(); Iterator<String> iterator = keySet.iterator(); List<String> serverList = new ArrayList<String>(); while (iterator.hasNext()) { String server = iterator.next(); int weight = serverMap.get(server); for (int i = 0; i < weight; i++) serverList.add(server); } java.util.Random random = new java.util.Random(); int randomPos = random.nextInt(serverList.size()); return serverList.get(randomPos); } }
This code is equivalent to a combination of random method and weighted polling method, which is better understood and not explained.
Least number of connections Connections method
The first few methods are trying to achieve a balanced distribution of the number of requests from service consumers. Of course, it is good to do so. They can distribute the workload equally for multiple servers in the back end and maximize the utilization of servers. But is this true? In practice, can the balance of requests really represent the balance of load? This is a question worth thinking about.
The above problem, in another way, is to observe the load of the system from the perspective of the back-end server, rather than the originator of the request. The minimum number of connections method belongs to this category.
The minimum number of connections algorithm is flexible and intelligent. Because the configuration of the back-end server is different, the processing of requests is fast or slow. According to the current connection situation of the back-end server, it dynamically selects the server with the least number of backlog connections to process the current requests, so as to improve the utilization efficiency of the back-end server as much as possible and distribute the load to the server reasonably. Each machine. Because of the summary and perception of the minimum number of connections, the design and implementation of the minimal number of connections is tedious, so we will not mention its implementation here.