load balancing

We all know that in a micro-service architecture, micro-services always need to call each other to fulfill some of the requirements of the combined business.For example, assembling order details data, since order details contain user information, the order service must invoke the user service to obtain user information.To make remote calls, you need to send network requests, and each microservice may have multiple instances distributed across different machines. When one microservice calls another microservice, you need to distribute the requests evenly across the instances to avoid overloading some instances and some instancesSpace is idle, so there must be a load balancer in this scenario.

There are two main ways to achieve load balancing at present:

1. Service-side load balancing; for example, the most classic use of Nginx as a load balancer.User requests are sent to Nginx before they are distributed to each instance by Nginx through a configured load balancing algorithm, which is called service-side load balancing because it needs to be deployed on the server side as a service.Figure:

2. Client-side load balancing; it is called client-side load balancing because this load balancing method is implemented by the client sending the request, and is also a common load balancing method used to balance requests between services in the current micro-service architecture.Because this way of calling services directly from one another, there is no need for a dedicated load balancer to improve performance and availability.Take MicroService A to invoke MicroService B as an example. Simply put, MicroService A first obtains the invocation address of all instances of MicroService B through the service discovery component, and then selects one of the invocation addresses to request through the locally implemented load balancing algorithm.Figure:

Let's write a very simple client-side load balancer using DiscoveryClient provided by Spring Cloud to get an intuitive view of its workflow. The load balancing strategy used in this example is random with the following code:

package com.zj.node.contentcenter.discovery;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.client.discovery.DiscoveryClient;

import java.util.List;
import java.util.concurrent.ThreadLocalRandom;
import java.util.stream.Collectors;

/**
 * Client side load balancer
 *
 * @author 01
 * @date 2019-07-26
 **/
public class LoadBalance {

    @Autowired
    private DiscoveryClient discoveryClient;

    /**
     * Randomly get the request address of the target microservice
     *
     * @return Request Address
     */
    public String randomTakeUri(String serviceId) {
        // Get the request address of all instances of the target microservice
        List<String> targetUris = discoveryClient.getInstances(serviceId).stream()
                .map(i -> i.getUri().toString())
                .collect(Collectors.toList());
        // Randomly get uri in list
        int i = ThreadLocalRandom.current().nextInt(targetUris.size());

        return targetUris.get(i);
    }
}

Load balancing using Ribbon

What is Ribbon:

• Ribbon is Netflix open source client side load balancer
• Ribbon has a very rich load balancing strategy algorithm built in

Although Ribbon is a small component mainly used for load balancing, there are many interface components for Ribbon, although sparrows are all small.The following table:

Ribbon has eight load balancing policies built in by default, and if you want to customize them, you can implement the IRule interface or AbstractLoadBalancerRule abstract class mentioned in the table above.The built-in load balancing strategy is as follows:

• The default policy rule is ZoneAvoidanceRule

There are two main ways to use Ribbon. One is to use Feign, which integrates Ribbon internally so that Ribbon is not perceived if it is used only in general; the other is to use RestTemplate with Ribbon dependencies and the @LoadBalanced annotation.

Here's a demonstration of the second way to use it. Since the NACS dependencies added to the project already contain Ribbon, no additional dependencies need to be added. First, define a RestTemplate with the following code:

package com.zj.node.contentcenter.configuration;

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

/**
 * bean Configuration Class
 *
 * @author 01
 * @date 2019-07-25
 **/
@Configuration
public class BeanConfig {

    @Bean
    @LoadBalanced  // Add this comment to indicate using Ribbon
    public RestTemplate restTemplate() {
        return new RestTemplate();
    }
}

Then when you use RestTemplate to call other services, you only need to write the service name, not the ip address and port number.Examples include the following:

public ShareDTO findById(Integer id) {
    // Get sharing details
    Share share = shareMapper.selectByPrimaryKey(id);
    // Publisher id
    Integer userId = share.getUserId();
    // Call User Center to get user information
    UserDTO userDTO = restTemplate.getForObject(
            "http://user-center/users/{id} ", //Just write the service name
            UserDTO.class, userId
    );

    ShareDTO shareDTO = objectConvert.toShareDTO(share);
    shareDTO.setWxNickname(userDTO.getWxNickname());

    return shareDTO;
}

If you are not sure about the use of RestTemplate, you can refer to the following article:

• Ways of communication between micro services

Customize Ribbon Load Balance Configuration

In the actual development, we may encounter that the default load balancing strategy can not meet the demand and need to change other load balancing strategies.There are two main ways to configure Ribbon load balancing, either in code or in a configuration file.

Ribbon supports fine-grained configurations, such as I want Micro Service A to use a random load balancing strategy when invoking Micro Service B and a default strategy when invoking Micro Service C. Here's how to implement this fine-grained configuration.

1. First, configure it through code, writing a configuration class to instantiate the specified Load Balancing Policy object:

@Configuration
public class RibbonConfig {

    @Bean
    public IRule ribbonRule(){
        // Random Load Balancing Policy Objects
        return new RandomRule();
    }
}

Then write a configuration class for configuring the Ribbon client that specifies the load balancing policy configured in Ribbon Config when calling user-center, so that fine-grained configuration can be achieved:

@Configuration
// This comment is used to customize the Ribbon client configuration, which is declared as a user-center configuration
@RibbonClient(name = "user-center", configuration = RibbonConfig.class)
public class UserCenterRibbonConfig {
}

It is important to note that RibbonConfig should be defined outside of the main boot class to avoid being scanned by Spring, or the problem of overlapping parent-child context scans can result in a variety of fantastic problems.Here in Ribbon, the configuration class is shared by all Ribbon clients, that is, the load-balancing policy defined in the configuration class is used for both user-center calls and other micro-services, which makes it a global configuration and violates our need for fine-grained configuration.So you need to define it outside the main startup class:

Refer to the official documentation for a description of this issue:

https://cloud.spring.io/spring-cloud-static/Greenwich.SR2/single/spring-cloud.html#_customizing_the_ribbon_client

2. Configuring with a configuration file is simpler. There are no overlapping pits scanned by parent-child contexts without writing code. Simply add the following section of configuration to the configuration file to achieve the equivalent effect of using code configuration above:

user-center:
  ribbon:
    NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RandomRule

Compare the two configurations:

Summary of best practices:

• Use profile configuration whenever possible, and consider using code configuration if the profile does not meet your needs
• Keep as single as possible within the same microservice, for example, use configuration file configurations uniformly, and try not to mix the two to avoid increasing the complexity of locating problems

The configuration described above is fine-grained for a particular Ribbon client, and let's show you how to achieve a global configuration.Simply change the comment to @RibbonClients with the following code:

@Configuration
// This note is used for global configuration
@RibbonClients(defaultConfiguration = RibbonConfig.class)
public class GlobalRibbonConfig {
}

Ribbon is lazy by default, so it will appear slower the first time a request occurs. We can turn on starvation loading by adding the following configuration to the configuration file:

ribbon:
  eager-load:
    enabled: true
    # For which clients to turn on hungry loading, multiple clients use comma-separated (not required)
    clients: user-center

Supports Nacos weights

The above subsections describe the basic use of load balancing and Ribbon. The next sections need to work with Nacos. If you don't know Nacos, you can refer to the following articles:

• Service Discovery Component for Spring Cloud Alibaba - Nacos

On the console page of Nacos Server, you can edit the weight of each instance of a microservice, the list of services - > details - > edit; the default weight is 1, and the larger the weight value, the more preferred it will be called:

Weights are useful in many scenarios, such as a microservice with many instances deployed on machines with different configurations, where you can set lower instance weights for deployments on poorly configured machines and higher instance weights for deployments on better configured machines.A larger portion of requests can be distributed to higher-performance machines.

However, neither Ribbon's built-in load balancing strategy supports Nacos'weights, so we need to customize a load balancing strategy that supports Nacos' weight configuration.Fortunately, the Nacos Client already has built-in load balancing capabilities, so it's easy to implement with the following code:

package com.zj.node.contentcenter.configuration;

import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.naming.NamingService;
import com.alibaba.nacos.api.naming.pojo.Instance;
import com.netflix.client.config.IClientConfig;
import com.netflix.loadbalancer.AbstractLoadBalancerRule;
import com.netflix.loadbalancer.BaseLoadBalancer;
import com.netflix.loadbalancer.Server;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.alibaba.nacos.NacosDiscoveryProperties;
import org.springframework.cloud.alibaba.nacos.ribbon.NacosServer;

/**
 * Load Balancing Strategy Supporting Nacos Weight Configuration
 *
 * @author 01
 * @date 2019-07-27
 **/
@Slf4j
public class NacosWeightedRule extends AbstractLoadBalancerRule {

    @Autowired
    private  NacosDiscoveryProperties discoveryProperties;

    /**
     * Read the configuration file and initialize NacosWeightedRule
     *
     * @param iClientConfig iClientConfig
     */
    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {
        // do nothing
    }

    @Override
    public Server choose(Object key) {
        BaseLoadBalancer loadBalancer = (BaseLoadBalancer) this.getLoadBalancer();
        log.debug("lb = {}", loadBalancer);

        // Requested microservice name
        String name = loadBalancer.getName();
        // Get API s for Service Discovery
        NamingService namingService = discoveryProperties.namingServiceInstance();

        try {
            // When this method is called, the nacos client automatically picks up an instance through a weight-based load balancing algorithm
            Instance instance = namingService.selectOneHealthyInstance(name);
            log.info("Examples selected are: instance = {}", instance);

            return new NacosServer(instance);
        } catch (NacosException e) {
            return null;
        }
    }
}

Then configure it in the configuration file to use the load balancing policy:

user-center:
  ribbon:
    NFLoadBalancerRuleClassName: com.zj.node.contentcenter.configuration.NacosWeightedRule

Think: Why should Spring Cloud Alibaba integrate Ribbon now that Nacos Client has load balancing capabilities?

Same Cluster Priority Call

stay Service Discovery Component for Spring Cloud Alibaba - Nacos The concept and function of clusters have been described in the article, so don't repeat them here, and how to customize load balancing strategies has been described in the previous section, so instead of verbose, just code the implementation as follows:

package com.zj.node.contentcenter.configuration;

import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.naming.NamingService;
import com.alibaba.nacos.api.naming.pojo.Instance;
import com.alibaba.nacos.client.naming.core.Balancer;
import com.netflix.client.config.IClientConfig;
import com.netflix.loadbalancer.AbstractLoadBalancerRule;
import com.netflix.loadbalancer.BaseLoadBalancer;
import com.netflix.loadbalancer.Server;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.alibaba.nacos.NacosDiscoveryProperties;
import org.springframework.cloud.alibaba.nacos.ribbon.NacosServer;
import org.springframework.util.CollectionUtils;

import java.util.List;
import java.util.Objects;
import java.util.stream.Collectors;

/**
 * Load balancing strategy based on random weights with priority calls to the same cluster
 *
 * @author 01
 * @date 2019-07-27
 **/
@Slf4j
public class NacosSameClusterWeightedRule extends AbstractLoadBalancerRule {

    @Autowired
    private NacosDiscoveryProperties discoveryProperties;

    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {
        // do nothing
    }

    @Override
    public Server choose(Object key) {
        // Gets the cluster name configured in the configuration file
        String clusterName = discoveryProperties.getClusterName();
        BaseLoadBalancer loadBalancer = (BaseLoadBalancer) this.getLoadBalancer();
        // Get the name of the microservice you need to request
        String serviceId = loadBalancer.getName();
        // Get API s for Service Discovery
        NamingService namingService = discoveryProperties.namingServiceInstance();

        try {
            // Get all health instances of this micro service
            List<Instance> instances = namingService.selectInstances(serviceId, true);
            // Filter out all instances under the same cluster
            List<Instance> sameClusterInstances = instances.stream()
                    .filter(i -> Objects.equals(i.getClusterName(), clusterName))
                    .collect(Collectors.toList());

            // If there are no instances in the same cluster, you need to use instances in other clusters
            List<Instance> instancesToBeChosen;
            if (CollectionUtils.isEmpty(sameClusterInstances)) {
                instancesToBeChosen = instances;
                log.warn("A cross-cluster call occurred, name = {}, clusterName = {}, instances = {}",
                        serviceId, clusterName, instances);
            } else {
                instancesToBeChosen = sameClusterInstances;
            }

            // Select an instance from the list of instances based on a random weight load balancing algorithm
            Instance instance = ExtendBalancer.getHost(instancesToBeChosen);
            log.info("Examples selected are: port = {}, instance = {}", instance.getPort(), instance);

            return new NacosServer(instance);
        } catch (NacosException e) {
            log.error("Get Instance Exception", e);
            return null;
        }
    }
}

class ExtendBalancer extends Balancer {

    /**
     * Because the getHostByRandomWeight method in the Balancer class is protected,
     * So it's called through this inheritance method, which picks up an example based on a random weight load balancing algorithm
     */
    static Instance getHost(List<Instance> hosts) {
        return getHostByRandomWeight(hosts);
    }
}

Similarly, if you want to use this load balancing policy, configure it in the configuration file:

user-center:
  ribbon:
    NFLoadBalancerRuleClassName: com.zj.node.contentcenter.configuration.NacosSameClusterWeightedRule

Metadata-based version control

In the above two subsections, we implement a Nacos weight-based load balancing strategy and a priority call under the same cluster, but in a real project, you may face the problem of multiversion coexistence, that is, a microservice has different versions of instances, and these different versions of instances may not be mutually exclusive.Tolerant.For example, the v1 version instance of MicroService A cannot invoke the v2 version instance of MicroService B, only the v1 version instance of MicroService B can be invoked.

Metadata in Nacos is a good solution to this version control problem, and the concept and configuration of metadata are already in place Service Discovery Component for Spring Cloud Alibaba - Nacos This article describes how to use Ribbon to implement metadata-based versioning.

For example, there are two microservices online, one as a service provider and one as a service consumer. They all have different versions of instances, as follows:

• There are two versions of service providers: v1, v2
• Service consumers also have two versions: v1, v2

V1 and V2 are incompatible.Service consumer V1 can only call service provider v1; consumer V2 can only call provider v2.How?Let's focus on this scenario to achieve versioning between microservices.

To sum up, there are two main points we need to achieve:

• Prefer metadata-compliant instances in the same cluster
• If there are no metadata-compliant instances under the same cluster, select metadata-compliant instances under other clusters

First we have to configure metadata in the configuration file, which is a bunch of descriptive information, configured as k-v, as follows:

spring:
  cloud:
    nacos:
      discovery:
        # Specify the address of the nacos server
        server-addr: 127.0.0.1:8848
        # Configuration Metadata
        metadata: 
          # Current Instance Version
          version: v1
          # Version of provider instance allowed to be invoked
          target-version: v1

Then you can write code, as before, through a load balancing strategy, as follows:

package com.zj.node.contentcenter.configuration;

import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.naming.NamingService;
import com.alibaba.nacos.api.naming.pojo.Instance;
import com.alibaba.nacos.client.naming.utils.CollectionUtils;
import com.alibaba.nacos.client.utils.StringUtils;
import com.netflix.client.config.IClientConfig;
import com.netflix.loadbalancer.AbstractLoadBalancerRule;
import com.netflix.loadbalancer.DynamicServerListLoadBalancer;
import com.netflix.loadbalancer.Server;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.alibaba.nacos.NacosDiscoveryProperties;
import org.springframework.cloud.alibaba.nacos.ribbon.NacosServer;

import java.util.Collections;
import java.util.List;
import java.util.Objects;
import java.util.function.Predicate;
import java.util.stream.Collectors;

/**
 * Metadata-based Version Control Load Balancing Strategy
 *
 * @author 01
 * @date 2019-07-27
 **/
@Slf4j
public class NacosFinalRule extends AbstractLoadBalancerRule {

    @Autowired
    private NacosDiscoveryProperties discoveryProperties;

    private static final String TARGET_VERSION = "target-version";
    private static final String VERSION = "version";

    @Override
    public void initWithNiwsConfig(IClientConfig iClientConfig) {
        // do nothing
    }

    @Override
    public Server choose(Object key) {
        // Gets the cluster name configured in the configuration file
        String clusterName = discoveryProperties.getClusterName();
        // Get the metadata configured in the configuration file
        String targetVersion = discoveryProperties.getMetadata().get(TARGET_VERSION);

        DynamicServerListLoadBalancer loadBalancer = (DynamicServerListLoadBalancer) getLoadBalancer();
        // Requested microservice name
        String serviceId = loadBalancer.getName();
        // Get all health instances of this micro service
        List<Instance> instances = getInstances(serviceId);

        List<Instance> metadataMatchInstances = instances;
        // If a version mapping is configured, it represents an instance where only metadata matches are invoked
        if (StringUtils.isNotBlank(targetVersion)) {
            // Filter instances that match version metadata for version control
            metadataMatchInstances = filter(instances,
                    i -> Objects.equals(targetVersion, i.getMetadata().get(VERSION)));

            if (CollectionUtils.isEmpty(metadataMatchInstances)) {
                log.warn("No target instance matching metadata was found!Please check the configuration. targetVersion = {}, instance = {}",
                        targetVersion, instances);
                return null;
            }
        }

        List<Instance> clusterMetadataMatchInstances = metadataMatchInstances;
        // If the cluster name is configured, you need to filter instances that match the metadata under the cluster
        if (StringUtils.isNotBlank(clusterName)) {
            // Filter out all instances under the same cluster
            clusterMetadataMatchInstances = filter(metadataMatchInstances,
                    i -> Objects.equals(clusterName, i.getClusterName()));

            if (CollectionUtils.isEmpty(clusterMetadataMatchInstances)) {
                clusterMetadataMatchInstances = metadataMatchInstances;
                log.warn("A cross-cluster call occurred. clusterName = {}, targetVersion = {}, clusterMetadataMatchInstances = {}", clusterName, targetVersion, clusterMetadataMatchInstances);
            }
        }

        // Select one of the load balancing algorithms based on random weights
        Instance instance = ExtendBalancer.getHost(clusterMetadataMatchInstances);

        return new NacosServer(instance);
    }

    /**
     * Filter the list of instances through filtering rules
     */
    private List<Instance> filter(List<Instance> instances, Predicate<Instance> predicate) {
        return instances.stream()
                .filter(predicate)
                .collect(Collectors.toList());
    }

    private List<Instance> getInstances(String serviceId) {
        // Get API s for Service Discovery
        NamingService namingService = discoveryProperties.namingServiceInstance();
        try {
            // Get all health instances of this micro service
            return namingService.selectInstances(serviceId, true);
        } catch (NacosException e) {
            log.error("exception occurred", e);
            return Collections.emptyList();
        }
    }
}

class ExtendBalancer extends Balancer {
    /**
     * Because the getHostByRandomWeight method in the Balancer class is protected,
     * So it's called through this inheritance method, which picks up an example based on a random weight load balancing algorithm
     */
    static Instance getHost(List<Instance> hosts) {
        return getHostByRandomWeight(hosts);
    }
}