Serial articles
Dubbo Analysis Serialize Layer
Transport Layer of Dubbo Analysis
Exchange Layer of Dubbo Analysis
Protocol Layer of Dubbo Analysis
Preface
Immediately preceding Protocol Layer of Dubbo Analysis This paper continues to analyze the cluster layer of dubbo, which encapsulates the routing and load balancing of multiple providers, bridges the registry, centers on Invoker, and extends the interface to Cluster, Directory, Router, LoadBalance.
Cluster interface
The entire cluster layer can be summarized with the following pictures:
Node relationships:
Here, Invoker is an abstraction of a callable Service of Provider, which encapsulates the Provider address and Service interface information.
Directory represents multiple Invoker s and can be considered a List, but unlike List, its values may be dynamic, such as registry push changes;
Cluster disguises many Invokers in Directory as an Invoker, which is transparent to the upper layer. The disguising process contains fault-tolerant logic. After the call fails, try another one again.
Router is responsible for selecting subsets from multiple Invoker s according to routing rules, such as read-write separation, application isolation, etc.
LoadBalance is responsible for selecting one specific Invoker for this call. The selection process includes load balancing algorithm. After the call fails, it needs to be re-selected.
Cluster gets a usable Invoker through directory, routing and load balancing and gives it to the upper layer for invocation. The interface is as follows:
@SPI(FailoverCluster.NAME) public interface Cluster { /** * Merge the directory invokers to a virtual invoker. * * @param <T> * @param directory * @return cluster invoker * @throws RpcException */ @Adaptive <T> Invoker<T> join(Directory<T> directory) throws RpcException; }
Cluster is a cluster fault-tolerant interface. Invoker acquired after routing and load balancing is handled by fault-tolerant mechanism. dubbo provides a variety of fault-tolerant mechanisms, including:
Failover Cluster: Failed automatic switching, when failure occurs, retry other servers [1]. Usually used for read operations, but retries cause longer latency. Retries="2" can be used to set the number of retries (excluding the first time).
Failfast Cluster: Quick failures, only one call is initiated, failures are reported immediately. Usually used for non-idempotent write operations, such as adding records.
Failsafe Cluster: Failsafe. When an exception occurs, it is ignored directly. Usually used for writing audit logs and other operations.
Failback Cluster: Failure automatic recovery, background record failed requests, and periodically resend. Usually used for message notification operations.
Forking Cluster: Call multiple servers in parallel and return as soon as one succeeds. Usually used for read operations with high real-time requirements, but more service resources need to be wasted. The maximum number of parallels can be set by forks ="2".
Broadcast Cluster: Broadcast calls all providers, one by one, and any one of them will report an error [2]. Usually used to notify all providers to update local resource information such as caches or logs.
Failover Cluster is used by default, and other servers will be retried by default when failing, default is twice; of course, other fault-tolerant mechanisms can also be extended; take a look at the default failure over Cluster fault-tolerant mechanism, the specific source code in Failover Cluster Invoker:
public Result doInvoke(Invocation invocation, final List<Invoker<T>> invokers, LoadBalance loadbalance) throws RpcException { List<Invoker<T>> copyinvokers = invokers; checkInvokers(copyinvokers, invocation); int len = getUrl().getMethodParameter(invocation.getMethodName(), Constants.RETRIES_KEY, Constants.DEFAULT_RETRIES) + 1; if (len <= 0) { len = 1; } // retry loop. RpcException le = null; // last exception. List<Invoker<T>> invoked = new ArrayList<Invoker<T>>(copyinvokers.size()); // invoked invokers. Set<String> providers = new HashSet<String>(len); for (int i = 0; i < len; i++) { //Reselect before retry to avoid a change of candidate `invokers`. //NOTE: if `invokers` changed, then `invoked` also lose accuracy. if (i > 0) { checkWhetherDestroyed(); copyinvokers = list(invocation); // check again checkInvokers(copyinvokers, invocation); } Invoker<T> invoker = select(loadbalance, invocation, copyinvokers, invoked); invoked.add(invoker); RpcContext.getContext().setInvokers((List) invoked); try { Result result = invoker.invoke(invocation); if (le != null && logger.isWarnEnabled()) { logger.warn("Although retry the method " + invocation.getMethodName() + " in the service " + getInterface().getName() + " was successful by the provider " + invoker.getUrl().getAddress() + ", but there have been failed providers " + providers + " (" + providers.size() + "/" + copyinvokers.size() + ") from the registry " + directory.getUrl().getAddress() + " on the consumer " + NetUtils.getLocalHost() + " using the dubbo version " + Version.getVersion() + ". Last error is: " + le.getMessage(), le); } return result; } catch (RpcException e) { if (e.isBiz()) { // biz exception. throw e; } le = e; } catch (Throwable e) { le = new RpcException(e.getMessage(), e); } finally { providers.add(invoker.getUrl().getAddress()); } } throw new RpcException(le != null ? le.getCode() : 0, "Failed to invoke the method " + invocation.getMethodName() + " in the service " + getInterface().getName() + ". Tried " + len + " times of the providers " + providers + " (" + providers.size() + "/" + copyinvokers.size() + ") from the registry " + directory.getUrl().getAddress() + " on the consumer " + NetUtils.getLocalHost() + " using the dubbo version " + Version.getVersion() + ". Last error is: " + (le != null ? le.getMessage() : ""), le != null && le.getCause() != null ? le.getCause() : le); }
Invocation is the relevant parameters passed to the server by the client, including (method name, method parameter, parameter value, attachment information). invokers is a list of servers after routing, and load balance is a specified load balancing strategy. First, check whether invokers are empty, throw an exception directly for empty, and then obtain the number of retries by default of 2, followed by the specified number of circular calls. If it is not the first call, the server list will be reloaded, and then the only Invoker will be obtained through load balancing strategy. Finally, the invocation will be sent to the server through Invoker, and the result Result will be returned.
The concrete doInvoke method is called in the abstract class AbstractClusterInvoker:
public Result invoke(final Invocation invocation) throws RpcException { checkWhetherDestroyed(); LoadBalance loadbalance = null; List<Invoker<T>> invokers = list(invocation); if (invokers != null && !invokers.isEmpty()) { loadbalance = ExtensionLoader.getExtensionLoader(LoadBalance.class).getExtension(invokers.get(0).getUrl() .getMethodParameter(RpcUtils.getMethodName(invocation), Constants.LOADBALANCE_KEY, Constants.DEFAULT_LOADBALANCE)); } RpcUtils.attachInvocationIdIfAsync(getUrl(), invocation); return doInvoke(invocation, invokers, loadbalance); } protected List<Invoker<T>> list(Invocation invocation) throws RpcException { List<Invoker<T>> invokers = directory.list(invocation); return invokers; }
Firstly, Invoker list is obtained through Directory, and routing is also done in Directory, then load balancing strategy is obtained, and finally specific fault-tolerant strategy is invoked.
Directory interface
The interface is defined as follows:
public interface Directory<T> extends Node { /** * get service type. * * @return service type. */ Class<T> getInterface(); /** * list invokers. * * @return invokers */ List<Invoker<T>> list(Invocation invocation) throws RpcException; }
The function of directory service is to get the list of services with specified interfaces. There are two specific implementations: Static Directory and Registry Directory, which both inherit from AbstractDirectory; Static Directory is a fixed directory service, which means that the list of Invoker in it will not change dynamically; Registry Directory is a dynamic directory service through registration. The list of services is dynamically updated; the list is implemented in abstract classes:
public List<Invoker<T>> list(Invocation invocation) throws RpcException { if (destroyed) { throw new RpcException("Directory already destroyed .url: " + getUrl()); } List<Invoker<T>> invokers = doList(invocation); List<Router> localRouters = this.routers; // local reference if (localRouters != null && !localRouters.isEmpty()) { for (Router router : localRouters) { try { if (router.getUrl() == null || router.getUrl().getParameter(Constants.RUNTIME_KEY, false)) { invokers = router.route(invokers, getConsumerUrl(), invocation); } } catch (Throwable t) { logger.error("Failed to execute router: " + getUrl() + ", cause: " + t.getMessage(), t); } } } return invokers; }
First check whether the directory is destroyed, then call doList, which is defined in the implementation class, and finally call the routing function. Next, we will focus on the doList method in Static Directory and Registry Directory.
1.RegistryDirectory
Registry Directory is a dynamic directory service. All can see that Registry Directory also inherits NotifyListener interface, which is a notification interface. When the registry updates the list of services, it also notifies Registry Directory. The notification logic is as follows:
public synchronized void notify(List<URL> urls) { List<URL> invokerUrls = new ArrayList<URL>(); List<URL> routerUrls = new ArrayList<URL>(); List<URL> configuratorUrls = new ArrayList<URL>(); for (URL url : urls) { String protocol = url.getProtocol(); String category = url.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY); if (Constants.ROUTERS_CATEGORY.equals(category) || Constants.ROUTE_PROTOCOL.equals(protocol)) { routerUrls.add(url); } else if (Constants.CONFIGURATORS_CATEGORY.equals(category) || Constants.OVERRIDE_PROTOCOL.equals(protocol)) { configuratorUrls.add(url); } else if (Constants.PROVIDERS_CATEGORY.equals(category)) { invokerUrls.add(url); } else { logger.warn("Unsupported category " + category + " in notified url: " + url + " from registry " + getUrl().getAddress() + " to consumer " + NetUtils.getLocalHost()); } } // configurators if (configuratorUrls != null && !configuratorUrls.isEmpty()) { this.configurators = toConfigurators(configuratorUrls); } // routers if (routerUrls != null && !routerUrls.isEmpty()) { List<Router> routers = toRouters(routerUrls); if (routers != null) { // null - do nothing setRouters(routers); } } List<Configurator> localConfigurators = this.configurators; // local reference // merge override parameters this.overrideDirectoryUrl = directoryUrl; if (localConfigurators != null && !localConfigurators.isEmpty()) { for (Configurator configurator : localConfigurators) { this.overrideDirectoryUrl = configurator.configure(overrideDirectoryUrl); } } // providers refreshInvoker(invokerUrls); }
This notification interface accepts three types of url s: router (routing), configurator (configuration), provider (service provider);
Routing rules: Decide the target server of a dubbo service call, which is divided into conditional routing rules and script routing rules, and support scalability. The operation of writing routing rules to the registry is usually completed by the page of the monitoring center or the governance center.
Configuration rule: Write dynamic configuration coverage rule [1] to the registry. This function is usually completed by the page of the monitoring center or the governance center.
provider: a list of dynamically provided services
Routing and configuration rules are actually updating and filtering provider service lists. The refreshInvoker method updates local invoker lists according to three url categories. Let's look at the doList interface implemented by Registry Directory:
public List<Invoker<T>> doList(Invocation invocation) { if (forbidden) { // 1. No service provider 2. Service providers are disabled throw new RpcException(RpcException.FORBIDDEN_EXCEPTION, "No provider available from registry " + getUrl().getAddress() + " for service " + getConsumerUrl().getServiceKey() + " on consumer " + NetUtils.getLocalHost() + " use dubbo version " + Version.getVersion() + ", please check status of providers(disabled, not registered or in blacklist)."); } List<Invoker<T>> invokers = null; Map<String, List<Invoker<T>>> localMethodInvokerMap = this.methodInvokerMap; // local reference if (localMethodInvokerMap != null && localMethodInvokerMap.size() > 0) { String methodName = RpcUtils.getMethodName(invocation); Object[] args = RpcUtils.getArguments(invocation); if (args != null && args.length > 0 && args[0] != null && (args[0] instanceof String || args[0].getClass().isEnum())) { invokers = localMethodInvokerMap.get(methodName + "." + args[0]); // The routing can be enumerated according to the first parameter } if (invokers == null) { invokers = localMethodInvokerMap.get(methodName); } if (invokers == null) { invokers = localMethodInvokerMap.get(Constants.ANY_VALUE); } if (invokers == null) { Iterator<List<Invoker<T>>> iterator = localMethodInvokerMap.values().iterator(); if (iterator.hasNext()) { invokers = iterator.next(); } } } return invokers == null ? new ArrayList<Invoker<T>>(0) : invokers; }
After refreshInvoker processing, the service list already exists in the method Invoker Map, and a method corresponds to the service list Map >.
The corresponding service list is obtained by the method specified in Invocation, and if the specific method does not have the corresponding service list, the corresponding service list of "*" is obtained. After processing, the routing process is carried out in the parent class, and the routing rules are also obtained through the notification interface. The routing rules are introduced in the next chapter.
2.StaticDirectory
This is a static directory service, in which the list of services already exists at the time of initialization and will not change; Static Directory is used less, mainly for the reference of services to multiple registries;
protected List<Invoker<T>> doList(Invocation invocation) throws RpcException { return invokers; }
Because it's static, all doList methods are simple, just return to the list of services in memory directly.
Router interface
Routing rules determine the target server of a dubbo service call, which is divided into conditional routing rules and script routing rules, and supports scalability. The interfaces are as follows:
public interface Router extends Comparable<Router> { /** * get the router url. * * @return url */ URL getUrl(); /** * route. * * @param invokers * @param url refer url * @param invocation * @return routed invokers * @throws RpcException */ <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException; }
The route method provided in the interface filters out a subset of invokers through certain rules; it provides three implementation classes: ScriptRouter, ConditionRouter and MockInvokersSelector.
Script Router: The script routing rules support all scripts of JDK script engine, such as: javascript, jruby, groovy, etc. The script type is set by type=javascript parameter, default is javascript;
ConditionRouter: Routing rules based on conditional expressions, such as: host = 10.20.153.10 => host = 10.20.153.11; => matching conditions for consumers before, all parameters are compared with consumers'URLs, => filtering conditions for the list of addresses of providers, all parameters are compared with providers' URLs;
MockInvokersSelector: Is it configured to use mock? This router guarantees that only callers with protocol MOCK will appear in the final list of callers, and all other callers will be excluded.
Let's focus on the ScriptRouter source code
public ScriptRouter(URL url) { this.url = url; String type = url.getParameter(Constants.TYPE_KEY); this.priority = url.getParameter(Constants.PRIORITY_KEY, 0); String rule = url.getParameterAndDecoded(Constants.RULE_KEY); if (type == null || type.length() == 0) { type = Constants.DEFAULT_SCRIPT_TYPE_KEY; } if (rule == null || rule.length() == 0) { throw new IllegalStateException(new IllegalStateException("route rule can not be empty. rule:" + rule)); } ScriptEngine engine = engines.get(type); if (engine == null) { engine = new ScriptEngineManager().getEngineByName(type); if (engine == null) { throw new IllegalStateException(new IllegalStateException("Unsupported route rule type: " + type + ", rule: " + rule)); } engines.put(type, engine); } this.engine = engine; this.rule = rule; }
The constructor initializes the script engine and the script code, respectively. The default script engine is javascript. Look at a specific url:
"script://0.0.0.0/com.foo.BarService?category=routers&dynamic=false&rule=" + URL.encode("(function route(invokers) { ... } (invokers))")
The script protocol represents a script protocol, followed by a javascript script with invokers as the input parameter.
(function route(invokers) { var result = new java.util.ArrayList(invokers.size()); for (i = 0; i < invokers.size(); i ++) { if ("10.20.153.10".equals(invokers.get(i).getUrl().getHost())) { result.add(invokers.get(i)); } } return result; } (invokers)); // Represents an immediate execution method
As the above script filters out the host of 10.20.153.10, how to execute the script is discussed in route method:
public <T> List<Invoker<T>> route(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException { try { List<Invoker<T>> invokersCopy = new ArrayList<Invoker<T>>(invokers); Compilable compilable = (Compilable) engine; Bindings bindings = engine.createBindings(); bindings.put("invokers", invokersCopy); bindings.put("invocation", invocation); bindings.put("context", RpcContext.getContext()); CompiledScript function = compilable.compile(rule); Object obj = function.eval(bindings); if (obj instanceof Invoker[]) { invokersCopy = Arrays.asList((Invoker<T>[]) obj); } else if (obj instanceof Object[]) { invokersCopy = new ArrayList<Invoker<T>>(); for (Object inv : (Object[]) obj) { invokersCopy.add((Invoker<T>) inv); } } else { invokersCopy = (List<Invoker<T>>) obj; } return invokersCopy; } catch (ScriptException e) { //fail then ignore rule .invokers. logger.error("route error , rule has been ignored. rule: " + rule + ", method:" + invocation.getMethodName() + ", url: " + RpcContext.getContext().getUrl(), e); return invokers; } }
First the script is compiled by the script engine, and then the script is executed. At the same time, the Bindings parameters are passed in, so that invokers can be obtained in the script, and then filtered. Finally, we look at the load balancing strategy.
LoadBalance interface
In cluster load balancing, Dubbo provides a variety of balancing strategies. By default, random random random calls can be used to expand load balancing strategies by itself. The interface classes are as follows:
@SPI(RandomLoadBalance.NAME) public interface LoadBalance { /** * select one invoker in list. * * @param invokers invokers. * @param url refer url * @param invocation invocation. * @return selected invoker. */ @Adaptive("loadbalance") <T> Invoker<T> select(List<Invoker<T>> invokers, URL url, Invocation invocation) throws RpcException; }
SPI defines the default policy as Random LoadBalance, which provides a select method to select an invoker from the list of services by policy; dubbo defaults to provide a variety of policies:
Random Load Balance: Random, set random probability according to weight, the probability of collision on a cross-section is high, but the larger the amount of calls, the more uniform the distribution, and the more uniform the weight is used according to probability, which is conducive to dynamically adjusting the weight of providers;
Round Robin Load Balance: Polling, setting the polling rate according to the weight of the convention; there is a problem of slow providers accumulating requests, such as: the second machine is slow, but it is not hanging, and when the request is transferred to the second machine, it is stuck there.
Over time, all requests were transferred to the second station.
Least Active Load Balance: The least number of active calls, the same number of random active, active count refers to the difference between before and after the call; make slow providers receive fewer requests, because the slower providers call, the greater the difference between before and after the call;
Consistent Hash LoadBalance: Consistent Hash, requests with the same parameters are always sent to the same provider; when a provider hangs up, requests originally sent to that provider are spread out to other providers based on virtual nodes without causing drastic changes;
Let's focus on the default AndomLoadBalance source code
protected <T> Invoker<T> doSelect(List<Invoker<T>> invokers, URL url, Invocation invocation) { int length = invokers.size(); // Number of invokers int totalWeight = 0; // The sum of weights boolean sameWeight = true; // Every invoker has the same weight? for (int i = 0; i < length; i++) { int weight = getWeight(invokers.get(i), invocation); totalWeight += weight; // Sum if (sameWeight && i > 0 && weight != getWeight(invokers.get(i - 1), invocation)) { sameWeight = false; } } if (totalWeight > 0 && !sameWeight) { // If (not every invoker has the same weight & at least one invoker's weight>0), select randomly based on totalWeight. int offset = random.nextInt(totalWeight); // Return a invoker based on the random value. for (int i = 0; i < length; i++) { offset -= getWeight(invokers.get(i), invocation); if (offset < 0) { return invokers.get(i); } } } // If all invokers have the same weight value or totalWeight=0, return evenly. return invokers.get(random.nextInt(length)); }
Firstly, the total weight is calculated, and whether each service has the same weight is checked. If the total weight is greater than 0 and the weight of the service is different, the weight is chosen randomly, otherwise the Random function is used directly.
summary
This paper introduces several important interfaces in Cluster layer from top to bottom, and focuses on some of the implementation classes. It is easy to understand this layer in combination with the official call graph.