Dubbo Extension Mechanism: Extension Loader

I. Preface

Dubbo's Extension Loader is an important component of "micro-kernel + plug-in" implementation, which is based on the java spi mechanism but provides the following extensions:

jdk spi only obtains all implementations by interface class name, while Extension Loader obtains an implementation by interface class name and key value.
Adaptive implementation is to generate a proxy class so that the class to be invoked can be dynamically determined based on some parameters of the actual invocation.
Automatic wrapping implementations, which are usually automatically activated, are often used for wrapping classes, such as ProtocolFilterWrapper, ProtocolListener Wrapper, and ProtocolFilterWrapper.

jdk spi has the following disadvantages:

Although Service Loader is also a kind of lazy loading, it can only be obtained by traversal, that is, the implementation classes of interfaces are loaded and instantiated once. If you don't want to use some implementation class, it's loaded and instantiated, which is wasteful.
The way to get an implementation class is not flexible enough. It can only be obtained by Iterator, but not by a parameter.

Dubbo framework is based on the model of URL bus, that is, all the state data information can be obtained through the URL in the running process, such as what serialization the current system uses, what communication is used, and what load balancing information is presented through the parameters of the URL, so in the running process of the framework, To run to a certain stage, the corresponding data can be obtained from the parameter list of the URL through the corresponding Key. For example, in the cluster module, when the service call is triggered to the module, the load balancing strategy of the current invoking service and the mock information will be obtained from the URL.

Extension Loader itself is a singleton factory class that exposes getExtension Loader static methods to return an Extension Loader entity. The method's entry is a Class type, which means returning an Extension Loader for an interface. For an interface, then, there will only be one Extension Loader entity. Extension Loader mainly provides the following methods to first extend the operation of the point:

static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type);
String getExtensionName(T extensionInstance);
String getExtensionName(Class<?> extensionClass);
List<T> getActivateExtension(URL url, String key);
List<T> getActivateExtension(URL url, String[] values);
List<T> getActivateExtension(URL url, String key, String group);
List<T> getActivateExtension(URL url, String[] values, String group);
boolean isMatchGroup(String group, String[] groups);
boolean isActive(Activate activate, URL url) ;
T getLoadedExtension(String name);
Set<String> getLoadedExtensions();
T getExtension(String name) ;
T getDefaultExtension();
boolean hasExtension(String name);
Set<String> getSupportedExtensions();
String getDefaultExtensionName();
void addExtension(String name, Class<?> clazz) ;
T getAdaptiveExtension() ;

Expansion Point Annotation

1,SPI

In addition to describing service implementation information under specified folders, the SPI specification of Dubbo, as an extension point, must annotate SPI annotations to tell Dubbo that the interface is extended through SPI. Otherwise, Extension Loader will not create Extension Loader entities for this interface, and Calling the Extension Loader. getExtension Loader method causes an IllegalArgumentException exception. When annotating SPI annotations on an interface, you can configure a value attribute to describe the default implementation alias of the interface. For example, @SPI("netty") of Transporter specifies that the default implementation of Transporter is Netty Transporter, because the alias of Netty Transporter is netty. The service aliases are added here. Aliases stand in the dimension of an interface to distinguish different implementations, so the implementation of an interface can not have the same alias, otherwise the Dubbo framework will fail to start. Of course, the implementation aliases of different interfaces can be the same. Now that the principles and basic principles of Extension Loader are introduced, let's see how to implement our own plug-in based on Dubbo's Extension Loader. It was also demonstrated in the dubbo-demo project, where a demo-extension module was created. _

There are many extension points in Dubbo, and these extension point interfaces have the following:

CacheFactory
Compiler
ExtensionFactory
LoggerAdapter
Serialization
StatusChecker
DataStore
ThreadPool
Container
PageHandler
MonitorFactory
RegistryFactory
ChannelHandler
Codec
Codec2
Dispatcher
Transporter
Exchanger
HttpBinder
Networker
TelnetHandler
ZookeeperTransporter
ExporterListener
Filter
InvokerListener
Protocol
ProxyFactory
Cluster
ConfiguratorFactory
LoadBalance
Merger
RouterFactory
RuleConverter
ClassNameGenerator
Validation

Look at the definition of the SPI interface:

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE})
public @interface SPI {

    /**
     * Default extension roll-call.
     */
	String value() default "";

}

For example, Transporter extension points:

@SPI("netty")
public interface Transporter {

    /**
     * Bind a server.
     * 
     * @see com.alibaba.dubbo.remoting.Transporters#bind(URL, Receiver, ChannelHandler)
     * @param url server url
     * @param handler
     * @return server
     * @throws RemotingException
     *
     * As a Server, notice that after bind, a Server is returned, and the parameters are the url to listen on and its corresponding dubbo handler
     */
    @Adaptive({Constants.SERVER_KEY, Constants.TRANSPORTER_KEY})
    Server bind(URL url, ChannelHandler handler) throws RemotingException;

    /**
     * Connect to a server.
     * 
     * @see com.alibaba.dubbo.remoting.Transporters#connect(URL, Receiver, ChannelListener)
     * @param url server url
     * @param handler
     * @return client
     * @throws RemotingException
     *
     * As a client, notice that the connection returns a Client with parameters of the url to be connected and the corresponding dubbo hanlder
     */
    @Adaptive({Constants.CLIENT_KEY, Constants.TRANSPORTER_KEY})
    Client connect(URL url, ChannelHandler handler) throws RemotingException;

}

2,Adaptive

A new annotation, Adaptive, means adaptation. Extension Loader dynamically generates adaptive classes by analyzing the adaptive rules of interface configuration and loads them into ClassLoader to achieve dynamic adaptation. The rules for configuring adaptive are also set through the Adaptive annotation, which has a value attribute. By setting this attribute, you can set the rules for adaptive of the interface. As mentioned above, all data for service invocation can be obtained from the URL (Dubbo's URL bus mode), so we need Dubbo to dynamically generate adaptive for us. The method input of ive's extended interface must include the URL, so that the specific implementation can be dynamically selected according to the running state.

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface Adaptive {
    
    /**
     * As the Key name of the URL, the corresponding Value is the Extension name to be adapted.
     * If none of these keys in the URL has a Value, the default extension (the value set in the SPI annotation of the interface) is used.
     * For example, String []{key1,""key2"} denotes
     * First, find the Value of key1 on the URL as the Extension name to be adapted.
     * key1 Without Value, the Value of key2 is used as the Extension name to be adapted.
     * key2 Without Value, use the default extension (specified in SPI).
     * If no default extension is set, the method call throws IllegalStateException.
     * If not set, the point-separated lowercase string with the Extension interface class name is used by default.
     * That is, for the Extension interface com. alibaba. dubbo. xxx. YyInvoker Wrapper, the default value is
     * String[] {"yyy.invoker.wrapper"}
     * 
     * @see SPI#value()
     */
    String[] value() default {};
    
}

For example, the Transporter interface provides two methods, one is connect (used to create client connections) and the other is bind (used to bind server ports to provide services), both of which are configured with VA via the Adaptive annotation. Lue attribute, bind configuration is server and transporter, connect configuration is client and transporter. So what's the use of configuring these value s? Let's look at what kind of adaptive code Extension Loader generates from these.

public class Transporter$Adpative implements com.alibaba.dubbo.remoting.Transporter {
        public com.alibaba.dubbo.remoting.Client connect(com.alibaba.dubbo.common.URL arg0, com.alibaba.dubbo.remoting.ChannelHandler arg1) throws com.alibaba.dubbo.remoting.RemotingException {
            if (arg0 == null) throw new IllegalArgumentException("url == null");
            com.alibaba.dubbo.common.URL url = arg0;
            String extName = url.getParameter("client", url.getParameter("transporter", "netty"));
            if (extName == null)
                throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.remoting.Transporter) name from url(" + url.toString() + ") use keys([client, transporter])");
            com.alibaba.dubbo.remoting.Transporter extension = (com.alibaba.dubbo.remoting.Transporter) ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.remoting.Transporter.class).getExtension(extName);
            return extension.connect(arg0, arg1);
        }

        public com.alibaba.dubbo.remoting.Server bind(com.alibaba.dubbo.common.URL arg0, com.alibaba.dubbo.remoting.ChannelHandler arg1) throws com.alibaba.dubbo.remoting.RemotingException {
            if (arg0 == null) throw new IllegalArgumentException("url == null");
            com.alibaba.dubbo.common.URL url = arg0;
            String extName = url.getParameter("server", url.getParameter("transporter", "netty"));
            if (extName == null)
                throw new IllegalStateException("Fail to get extension(com.alibaba.dubbo.remoting.Transporter) name from url(" + url.toString() + ") use keys([server, transporter])");
            com.alibaba.dubbo.remoting.Transporter extension = (com.alibaba.dubbo.remoting.Transporter) ExtensionLoader.getExtensionLoader(com.alibaba.dubbo.remoting.Transporter.class).getExtension(extName);
            return extension.bind(arg0, arg1);
        }
    }

You can see that the bind method first makes a non-null judgment on the URL parameter (arg0), then calls the url.getParameter method, first obtains the server parameter, then obtains the transporter parameter if it is not successful, and finally if both parameters are not available, extName is netty. After obtaining the parameters, we make a non-null judgment on extName. Then we get the Extension Loader of Transporter, and get the Transporter implementation with the alias extName, and call the corresponding bind to bind the service port operation. Connect is similar, except that it first obtains client parameters from url, and then obtains transporter parameters. Similarly, if the last two parameters are not available, extName is netty. It also obtains the implementation of the interface extension based on extName and calls the connect method.

It should be noted that methods that are not annotated with Adaptive cannot be invoked, otherwise an Unsupported OperationException exception will be thrown. At the same time, Adaptive can also be typed on the implementation class of an interface, which is usually used in the method parameters of an interface without a URL (so there is no way to dynamically route to the specific implementation class), so it is necessary to implement an Adaptive implementation class of an extension point interface manually at this time, such as the top of an extension point interface:

@SPI("default")
public interface MyFirstExtension {
    public String sayHello(String name,ExtensionType type);
}

The next step is to provide different implementations for this plug-in interface. There is no URL type in the input of the above interface method, sayHello method, so we can not dynamically generate adaptive class through Dubbo, so we need to implement an adaptive class by ourselves. The adapter classes are as follows:

@Adaptive
public class AdaptiveExtension implements MyFirstExtension {
    @Override
    public String sayHello(String name,ExtensionType type) {
        ExtensionLoader extensionLoader = ExtensionLoader.getExtensionLoader(MyFirst Extension.class);
        MyFirstExtension extension= (MyFirstExtension) extensionLoader.getDefaultExtension();
        switch (type){
            case DEFAULT:
                extension= (MyFirstExtension) extensionLoader.getExtension("default");
                break;
            case OTHER:
                extension= (MyFirstExtension) extensionLoader.getExtension("other");
                break;
        }
        return extension.sayHello(name,type);
    }
}

In Adaptive Extension, you can see that the specific implementation of the extension will be distributed according to Extension Type and its sayHello method will be called.

Note: Only one Adaptive adaptation implementation may exist for an interface.

3,Activate

Activate annotations are mainly used to annotate the plug-in interface implementation class and configure the activation conditions of the extended implementation class. Let's first look at the Activate annotation definition:

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface Activate {
    /**
     * Group Filtration conditions.
     * <br />
     * The value given by the group parameter containing {@link Extension Loader#getActivateExtension} returns the extension.
     * <br />
     * If there is no Group settings, no filtering is done.
     */
    String[] group() default {};

    /**
     * Key Filtration conditions. The parameter Key that contains the URL {@link Extension Loader# getActivateExtension} returns the extension.
     * <p />
     * Example: <br/>
     * The value of the comment < code >@Activate ("cache, validatioin")</code>,
     * Then the parameters of the URL of {@link Extension Loader#getActivateExtension} are <code>cache</code>Key, or <code>validatioin</code> and the extension is returned.
     * <br/>
     * If no settings are set, no filtering is performed.
     */
    String[] value() default {};

    /**
     * Sort information can not be provided.
     */
    String[] before() default {};

    /**
     * Sort information can not be provided.
     */
    String[] after() default {};

    /**
     * Sort information can not be provided.
     */
    int order() default 0;
}

Various implementation classes of Filter in the Dubbo framework are labeled with Activate to describe when the Filter will take effect. MonitorFilter, for example, is used to tell the Dubbo framework that the filter will take effect at both the service provider and the consumer through Activate annotations, while Timeout Filter will only take effect at the service provider, and the consumer will not call the filter. Validation Filter is configured with value in addition to activation on the consumer side and the service provider side. This expression describes another activation condition. The value of this value configuration describes that the URL must have a specified parameter to activate the extension. . For example, Validation Filter means that the URL must contain the parameter validation (the value of Constants. VALIDATION_KEY constant is validation), otherwise even the consumer and server will not activate the extended implementation. Careful students will find that there is also a parameter order in the Activate annotation in Validation Filter, which is a representation of the parameter order. A sort rule. Because there are many implementations of an interface, the result returned is a list. If no sorting rules are specified, the sorting of the list may be uncontrollable. In order to achieve this, the order attribute is added to control the sorting. The larger the value of the order, the more advanced the extended implementation of sorting. In addition to ordering to control sorting, there are before and after to configure the location of the current extension. The values of before and after configurations are the aliases of the extension (the aliases of the extension implementation are on the left side of the middle number in Figure 23, and the aliases appearing below are all this).

@SPI
public interface Filter {
	Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException;
}

@Activate(group = {Constants.PROVIDER, Constants.CONSUMER})
public class MonitorFilter implements Filter {......}
@Activate(group = Constants.PROVIDER)
public class TimeoutFilter implements Filter {......}
@Activate(group = { Constants.CONSUMER, Constants.PROVIDER }, value = Constants.VALIDATION_KEY, order = 10000)
public class ValidationFilter implements Filter {......}

Active is almost introduced above. The most used implementations of Filter in Dubbo framework are various implementations of Filter, because Dubbo calls go through a filter chain, which filters are controlled by Activate annotations of various filter implementation classes. Including the sort mentioned above, it can also be understood as the order of each filter in the filter chain. One thing to note about the order here is that the framework itself implements the sort of extensions, and user-defined extensions are appended to the list by default. Here are some specific examples:

<dubbo:reference id="fooRef" interface="com.foo.Foo" ..... filter="A,B,C"/>

Assuming that the above is a valid consumer service reference with a filter attribute configured and three filters A,B,C (A,B,C are aliases implemented by Filter) separated by commas, what about the filter chain for interface Foo calls? First, Dubbo loads default filters (there are three ConsumerContext Filters, Monitor Filters, FutureFilters on the consumer side) and sorts these default filter implementations (Activate Comparator implements sort logic). This means that the default filter implementations will be in front of the filter chain, followed by A, B. C three custom filters.

3. Summary of usage methods

Each defined SPI interface builds an Extension Loader instance. Extension Loader uses factory mode to provide external Extension Loader instances in a static way. Instances are stored in static immutable Map s inside Extension Loader.
Called when used externally:
Extension Loader. getExtension Loader (Protocol. class). getAdaptive Extension (); getExtension Loader method creates Extension Loader instance, getAdaptive Extension method loads implementation classes in extension points and creates or selects adapters.
- Read the value value value of the SPI annotation, and if the value is not empty, name it as the default extension
- Read the extension points in the specified path in turn
  META-INF/dubbo/internal/
  META-INF/dubbo/
  META-INF/dubbo/services/
The getAdaptiveExtension method finally calls the loadFile method to read and parse the contents of the SPI file line by line
- Whether the implementation class contains @Adaptive annotation, if it does, caches it as an adapter to the cachedAdaptiveClass and goes to the next line of configuration for analysis. A SPI can only have one adapter, otherwise it will make an error.
- If there is no @Adaptive annotation on the implementation class, see if there is a constructor that takes the current access interface type as its parameter, and if so, store it as a wrapper in the cachedWrapperClasses variable.
- If the implementation class has neither an @Adaptive annotation nor a wrapper, it is a concrete implementation of the extension point.
- Determine if there is an @Activate annotation on the extension implementation, if there is one, cache it into cached Activates (a variable of type Map < String, Activate >), and then put its key as the name of the extension point into cached Classes (a variable of type Holder < Map < String, Class <?>), dubbo supports n:1 in the configuration file. Configuration, that is, protocols with different names are implemented using the same SPI, as long as the configuration name is named after regular s*[,]+ s*.
After parsing the file, the getAdaptiveExtension method starts to create an adapter instance. If the cachedAdaptiveClass has been determined in the parse file, the object is instantiated; if not, the adapter class bytecode is created.

Dubbo is able to generate adapter bytecodes for SPI s without adapters:

At least one method in the interface method must be annotated with @Adaptive
Method parameters annotated with @Adaptive must have URL type parameters or getURL() methods in parameters

After Dubbo generates code, it is necessary to compile the code. It is noted that all services in Dubbo are SPI, and the acquisition of compiler still needs Extension Loader to load. The default compiler of Dubbo is javassist. When Dubbo loads Compiler, one of Compiler's implementation classes has an @Adaptive annotation in the @Adaptive Compiler, that is, there are already implemented adapters. Dubbo does not need to generate bytecodes for Compiler, otherwise it will die.

After getting the adapter Class, Dubbo instantiates the adapter and implements a variable injection similar to IOC function. IOC code is very simple, take out the set method of public in the class, and get the corresponding attribute value from the objectFactory for setting.