Before going into Dubbo, you must understand the SPI mechanism in Dubbo.

Dubbo's extension point loading is enhanced from the SPI (Service Provider Interface) extension point discovery mechanism of JDK standard. However, it is different. It improves the following problems of SPI of JDK standard:

• The SPI of the JDK standard instantiates all implementations of the extension point at one time. If there is an extension implementation, initialization is time-consuming, but if it is not used, it will also be loaded, which will be a waste of resources.
• If the extension point fails to load, you can't even get the name of the extension point. For example, the JDK standard ScriptEngine obtains the name of the script type through getName(). However, if the ruby ScriptEngine fails to load the ruby ScriptEngine class because the jruby.jar it depends on does not exist, the failure reason is eaten and does not correspond to ruby. When the user executes the ruby script, it will report that it does not support ruby, rather than the real failure reason.
• Added support for IoC and AOP extension points. One extension point can directly inject setter into other extension points.

In the jar package of the extension class, place the fully qualified name of the extension point configuration file META-INF/dubbo / interface, which is: configuration name = fully qualified name of the extension implementation class, and multiple implementation classes are separated by newline characters.

The configuration files required by Dubbo SPI need to be placed in the META-INF/dubbo path, and almost all functions have extension points.

Take the Protocol interface as an example, which has many implementations.

 

2, Dubbo SPI

We can see from the above figure that the configuration of Dubbo SPI and JDK SPI are different. In Dubbo SPI, you can configure it by key value pairs, so that you can load the specified implementation classes as needed. The relevant logic of Dubbo SPI is encapsulated into the ExtensionLoader class. Through ExtensionLoader, we can load the specified implementation class. An extension interface corresponds to an ExtensionLoader object. Here, we affectionately call it extension point loader.

Let's first look at its properties:

public class ExtensionLoader<T> {
    
    //The path of the extension point configuration file, which can be loaded into the extension point configuration file from three places
    private static final String SERVICES_DIRECTORY = "META-INF/services/";
    private static final String DUBBO_DIRECTORY = "META-INF/dubbo/";
    private static final String DUBBO_INTERNAL_DIRECTORY = DUBBO_DIRECTORY + "internal/";   
    //Collection of extension point loaders
    private static final ConcurrentMap<Class<?>, ExtensionLoader<?>> EXTENSION_LOADERS = new ConcurrentHashMap<Class<?>, ExtensionLoader<?>>();
    //Collection of extension point implementations
    private static final ConcurrentMap<Class<?>, Object> EXTENSION_INSTANCES = new ConcurrentHashMap<Class<?>, Object>();
    //Mapping cache for extension point name and Implementation
    private final ConcurrentMap<Class<?>, String> cachedNames = new ConcurrentHashMap<Class<?>, String>();
    //Extension point implementation class collection cache
    private final Holder<Map<String, Class<?>>> cachedClasses = new Holder<Map<String, Class<?>>>();
    //Extension point name and@Activate Mapping cache for
    private final Map<String, Activate> cachedActivates = new ConcurrentHashMap<String, Activate>();
    //Cache implemented by extension point
    private final ConcurrentMap<String, Holder<Object>> cachedInstances = new ConcurrentHashMap<String, Holder<Object>>();
}

ExtensionLoader caches different extension point configurations and implementations. At the same time, Dubbo also reminded us on the official website that the extension point uses single instance loading (please ensure the thread safety of the extension Implementation) and is cached in ExtensionLoader. Let's look at some key methods.

We first obtain an instance of ExtensionLoader through the ExtensionLoader.getExtensionLoader() method, which is the extension point loader. Then get the extension class object through the getExtension method of ExtensionLoader. The getExtensionLoader method is used to get the ExtensionLoader corresponding to the extension class from the cache. If the cache misses, a new instance will be created.

public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
    if (type == null)
        throw new IllegalArgumentException("Extension type == null");
    if (!type.isInterface()) {
        throw new IllegalArgumentException("Extension type(" + type + ") is not interface!");
    }
    if (!withExtensionAnnotation(type)) {
        throw new IllegalArgumentException("Extension type(" + type +
                ") is not extension, because WITHOUT @" + SPI.class.getSimpleName() + " Annotation!");
    }
    ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
    if (loader == null) {
        EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type));
        loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
    }
    return loader;
}

For example, you can obtain the ExtensionLoader instance of the Protocol interface through the following:

ExtensionLoader extensionLoader = ExtensionLoader.getExtensionLoader(Protocol.class);

You can get the object instance of the extension point loader:

com.alibaba.dubbo.common.extension.ExtensionLoader[com.alibaba.dubbo.rpc.Protocol]

2. Get extension class object

In the previous step, we have obtained the loader, and then we can obtain the extension class object through the name of the extension point according to the loader instance.

public T getExtension(String name) {
    //Verify the validity of extension point name
    if (name == null || name.length() == 0)
        throw new IllegalArgumentException("Extension name == null");
    // Get the default extension implementation class
    if ("true".equals(name)) {
        return getDefaultExtension();
    }
    //Used to hold the target object
    Holder<Object> holder = cachedInstances.get(name);
    if (holder == null) {
        cachedInstances.putIfAbsent(name, new Holder<Object>());
        holder = cachedInstances.get(name);
    }
    Object instance = holder.get();
    if (instance == null) {
        synchronized (holder) {
            instance = holder.get();
            if (instance == null) {
                instance = createExtension(name);
                holder.set(instance);
            }
        }
    }
    return (T) instance;
}

 

It first attempts to get from the cache and creates an extension object if it misses. So what is its creation process?

private T createExtension(String name) {
    //Get all extension classes from the configuration file, Map data structure
    //Then get the corresponding extension class according to the name
    Class<?> clazz = getExtensionClasses().get(name);
    if (clazz == null) {
        throw findException(name);
    }
    try {
        //Create an instance through reflection and put it into the cache
        T instance = (T) EXTENSION_INSTANCES.get(clazz);
        if (instance == null) {
            EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance());
            instance = (T) EXTENSION_INSTANCES.get(clazz);
        }
        //Injection dependency
        injectExtension(instance);
        Set<Class<?>> wrapperClasses = cachedWrapperClasses;
        if (wrapperClasses != null && !wrapperClasses.isEmpty()) {
            // Package as Wrapper example
            for (Class<?> wrapperClass : wrapperClasses) {
                instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance));
            }
        }
        return instance;
    } catch (Throwable t) {
        throw new IllegalStateException("Extension instance(name: " + name + ", class: " +
                type + ")  could not be instantiated: " + t.getMessage(), t);
    }
}

 

There are two key points here, dependency injection and Wrapper wrapper classes, which are the concrete implementations of IOC and AOP in Dubbo.

Inject dependencies into the extended object, which will get all the methods of the class. Judge whether the method starts with set, and if the method has only one parameter and the method access level is public, set the attribute value through reflection. Therefore, IOC in Dubbo only supports setter injection.

private T injectExtension(T instance) {
    try {
        if (objectFactory != null) {
            for (Method method : instance.getClass().getMethods()) {
                if (method.getName().startsWith("set")
                        && method.getParameterTypes().length == 1
                        && Modifier.isPublic(method.getModifiers())) {
                    Class<?> pt = method.getParameterTypes()[0];
                    try {
                        String property = method.getName().length() > 3 ? method.getName().substring(3, 4).toLowerCase() + method.getName().substring(4) : "";
                        Object object = objectFactory.getExtension(pt, property);
                        if (object != null) {
                            method.invoke(instance, object);
                        }
                    } catch (Exception e) {
                        logger.error("fail to inject via method " + method.getName()
                                + " of interface " + type.getName() + ": " + e.getMessage(), e);
                    }
                }
            }
        }
    } catch (Exception e) {
        logger.error(e.getMessage(), e);
    }
    return instance;
}

It will pass the current instance as a parameter to the constructor of the Wrapper, and create the Wrapper instance through reflection. Then inject dependencies into the Wrapper instance, and finally assign the Wrapper instance to the instance variable again. It may be windy. Let's just look at the last generated object. Take DubboProtocol as an example. The packaged objects are:

  To sum up, if we get an extension class object, the last thing we get is an instance of the Wrapper class. Like this:

ExtensionLoader<Protocol> extensionLoader = ExtensionLoader.getExtensionLoader(Protocol.class);
Protocol extension = extensionLoader.getExtension("dubbo");
System.out.println(extension);

The output is: com.alibaba.dubbo.rpc.protocol ProtocolListenerWrapper@4cdf35a9

3. Get all extension classes

Before obtaining extension class objects by name, first resolve all extension classes according to the configuration file. It is a mapping table of extension point name and extension class map < string, class <= "">>. First, get cachedClasses from the cache. If not, call loadExtensionClasses to load from the configuration file. The configuration file has three paths:

• META-INF/services/
• META-INF/dubbo/
• META-INF/dubbo/internal/

Try getting from the cache first.

private Map<String, Class<?>> getExtensionClasses() {
    //Get from cache
    Map<String, Class<?>> classes = cachedClasses.get();
    if (classes == null) {
        synchronized (cachedClasses) {
            classes = cachedClasses.get();
            if (classes == null) {
                //Load extension class
                classes = loadExtensionClasses();
                cachedClasses.set(classes);
            }
        }
    }
    return classes;
} 

 

If not, call loadExtensionClasses to read from the configuration file.

private Map<String, Class<?>> loadExtensionClasses() {
    //obtain SPI Annotation, here type Variable is called getExtensionLoader Method
    final SPI defaultAnnotation = type.getAnnotation(SPI.class);
    if (defaultAnnotation != null) {
        String value = defaultAnnotation.value();
        if ((value = value.trim()).length() > 0) {
            String[] names = NAME_SEPARATOR.split(value);
            if (names.length > 1) {
                throw new IllegalStateException("more than 1 default extension 
                    name on extension " + type.getName()+ ": " + Arrays.toString(names));
            }
            //Set the default extension name, refer to getDefaultExtension method
            //If the name is true，Is to call the default extension class
            if (names.length == 1) cachedDefaultName = names[0];
        }
    }
    //Loads the configuration file for the specified path
    Map<String, Class<?>> extensionClasses = new HashMap<String, Class<?>>();
    loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY);
    loadDirectory(extensionClasses, DUBBO_DIRECTORY);
    loadDirectory(extensionClasses, SERVICES_DIRECTORY);
    return extensionClasses;
}

Taking the Protocol interface as an example, the obtained implementation class collection is as follows, and we can load specific extension class objects according to the name.

{
    registry=class com.alibaba.dubbo.registry.integration.RegistryProtocol
    injvm=class com.alibaba.dubbo.rpc.protocol.injvm.InjvmProtocol
    thrift=class com.alibaba.dubbo.rpc.protocol.thrift.ThriftProtocol
    mock=class com.alibaba.dubbo.rpc.support.MockProtocol
    dubbo=class com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol
    http=class com.alibaba.dubbo.rpc.protocol.http.HttpProtocol
    redis=class com.alibaba.dubbo.rpc.protocol.redis.RedisProtocol
    rmi=class com.alibaba.dubbo.rpc.protocol.rmi.RmiProtocol
}

In Dubbo, many extensions are loaded through SPI mechanism, such as Protocol, Cluster, LoadBalance, etc. These extensions are not loaded at the framework startup stage, but are loaded according to the URL object parameters when the extension method is called. Well, Dubbo solves this problem through adaptive extension mechanism.

The implementation logic of the adaptive expansion mechanism is as follows: firstly, Dubbo will generate proxy code for the expansion interface. Then compile this code through javassist or jdk to get the Class class. Finally, create a proxy Class through reflection. In the proxy Class, you can determine which implementation Class to call through the parameters of the URL object.

Before we begin, we need to take a look at an annotation closely related to Adaptive expansion, namely Adaptive annotation.

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.TYPE, ElementType.METHOD})
public @interface Adaptive {
    String[] value() default {};
}

 

As can be seen from the above code, Adaptive can be annotated on classes or methods.

• Label on class

Dubbo does not generate proxy classes for this class.

• Label on method

Dubbo will generate proxy logic for the method, indicating that the current method needs to call the corresponding extension point implementation according to the parameter URL.

getAdaptiveExtension method is the entry method to obtain adaptive extension.

public T getAdaptiveExtension() {
    // Get adaptive extension from cache
    Object instance = cachedAdaptiveInstance.get();
    if (instance == null) {
        if (createAdaptiveInstanceError == null) {
            synchronized (cachedAdaptiveInstance) {
                instance = cachedAdaptiveInstance.get();
                //If the cache is missed, an adaptive expansion is created and put into the cache
                if (instance == null) {
                    try {
                        instance = createAdaptiveExtension();
                        cachedAdaptiveInstance.set(instance);
                    } catch (Throwable t) {
                        createAdaptiveInstanceError = t;
                        throw new IllegalStateException("fail to create 
                                                  adaptive instance: " + t.toString(), t);
                    }
                }
            }
        }
    }
    return (T) instance;
} 

The getAdaptiveExtension method first checks the cache. If the cache misses, it calls the createAdaptiveExtension method to create an adaptive extension.

private T createAdaptiveExtension() {
    try {
        return injectExtension((T) getAdaptiveExtensionClass().newInstance());
    } catch (Exception e) {
        throw new IllegalStateException("
            Can not create adaptive extension " + type + ", cause: " + e.getMessage(), e);
    }
}

There is less code here. The getAdaptiveExtensionClass method is called to get the Class object adaptively, and then the reflection is instantiated, and finally the injectExtension method is used to inject dependency into the extended instance. The process of obtaining the adaptive extension class is as follows:

private Class<?> getAdaptiveExtensionClass() {
    //Gets all implementation classes of the current interface
    //If an implementation class is labeled@Adaptive，here cachedAdaptiveClass Not empty
    getExtensionClasses();
    if (cachedAdaptiveClass != null) {
        return cachedAdaptiveClass;
    }
    //If the above conditions are not true, create an adaptive extension class
    return cachedAdaptiveClass = createAdaptiveExtensionClass();
}

 

 

In the above method, it will first obtain all the implementation classes of the current interface. If an implementation class is marked with @ Adaptive, then the class will be assigned to the cacheddadaptiveclass variable and returned. If not, call createadaptive extensionclass to create an Adaptive extension class.

The createadaptive extensionclass method is used to generate the adaptive extension Class. This method first generates the source code of the adaptive extension Class, and then compiles the source code through the Compiler instance (Dubbo uses javassist as the Compiler by default) to obtain the proxy Class instance.

private Class<?> createAdaptiveExtensionClass() {
    //Build adaptive extension code
    String code = createAdaptiveExtensionClassCode();
    ClassLoader classLoader = findClassLoader();
    // Get compiler implementation class Dubbo The default is to use javassist 
    Compiler compiler =ExtensionLoader.getExtensionLoader(Compiler.class).getAdaptiveExtension();
    //Compile the code and return the object of the class instance
    return compiler.compile(code, classLoader);
} 

Before generating an Adaptive extension class, Dubbo checks whether the interface method contains @ Adaptive. If there is no such annotation on the method, an exception will be thrown.

if (!hasAdaptiveAnnotation){
    throw new IllegalStateException(
        "No adaptive method on extension " + type.getName() + ", 
          refuse to create the adaptive class!");
}

 

Let's take the Protocol interface as an example. Its export() and refer() methods are labeled @ Adaptive. destroy and getDefaultPort are not annotated with @ Adaptive annotation. Dubbo will not generate proxy logic for methods without Adaptive annotation. For this type of method, it will only generate an exception throwing code.

package com.alibaba.dubbo.rpc;
import com.alibaba.dubbo.common.URL;
import com.alibaba.dubbo.common.extension.Adaptive;
import com.alibaba.dubbo.common.extension.SPI;

@SPI("dubbo")
public interface Protocol {
    int getDefaultPort();
    @Adaptive
    <T> Exporter<T> export(Invoker<T> invoker) throws RpcException;
    @Adaptive
    <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException;
    void destroy();
} 

Therefore, when we call these two methods, we will first get the protocol name in the URL object, find the specific extension point implementation class according to the name, and then call it. The following is the source code for generating an adaptive extension class instance:

package com.viewscenes.netsupervisor.adaptive;

import com.alibaba.dubbo.common.URL;
import com.alibaba.dubbo.common.extension.ExtensionLoader;
import com.alibaba.dubbo.rpc.Exporter;
import com.alibaba.dubbo.rpc.Invoker;
import com.alibaba.dubbo.rpc.Protocol;
import com.alibaba.dubbo.rpc.RpcException;

public class Protocol$Adaptive implements Protocol {
    public void destroy() {
        throw new UnsupportedOperationException(
                "method public abstract void Protocol.destroy() of interface Protocol is not adaptive method!");
    }
    public int getDefaultPort() {
        throw new UnsupportedOperationException(
                "method public abstract int Protocol.getDefaultPort() of interface Protocol is not adaptive method!");
    }
    public Exporter export(Invoker invoker)throws RpcException {
        if (invoker == null) {
            throw new IllegalArgumentException("Invoker argument == null");
        }
        if (invoker.getUrl() == null) {
            throw new IllegalArgumentException("Invoker argument getUrl() == null");
        }
            
        URL url = invoker.getUrl();
        String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());
        if (extName == null) {
            throw new IllegalStateException("Fail to get extension(Protocol) name from url("
                    + url.toString() + ") use keys([protocol])");
        }
            
        Protocol extension = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(extName);
        return extension.export(invoker);
    }
    public Invoker refer(Class clazz,URL ur)throws RpcException {
        if (ur == null) {
            throw new IllegalArgumentException("url == null");
        }
        URL url = ur;
        String extName = (url.getProtocol() == null ? "dubbo" : url.getProtocol());
        if (extName == null) {
            throw new IllegalStateException("Fail to get extension(Protocol) name from url("+ url.toString() + ") use keys([protocol])");
        }
        Protocol extension = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(extName);
        return extension.refer(clazz, url);
    }
}

To sum up, when we get the Adaptive extension class of an interface, it is actually an Adaptive class instance.

ExtensionLoader<Protocol> extensionLoader = 
                  ExtensionLoader.getExtensionLoader(Protocol.class);            
Protocol adaptiveExtension = extensionLoader.getAdaptiveExtension();
System.out.println(adaptiveExtension);

Output: com.alibaba.dubbo.rpc.Protocol$Adaptive@47f6473

After reading the above process, if you want to write your own logic to replace the process in Dubbo, it becomes very simple. Dubbo uses the Dubbo protocol by default to expose services. You can make a custom protocol to replace it.

1. Implementation class

First, we create a MyProtocol class that implements the Protocol interface.

package com.viewscenes.netsupervisor.protocol;

import com.alibaba.dubbo.common.URL;
import com.alibaba.dubbo.rpc.Exporter;
import com.alibaba.dubbo.rpc.Invoker;
import com.alibaba.dubbo.rpc.Protocol;
import com.alibaba.dubbo.rpc.RpcException;
import com.alibaba.dubbo.rpc.protocol.dubbo.DubboProtocol;

public class MyProtocol extends DubboProtocol implements Protocol{

    public int getDefaultPort() {
        return 28080;
    }
    public <T> Exporter<T> export(Invoker<T> invoker) throws RpcException { 
        URL url = invoker.getUrl();
        System.out.println("Customize the agreement for service exposure:"+url);    
        return super.export(invoker);
    }
    public <T> Invoker<T> refer(Class<T> type, URL url) throws RpcException {
        return super.refer(type, url);
    }
    public void destroy() {}
}

Then, in their own project, META-INF/services creates a com.alibaba.dubbo.rpc.Protocol file with the following contents:

myProtocol=com.viewscenes.netsupervisor.protocol.MyProtocol

Finally, modify the producer's configuration file:

<!-- Expose services on port 20880 with custom protocols -->
<dubbo:protocol name="myProtocol" port="20880"/>   

In this way, when starting the producer side project, Dubbo will call our custom MyProtocol class to complete the corresponding logical processing in the process of service exposure.