dubbo
SPI(Service Provider Interface)
- In essence, the fully qualified name of the interface implementation class is configured in the file, and the service loader reads the configuration file and loads the implementation class. This allows you to dynamically replace the implementation class for the interface at run time.
- SPI is used to design plug-ins for service providers in Java. The mechanism of dynamic loading based on policy pattern. We only define one interface in the program, and the specific implementation is handed over to different service providers; when the program is started, read the configuration file, and the configuration determines which implementation to call;
- Through SPI mechanism, we can provide extension function for our program. In Dubbo, based on SPI, we can easily expand Dubbo. For example, the protocol s and LoadBalance in Dubbo are all extended through SPI mechanism.
If you want to learn the source code of dubbo, you must understand the SPI mechanism. Next, we will understand the usage of JAVA SPI and dubbo SPI, and then analyze the source code of dubbo SPI. The dubbo source code of this article is based on version 2.7.5.
JAVA Native SPI example
- First, introduce the application of JAVA SPI. First, we define a Car interface
public interface Car { String getBrand(); }
- Define two implementation classes for the interface.
public class BM implements Car { public String getBrand() { System.out.println("BM car"); return "BM"; } } public class Benz implements Car { public String getBrand() { System.out.println("benz car"); return "Benz"; } }
- Then create the META-INF/services folder under resources, and create a file with the full name of the Car interface com.dubbo.dp.spi.Car. The content is the fully qualified class name of the interface implementation class.
com.dubbo.dp.spi.Benz com.dubbo.dp.spi.BM - Use the following to call all the implementation classes of the Car interface in the configuration file.
JAVA SPI realizes the decoupling between the interface definition and the specific business implementation. The application process can enable or replace specific components according to the actual business situation.public class JavaSPITest { @Test public void sayHello() throws Exception { ServiceLoader<Car> serviceLoader = ServiceLoader.load(Car.class); serviceLoader.forEach(car -> System.out.println(car.getBrand())); } }
Give an example : java.sql package defines an interface Driver, which is implemented by various service providers. When we need to use a database, we import the corresponding jar package.
shortcoming
- Cannot load on demand. When Java SPI loads extension points, it will load all available extension points at once, many of which are unnecessary and waste system resources;
- The way to get an implementation class is not flexible enough. It can only be obtained in the form of Iterator, not based on a parameter.
- AOP and dependency injection are not supported.
- JAVA SPI may lose the abnormal information of loading extension point, which makes it difficult to trace;
dubbo SPI example
- dubbo re implements a more powerful SPI mechanism, supports AOP and dependency injection, and uses cache to improve the performance of loading implementation classes, and supports the flexible acquisition of implementation classes. In the next part, the application and principle of SPI will be described.
Dubbo's SPI interface will be identified with @ SPI annotation. The main function of this annotation is to mark this interface as an SPI interface. The source code is as follows:
@Documented @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.TYPE}) public @interface SPI { /** * default extension name * Set default extension class */ String value() default ""; }
This annotation only works on the interface. value is used to set the default extension class
- First of all, I will explain the use of dubbo SPI. Add @ SPI annotation to the above Car interface. Its implementation class is temporarily unchanged, and the path and file name of the configuration file are also temporarily unchanged. The file content is adjusted as follows:
@SPI public interface Car { String getBrand(); }
The configuration file is configured by means of key value pairs, so that we can load the specified implementation classes on demand. Use the following:benz=com.dubbo.dp.spi.Benz bm=com.dubbo.dp.spi.BM
public class JavaSPITest { @Test public void sayHello() throws Exception { ExtensionLoader<Car> carExtensionLoader = ExtensionLoader.getExtensionLoader(Car.class); //Get the implementation class object on demand Car car = carExtensionLoader.getExtension("benz"); System.out.println(car.getBrand()); } }
The output is
benz car Benz
Dubbo SPI source code analysis
In the dubbo SPI example method, we first obtain an extension loader instance of the interface through the getExtensionLoader method of the ExtensionLoader, and then obtain the extension class object through the getExtension method of the ExtensionLoader. The source code is as follows. First, the getExtensionLoader method:
/** * Extension class loader cache, that is, extension point ExtendsLoader instance cache; key = extension interface value = extension class loader */ private static final ConcurrentMap<Class<?>, ExtensionLoader<?>> EXTENSION_LOADERS = new ConcurrentHashMap<>(); public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) { //Verify that the type class passed in is empty if (type == null) { throw new IllegalArgumentException("Extension type == null"); } //Verify that the type class passed in is an interface if (!type.isInterface()) { throw new IllegalArgumentException("Extension type (" + type + ") is not an interface!"); } //Verify that the type class passed in has @ SPI annotation if (!withExtensionAnnotation(type)) { throw new IllegalArgumentException("Extension type (" + type + ") is not an extension, because it is NOT annotated with @" + SPI.class.getSimpleName() + "!"); } //Query from the ExtensionLoader cache whether there is already an ExtensionLoader instance of the corresponding type ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type); if (loader == null) { //There is no new instance of ExtensionLoader and it is stored in the local cache EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type)); loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type); } return loader; }
getExtensionLoader will verify the incoming interface, including whether there is @ SPI annotation verification, which is also the reason for adding @ SPI to the interface. Then get the ExtensionLoader of the interface type from the extension ⒌ loaders cache. If not, create an ExtensionLoader of the interface type and put it into the cache, and return the ExtensionLoader.
Note that the construction method of the ExtensionLoader object is as follows: ExtensionLoader.getExtensionLoader obtains the extension class of the ExtensionFactory interface, and then obtains the target extension class from the extension class through getadaptive extension. It sets the objectFactory constant corresponding to the interface to Adaptive ExtensionFactory. Because @ Adaptive annotation is added to the class of Adaptive ExtensionFactory, the reason why it is Adaptive ExtensionFactory will be explained in later articles, and objectFactory will also be used later.
private ExtensionLoader(Class<?> type) { this.type = type; //Generally, type is not an ExtensionFactory class, then objectFactory is the default extension class of ExtensionFactory interface, AdaptiveExtensionFactory objectFactory = (type == ExtensionFactory.class ? null : ExtensionLoader.getExtensionLoader(ExtensionFactory.class).getAdaptiveExtension()); }
After getting the Loader loader of the interface through ExtensionLoader.getExtensionLoader, get the class object to be extended through getExtension method. The whole implementation process of this method is shown in the figure below
Refer to the execution flow chart to obtain the source code of the extended class object as follows:
/** * Extension point instance cache key = extension point name, value = Holder instance of extension instance */ private final ConcurrentMap<String, Holder<Object>> cachedInstances = new ConcurrentHashMap<>(); /** * Get interface extension class instance * 1.Check for presence in cache * 2.Create and return extension class instance * @param name The key of the extension class in the configuration file to be obtained * @return */ public T getExtension(String name) { if (StringUtils.isEmpty(name)) { throw new IllegalArgumentException("Extension name == null"); } if ("true".equals(name)) { // Get the default extension implementation class, that is, the default implementation class on the @ SPI annotation, such as @ SPI("benz") return getDefaultExtension(); } // Holder, as the name implies, is used to hold the target object, take it from the cache, and create if it is not final Holder<Object> holder = getOrCreateHolder(name); Object instance = holder.get(); // duplication check if (instance == null) { synchronized (holder) { instance = holder.get(); if (instance == null) { // Create an extension instance instance = createExtension(name); // Set instance to holder holder.set(instance); } } } return (T) instance; } /** * Get or create a Holder object */ private Holder<Object> getOrCreateHolder(String name) { // First, get the Holder object from the extension instance cache through the extension Holder<Object> holder = cachedInstances.get(name); if (holder == null) { //If not, a new empty Holder instance will be cached cachedInstances.putIfAbsent(name, new Holder<>()); holder = cachedInstances.get(name); } return holder; }
The logic of the above code is relatively simple. First, check the cache. If the cache misses, create an extension object. Dubbo contains a large number of extended point caches. This is a typical way of using space for time. It is also one of the reasons why Dubbo has strong performance, including
- When Dubbo SPI obtains the extension point, it will first read from the cache. If the cache does not exist, it will load the configuration file, cache the Class into memory according to the configuration, and it will not initialize directly.
- Extend the point instance cache. Dubbo will not only cache Class, but also the instance of Class. Each time an instance is fetched, it will take priority from the cache. If not, it will be loaded from the configuration, instantiated and cached in memory.
Let's take a look at the process of creating extension objects
/** * The extension instance is stored in memory and cached; key = extension class; value = extension class instance */ private static final ConcurrentMap<Class<?>, Object> EXTENSION_INSTANCES = new ConcurrentHashMap<>(); /** * To create an extension class instance, follow these steps * 1. Get all extension classes through getExtensionClasses and get the map map map of extension classes from the configuration file load * 2. Creating extended objects by reflection * 3. Injecting dependency (IOC) into extended objects * 4. Wrap the extension object in the corresponding Wrapper object (AOP) * @param name The key of the extension class in the configuration file to be obtained * @return Examples of extension classes */ private T createExtension(String name) { // Load all extension classes from the configuration file to get the map from "configuration item name" to "configuration class". Then take the corresponding extension class from the map according to the extension item name Class<?> clazz = getExtensionClasses().get(name); if (clazz == null) { throw findException(name); } try { //Get the corresponding instance object from the extension point cache T instance = (T) EXTENSION_INSTANCES.get(clazz); if (instance == null) { // //If there is no such extension point in the cache, the instance is created through reflection and stored in the cache EXTENSION_INSTANCES.putIfAbsent(clazz, clazz.newInstance()); //Then get the corresponding instance from the cache instance = (T) EXTENSION_INSTANCES.get(clazz); } // Inject dependency into the instance, and automatically inject the corresponding property instance through setter method injectExtension(instance); //Take all packaging classes out of the cache to form a packaging chain Set<Class<?>> wrapperClasses = cachedWrapperClasses; if (CollectionUtils.isNotEmpty(wrapperClasses)) { // Create Wrapper instance circularly to form Wrapper chain for (Class<?> wrapperClass : wrapperClasses) { instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance)); } } //Initialize instance and return initExtension(instance); return instance; } catch (Throwable t) { throw new IllegalStateException("....."); } }
The steps to create extension class objects are as follows:
- Load all extension classes from the configuration file through getExtensionClasses, and then get the target extension class by name
- Creating extended objects by reflection
- Injecting dependency into extended objects
- Wrap the extension object in the corresponding Wrapper object
The third and fourth steps are the concrete implementation of Dubbo IOC and AOP. First, we focus on the logic of the getExtensionClasses method.
Load all extension classes from the configuration file
-
Before obtaining the extension class by name, the mapping map of the extension item name and the extension class needs to be resolved according to the configuration file, and then the corresponding extension class can be taken out from the map according to the extension item name. The source code of getExtensionClasses method is as follows
/** * Extension point class cache key = extension, value = corresponding class object */ private final Holder<Map<String, Class<?>>> cachedClasses = new Holder<>(); /** * Parsing the mapping table map of interface extension name and extension class in configuration file * @return */ private Map<String, Class<?>> getExtensionClasses() { // Get the loaded extension point class from the cache Map<String, Class<?>> classes = cachedClasses.get(); //duplication check if (classes == null) { synchronized (cachedClasses) { classes = cachedClasses.get(); if (classes == null) { // Load extension class classes = loadExtensionClasses(); cachedClasses.set(classes); } } } return classes; }Check the cache first. If the cache fails to hit, load the extension class through loadExtensionClasses. The cache avoids the time-consuming of reading the configuration file multiple times. The logic of loading configuration file by loadExtensionClasses method is analyzed as follows
/** * Path of three default dubbo SPI scans */ private static final String SERVICES_DIRECTORY = "META-INF/services/"; private static final String DUBBO_DIRECTORY = "META-INF/dubbo/"; private static final String DUBBO_INTERNAL_DIRECTORY = DUBBO_DIRECTORY + "internal/"; private Map<String, Class<?>> loadExtensionClasses() { //Get and cache the default implementation class on the @ SPI annotation of the interface, value in @ SPI("value") cacheDefaultExtensionName(); Map<String, Class<?>> extensionClasses = new HashMap<>(); // Load the configuration file under the specified folder. The constants contain three folders: META-INF/dubbo/internal /, META-INF/dubbo /, META-INF/services / loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName(), true); //Compatible with historical versions loadDirectory(extensionClasses, DUBBO_INTERNAL_DIRECTORY, type.getName().replace("org.apache", "com.alibaba"), true); loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName()); loadDirectory(extensionClasses, DUBBO_DIRECTORY, type.getName().replace("org.apache", "com.alibaba")); loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName()); loadDirectory(extensionClasses, SERVICES_DIRECTORY, type.getName().replace("org.apache", "com.alibaba")); return extensionClasses; }The loadExtensionClasses method does two things in all. First, the method calls cacheDefaultExtensionName to parse the SPI annotation, and gets and caches the default extension class on the @ SPI annotation of the interface in cachedefaultname. Then call the loadDirectory method to load the specified folder profile.
SPI annotation parsing process is relatively simple, the source code is as follows. Only one default extension class is allowed.
private void cacheDefaultExtensionName() { // Get the SPI annotation. The type variable here is passed in when the getExtensionLoader method is called, representing the interface class final SPI defaultAnnotation = type.getAnnotation(SPI.class); if (defaultAnnotation == null) { return; } String value = defaultAnnotation.value(); if ((value = value.trim()).length() > 0) { String[] names = NAME_SEPARATOR.split(value); // Check whether the SPI annotation content is legal (at most one default implementation class) and throw an exception if it is illegal if (names.length > 1) { throw new IllegalStateException("..."); } // Set default extension class name if (names.length == 1) { cachedDefaultName = names[0]; } } }
From the source code, we can see that the loadExtensionClasses method has three paths to load the configuration file, which are META-INF/dubbo/internal /, META-INF/dubbo /, META-INF/services /. The method source code is as follows:
private void loadDirectory(Map<String, Class<?>> extensionClasses, String dir, String type) { loadDirectory(extensionClasses, dir, type, false); } /** * Load profile content * @param extensionClasses Developing map * @param dir Folder path * @param type Interface name * @param extensionLoaderClassLoaderFirst Whether to load ClassLoader of ExtensionLoader first */ private void loadDirectory(Map<String, Class<?>> extensionClasses, String dir, String type, boolean extensionLoaderClassLoaderFirst) { // fileName = folder path + type fully qualified name String fileName = dir + type; try { Enumeration<java.net.URL> urls = null; //Get the classloader of the current thread ClassLoader classLoader = findClassLoader(); // try to load from ExtensionLoader's ClassLoader first if (extensionLoaderClassLoaderFirst) { //Get the class loader to load ExtensionLoader.class ClassLoader extensionLoaderClassLoader = ExtensionLoader.class.getClassLoader(); //If the extensionLoaderClassLoaderFirst=true and the two class loaders are different, the extensionLoaderClassLoader is preferred if (ClassLoader.getSystemClassLoader() != extensionLoaderClassLoader) { urls = extensionLoaderClassLoader.getResources(fileName); } } // Load all files with the same name according to the file name if(urls == null || !urls.hasMoreElements()) { if (classLoader != null) { urls = classLoader.getResources(fileName); } else { urls = ClassLoader.getSystemResources(fileName); } } if (urls != null) { while (urls.hasMoreElements()) { java.net.URL resourceURL = urls.nextElement(); // Parse and load the implementation classes configured in the configuration file to the extensionClasses loadResource(extensionClasses, classLoader, resourceURL); } } } catch (Throwable t) { logger.error("").", t); } }
First, find the configuration file under the folder. The file name should be the fully qualified name of the interface. The class loader is used to obtain the file resource link, and then the implementation class configured in the configuration file is parsed and added to the extensionClasses. Let's continue to see how loadResource loads resources.
private void loadResource(Map<String, Class<?>> extensionClasses, ClassLoader classLoader, java.net.URL resourceURL) { try { try (BufferedReader reader = new BufferedReader(new InputStreamReader(resourceURL.openStream(), StandardCharsets.UTF_8))) { String line; // Read configuration content by line while ((line = reader.readLine()) != null) { final int ci = line.indexOf('#'); if (ci >= 0) { // Intercept the string before ා, and the content after ා, which is a comment, needs to be ignored line = line.substring(0, ci); } line = line.trim(); if (line.length() > 0) { try { String name = null; int i = line.indexOf('='); if (i > 0) { // Cut off the key and value by the equal sign = name = line.substring(0, i).trim(); line = line.substring(i + 1).trim(); } if (line.length() > 0) { // Load the class through reflection and cache the class through the loadClass method loadClass(extensionClasses, resourceURL, Class.forName(line, true, classLoader), name); } } catch (Throwable t) { ..... } } } } } catch (Throwable t) { logger.error(....); } }
The loadResource method is used to read and parse the configuration file. The configuration file is read by line. Each line is bounded by the equal sign = to intercept the key and value. The class is loaded through reflection. Finally, the class of the class is loaded into the map through the loadClass method, and the loaded class is classified and cached. The loadClass method is implemented as follows
/** * Load the class of the extension point implementation class into the map, and cache the loaded class by classification * For example, cacheadadaptiveclass, cachedWrapperClasses, cachedNames, etc * @param extensionClasses Container for loading profile classes * @param resourceURL Profile resource URL * @param clazz Class of extension point implementation class * @param name Name of extension point implementation class, key in the configuration file line * @throws NoSuchMethodException */ private void loadClass(Map<String, Class<?>> extensionClasses, java.net.URL resourceURL, Class<?> clazz, String name) throws NoSuchMethodException { //Determine whether the configured implementation class implements the type interface if (!type.isAssignableFrom(clazz)) { throw new IllegalStateException("..."); } //Cache according to the type of implementation class in the configuration // Check whether there is an Adaptive annotation on the target class, indicating that this class is an Adaptive implementation class, and cache it to cacheadadaptiveclass if (clazz.isAnnotationPresent(Adaptive.class)) { cacheAdaptiveClass(clazz); // Check whether clazz is a Wrapper type, and judge whether there is a construction method whose parameter is the interface class, and cache it to cachedWrapperClasses } else if (isWrapperClass(clazz)) { cacheWrapperClass(clazz); } else { // Check whether clazz has a default constructor, if not, throw an exception clazz.getConstructor(); // If the name of the key in the configuration file is empty, try to get the name from the Extension annotation, or use the lowercase class name as the name. // It's abandoned. We're not talking about it if (StringUtils.isEmpty(name)) { name = findAnnotationName(clazz); if (name.length() == 0) { throw new IllegalStateException("..."); } } //Use comma to divide name into string array String[] names = NAME_SEPARATOR.split(name); if (ArrayUtils.isNotEmpty(names)) { //If the implementation class configured by the extension point uses the @Activate annotation, the corresponding annotation information will be cached cacheActivateClass(clazz, names[0]); for (String n : names) { //Cache extension point implementation class and extension point name correspondence cacheName(clazz, n); //Finally, save the class to extensionClasses saveInExtensionClass(extensionClasses, clazz, n); } } } }
The loadClass method implements the classified caching function of extension points, such as packaging class, Adaptive extension point implementation class, and common extension point implementation class. Note that the Adaptive extension point implementation class @ Adaptive annotation has the following source code
*For example, given <code>String[] {"key1", "key2"}</code>: * <ol> * <li>find parameter 'key1' in URL, use its value as the extension's name</li> * <li>try 'key2' for extension's name if 'key1' is not found (or its value is empty) in URL</li> * <li>use default extension if 'key2' doesn't exist either</li> * <li>otherwise, throw {@link IllegalStateException}</li> * @return */ @Documented @Retention(RetentionPolicy.RUNTIME) @Target({ElementType.TYPE, ElementType.METHOD}) public @interface Adaptive { String[] value() default {}; }
The function of the annotation is to determine which adaptive extension class is injected. The target extension class is determined by the parameters in the URL. The parameter key in the URL is given by the value of the annotation. The value of the key is the name of the target extension class.
- If there are multiple values in the annotation, search for the corresponding key in the URL according to the subscript from small to large. Once found, use the value of the key as the target extension class name.
- If none of these values have a corresponding key in the url, use the default value on the spi.
@In most cases, the adaptive annotation works on methods and classes. When the adaptive annotation works on classes, Dubbo will not generate a proxy class for the class. When annotated on a method (interface method), Dubbo generates a proxy class for that method. Adaptive annotation in the interface method indicates that the extended loading logic needs to be automatically generated by the framework. Annotation on the class, the extended loading logic is manually encoded.
The loadClass scan above is for classes. In Dubbo, only two classes are annotated by @ Adaptive, namely, Adaptive compiler and Adaptive extensionfactory.
Setting cacheAdaptiveClass with the loadClass method will cause the cacheAdaptiveClass of the interface not to be empty. Later, this extension class will be used by default with the highest priority.
Back to the main line, when the loadClass method is executed, all the extension classes in the configuration file have been loaded into the map, so far, the process of loading the cache class is analyzed.
Dubbo IOC
After the execution process of getExtensionClasses() method is completed, the extension Class can be obtained by taking the corresponding extension Class from the map according to the extension item name, creating an instance through reflection, and injecting dependency into the instance through the injectExtension(instance); this part will be introduced in the next article.
DUBBO AOP
When the object extension (t instance) method is executed, the wrapper is executed in createExtension(String name). Similar to AOP in spring, dubbo uses the decorator pattern.
Set<Class<?>> wrapperClasses = cachedWrapperClasses; if (CollectionUtils.isNotEmpty(wrapperClasses)) { // Create Wrapper instance circularly to form Wrapper chain for (Class<?> wrapperClass : wrapperClasses) { instance = injectExtension((T) wrapperClass.getConstructor(type).newInstance(instance)); } }
The cachedWrapperClasses here is known through the previous analysis, that is, when parsing the configuration file, judge whether it is an extension class of Wrapper type, and + + judge whether it is based on whether there is an interface class + + in the construction method, and cache it to cachedWrapperClasses.
Executing wrapperClass.getConstructor(type).newInstance(instance) will get the construction method of the Wrapper class. The parameter of the method is the interface type, and generate a Wrapper object containing the instance of the extension class through reflection, and then inject the dependency of the Wrapper object through the injectExtension. In this way, the Wrapper chain is generated. Note here that the Wrapper class behind the content in the configuration file will be wrapped in a relatively outer layer. The following is an example of DUBBO AOP. We continue to use the above Car interface and implementation class, and add an implementation class. The code is as follows
public class CarWrapper implements Car{ private Car car; /** * There is a constructor that contains the parameters of the interface class */ public CarWrapper(Car car) { this.car = car; } @Override public String getBrand() { System.out.println("check"); String result = car.getBrand(); System.out.println("Log"); return result; } }
If the interface implements car, holds a car object, and has a constructor whose parameter is car interface type, then the class will be recognized as the Wrapper class of the interface. Then we can extend the facet function in the method, and then call the car object to execute its method to realize the AOP function.
Add the wrapper implementation class to the content of the configuration file, as follows
benz=com.xiaoju.automarket.energy.scm.rpc.Benz bm=com.xiaoju.automarket.energy.scm.rpc.BM com.xiaoju.automarket.energy.scm.rpc.CarWrapper #Packaging class
After executing the following code to obtain the extended class instance of benz, calling its method will be wrapped by Wrapper
public class TestAOP { public static void main(String[] args) { ExtensionLoader<Car> carExtensionLoader = ExtensionLoader.getExtensionLoader(Car.class); Car car = carExtensionLoader.getExtension("benz"); System.out.println(car.getBrand(null)); } }
give the result as follows
check benz car Log Benz
As we expected, it implements the aspect function of Wrapper class.
summary
This paper introduces the usage of Java SPI and Dubbo SPI respectively, and analyzes the process of loading extended classes of Dubbo SPI. At the same time, the principle of Dubbo AOP is analyzed. If there are mistakes in the article, I hope you can correct them.
The next article will talk about Dubbo SPI's extension point adaptation mechanism, which is also involved in dubbo automatic injection, that is, spring extensionfactory's implementation of factory.getExtension involves loading adaptive extension points.
Author: Ding Peng