Written in front

This article is original and reproduced. Please indicate the address in the form of a link: http://kymjs.com/code/2016/05/08/01

Chapter 1 of "Plug-in from Abandonment to Pick-up" begins with a picture:
 
This picture is my understanding of the three technical points of Android plug-in technology and their application scenarios. Today with [Qzone thermal restoration scheme as an example] Let's talk about the implementation of hot repair in plug-in.

principle

ClassLoader

In Java, ClassLoader is used to load a class.
There are three classLoaders in Android: URLClassLoader, PathClassLoader and DexClassLoader. among

• URLClassLoader can only be used to load jar files, but because dalvik can't recognize jar directly, it can't be used in Android.
• PathClassLoader can only load installed apk s. Because PathClassLoader only reads dex files in the / data/dalvik-cache directory. For example, we installed a package named com.hujiang.xxx When the APK is installed, a data@app@com.hujiang.xxx-1.apk@classes.dex will be produced in the / data/dalvik-cache directory. ODEX file. When using PathClassLoader to load the apk, it will go to this folder to find the corresponding ODEX file. If the APK is not installed, it will naturally report ClassNotFoundException.
• DexClassLoader is the ideal loader. Its constructor contains four parameters, namely:
  1. dexPath refers to the path of the APK or jar file where the target class is located. The class loader will find the specified target class from the path. The class must be the full path of the APK or jar. If multiple paths are to be included, the paths must be separated by a specific partitioner, which can be obtained using System.getProperty("path.separtor").
  2. dexOutputDir, because dex files are included in APK or Jar files, it is necessary to extract dex files from APK or Jar files before loading the target classes. This parameter is to determine the storage path of the extracted dex files. In Android systems, an application generally corresponds to a Linux user id, and the application only has the right to write its own data directory path. Therefore, this parameter can be used to extract dex files from APK or Jar files. Parameters can use the program's data path.
  3. libPath, refers to the path of C/C++ inventory used in the target class
  4. classload is the parent loader of the loader, usually the loader of the current executing class.

from framework source code Under the dalvik.system package, we found the source code of DexClassLoader, which has no egg use. The actual content is in its parent class BaseDexClassLoader, which, incidentally, is at least useful at API 14. It contains two variables:

/** originally specified path (just used for {@code toString()}) */
private final String originalPath;
 
/** structured lists of path elements */
private final DexPathList pathList;

You can see the comment: pathList is a multi-dex structure list, see Its source code

/*package*/ final class DexPathList {
    private static final String DEX_SUFFIX = ".dex";
    private static final String JAR_SUFFIX = ".jar";
    private static final String ZIP_SUFFIX = ".zip";
    private static final String APK_SUFFIX = ".apk";

    /** class definition context */
    private final ClassLoader definingContext;

    /** list of dex/resource (class path) elements */
    private final Element[] dexElements;

    /** list of native library directory elements */
    private final File[] nativeLibraryDirectories;

As you can see, the dexElements annotation is a list of dex, so we can put each Element in the list. Think of it as an dex.

Now let's sort out the idea that DexClassLoader contains an array of dex Elements [] dexElements, where each dex file is an Element that traverses when a class needs to be loaded. dexElements, if a class is found, loads it, if it is not found to continue searching from the next dex file.

So our implementation is to insert the plug-in dex into the front of Elements. The advantage of this is that not only can a class be loaded dynamically, but also because DexClassLoader will load the front class first, we also implement the hot repair function of the host apk.

ODEX process

This is the whole principle of hot repair, which is to insert an dex into the Classloader list. But if you do it here, you'll find one problem, which is Problems arising from the ODEX process.
Before we talk about the process of this egg pain, there are several questions to understand.
Why can't Android recognize. class files, but only dex files?
Because dex is an optimization of class, it compresses class greatly, such as the following structure of a class file (extracted from Mr. Deng Fanping's blog)

DEX compresses all classes in the entire Android project into one (or several) DEX file, incorporates the constants of each class, class version information, etc., for example, each class has the same string, and only one in dex is enough. So on Android, the dalvik virtual machine can't recognize a normal class file because it can't recognize the structure of the class file.
Here is the structure of an dex file

Interested readers can read The Deep Understanding of Android.

In fact, the dalvik virtual machine does not read DEX files directly, but when an APK is installed, it will first optimize and generate an ODEX file, namely Optimized dex. Why should we optimize it? It's still for efficiency.  
However, Class - > DEX is for platform independent optimization;
And DEX - > ODEX optimizes the hardware configuration of different mobile phones for different platforms.
In this process, when the virtual machine starts optimizing, there will be an option called verify. When the verify option is turned on, a verification will be performed. The purpose of the verification is to determine whether the class refers to other classes in dex. If not, the class will be labeled CLASS_ISPREVERIFIED. Once this flag is hit, it is no longer possible to replace this class from other dex. When this option is turned on, it is controlled by the virtual machine.

Bytecode operation

Now that we know the reason, the solution will come naturally. If you don't refer to classes in other dex, they will be marked, so let's refer to classes in other dex.

ClassReader: This class is used to parse compiled class bytecode files.
ClassWriter: This class is used to rebuild compiled classes, such as modifying class names, attributes, and methods, and even generating bytecode files for new classes.
ClassAdapter: This class also implements the ClassVisitor interface, which delegates method calls to another ClassVisitor object.

/**
 * Inject the Inject class when the object is initialized
 *
 * @Note https://www.ibm.com/developerworks/cn/java/j-lo-asm30/
 * @param inputStream File input stream for lass to be injected
 * @return Returns the binary array of lass files after injection
 */
private static byte[] referHackWhenInit(InputStream inputStream) {
    //This class is used to parse compiled class bytecode files.
    ClassReader cr = new ClassReader(inputStream);
    //This class is used to rebuild compiled classes, such as modifying class names, attributes, and methods, and even to generate bytecode files for new classes.
    ClassWriter cw = new ClassWriter(cr, 0);
    //Visitors to a Class can be used to create changes to a Class
    ClassVisitor cv = new ClassVisitor(Opcodes.ASM4, cw) {
        @Override
        public MethodVisitor visitMethod(int access, String name, String desc,
                                         String signature, String[] exceptions) {
            MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
            //If the method name is <init>, the constructor function for each class is <init>.
            if ("<init>".equals(name)) {
                //Add your own defined operation to the original visitMethod operation
                mv = new MethodVisitor(Opcodes.ASM4, mv) {
                    @Override
                    void visitInsn(int opcode) {
                        //Opcodes can be regarded as keywords
                        if (opcode == Opcodes.RETURN) {
                            //visitLdcInsn() writes a value to the stack, which can be a Class class name / method method name / desc method description
                            //This is equivalent to inserting a statement: Class a = Inject.class;
                            super.visitLdcInsn(Type.getType("Lcom/hujiang/hotfix/Inject;"));
                        }
                        //Perform other operations corresponding to opcode
                        super.visitInsn(opcode);
                    }
                }
            }
            //Chain of Responsibility Completed, Return
            return mv;
        }
    };
    //This method accept s an object instance that implements the ClassVisitor interface as a parameter, and then invokes each method of the ClassVisitor interface in turn.
    //Users cannot control the order of method calls, but they can provide different Visitors to modify the bytecode tree differently.
    //Here, the purpose of calling this step is to have the visitMethod method above called
    cr.accept(cv, 0);
    return cw.toByteArray();
}

code implementation

For reference nuwa The implementation in this paper, first of all, is how dex is inserted into the Classloader list. In fact, it is a reflection:

public static void injectDexAtFirst(String dexPath, String defaultDexOptPath) throws NoSuchFieldException, IllegalAccessException, ClassNotFoundException {
    DexClassLoader dexClassLoader = new DexClassLoader(dexPath, defaultDexOptPath, dexPath, getPathClassLoader());
    Object baseDexElements = getDexElements(getPathList(getPathClassLoader()));
    Object newDexElements = getDexElements(getPathList(dexClassLoader));
    Object allDexElements = combineArray(newDexElements, baseDexElements);
    Object pathList = getPathList(getPathClassLoader());
    ReflectionUtils.setField(pathList, pathList.getClass(), "dexElements", allDexElements);
}

Firstly, PathList.dexElements in dex of host application and patch are obtained, and two dexElements are added. Arrays are spliced, patch arrays are put in front, and finally the spliced arrays are assigned back to Classloader.

More importantly, nuwa is his groovy script, complete code: Here Because there are a lot of codes, we will only talk about the realization and purpose of two key points. Specific content can be directly viewed the source code.

//Get all the input files, that is, all the jar files of preDex
Set<File> inputFiles = preDexTask.inputs.files.files
inputFiles.each { inputFile ->
    def path = inputFile.absolutePath
    //If it's not a support package or an imported dependency library, start generating the hotfix package for the code modification section
    if (HotFixProcessors.shouldProcessPreDexJar(path)) {
        HotFixProcessors.processJar(classHashFile, inputFile, patchDir, classHashMap, includePackage, excludeClass)
    }
}

HotFixProcessors.processJar() is the first function of the script to find out which classes have changed and which patches should be generated.
By comparing the hashFile file with the hashFile file of the original host project (i.e. the class HashMap parameter here), all the modified classes are generated into the class files of these classes, as well as the set jar files of all the modified class files.

Set<File> inputFiles = dexTask.inputs.files.files
inputFiles.each { inputFile ->
    def path = inputFile.absolutePath
	if (path.endsWith(".class") && !path.contains("/R\$") && !path.endsWith("/R.class") && !path.endsWith("/BuildConfig.class")) {
        if (HotFixSetUtils.isIncluded(path, includePackage)) {
            if (!HotFixSetUtils.isExcluded(path, excludeClass)) {
                def bytes = HotFixProcessors.processClass(inputFile)
                path = path.split("${dirName}/")[1]
                def hash = DigestUtils.shaHex(bytes)
                classHashFile.append(HotFixMapUtils.format(path, hash))

                if (HotFixMapUtils.notSame(classHashMap, path, hash)) {
                    HotFixFileUtils.copyBytesToFile(inputFile.bytes, HotFixFileUtils.touchFile(patchDir, path))
                }
            }
        }
    }
}

This section is the second function of the script, that is, the purpose of bytecode operation above. In order to prevent the class from being hit by the virtual machine CLASS_ISPREVERIFIED, bytecode writing is required. HotFixProcessors.processClass() is the code that actually writes the bytecode.

It's like a bad ending.

In addition to nuwa, there is an open source implementation for the same solution. HotFix They are almost the same, so just look at one.

Supplementary on 10 May
See a lot of friends ask, if confuse the code what to do. During the Gradle plug-in compilation process, there is a Proguard Task, whose name should show that he is responsible. For the proguard task, we can save the obfuscation rules (i.e. the bundles out of BUG) when we first execute. This obfuscation rule is stored in a mapping file in the project directory. When we need to execute the generation of hot fix patches, we can apply the mapping rules of online bundles to this compilation, and then we can generate the obfuscated classes which are the same as those on-line obfuscated classes. The class name has been patched. The concrete implementation can be seen The applymapping() method for nuwa projects.