Android Thermal Repair Technology: Analysis of QQ Space Patch Scheme (3)

Keywords: Gradle Android Java Apache

As mentioned earlier, in order to achieve the goal of hot update, bytecode files must be manipulated after dex subcontracting is completed. ASM and javaassist are commonly used bytecode manipulation tools. By contrast, ASM provides a series of bytecode instructions, which are more efficient but require users to have a certain understanding of bytecode operations. Java asist is inefficient, but the threshold is low, so this paper chooses Java asist. Reference to Java asist The Dynamics of Java Programming, Part 4: Class Conversion with Javassist

In normal App development process, the compilation and packaging process is automatically completed by Android Studio. There is no need for human intervention if there is no special requirement, but to achieve pile insertion, the process of pile insertion must be added to the automated packaging process of Android Studio.

1. Gradle,Task,Transform,Plugin

Android Studio uses Gradle as a build tool, so it's necessary to understand the basic concepts and processes of Gradle construction. If you are not familiar with it, you can refer to the following articles:
- Gradle Quick Start, One of Gradle Learning Series
- Deep Understanding of Android Gradle

Gradle's construction project is essentially completed through a series of Tasks, so there is a Task of packaging dex in the process of building apk. Gradle version 1.5 provides a new API: Transform. The official document describes Transform as follows:

The goal of this API is to simplify injecting custom class manipulations without having to deal with tasks, and to offer more flexibility on what is manipulated. The internal code processing (jacoco, progard, multi-dex) have all moved to this new mechanism already in 1.5.0-beta1.

    1. The Dex class is gone. You cannot access it anymore through the variant API (the getter is still there for now but will throw an exception)
    1. Transform can only be registered globally which applies them to all the variants. We'll improve this shortly.
    1. There's no way to control ordering of the transforms.

Once registered, the Transform task is inserted into the task execution queue just before dex packages task. So in order to implement piling, you must create a Task of the Transform class.

1.1 Task

Gradle's execution script is done by a series of Tasks. Task has an important concept: output of input. Each task needs input, then the input is processed and the output is output.

1.2 Plugin

Another important concept of Gradle is Plugin. The whole Gradle construction system is composed of one plugin, the actual Gradle is only a framework, providing basic task and specified standards. The execution logic of each task is defined in one plugin after another. Detailed concepts can be referred to: Writing Custom Plugins
In Android development, we often use plugin s such as "com.android.application", "com.android.library", "java" and so on.
Each Plugin contains a series of tasks, so the process of executing the gradle script is the task contained in the plugin of the apply of the target script.

1.3 Create a Plugin containing the Transform task

    1. Create a new module, select library module, module name must be BuildSrc
    1. Delete all files under module, except build.gradle, and empty the contents of build.gradle
    1. Then create the following directory src-main-groovy
    1. Modify build.gradle as follows, synchronize
apply plugin: 'groovy'

repositories {
    jcenter()
}

dependencies {
    compile gradleApi()
    compile 'com.android.tools.build:gradle:1.5.0'
    compile 'org.javassist:javassist:3.20.0-GA'//Java asist dependency
}
    1. Create new package s and classes like normal module s, but the classes here end with groovy, select file when creating new classes, and use. groovy as the suffix
    1. Custom Plugin:
package com.hotpatch.plugin

import com.android.build.api.transform.*
import com.android.build.gradle.internal.pipeline.TransformManager
import org.apache.commons.codec.digest.DigestUtils
import org.apache.commons.io.FileUtils
import org.gradle.api.Project

public class PreDexTransform extends Transform {

    Project project;

    public PreDexTransform(Project project1) {
        this.project = project1;

        def libPath = project.project(":hack").buildDir.absolutePath.concat("/intermediates/classes/debug")
        println libPath
        Inject.appendClassPath(libPath)
        Inject.appendClassPath("/Users/liyazhou/Library/Android/sdk/platforms/android-24/android.jar")
    }
    @Override
    String getName() {
        return "preDex"
    }

    @Override
    Set<QualifiedContent.ContentType> getInputTypes() {
        return TransformManager.CONTENT_CLASS
    }

    @Override
    Set<QualifiedContent.Scope> getScopes() {
        return TransformManager.SCOPE_FULL_PROJECT
    }

    @Override
    boolean isIncremental() {
        return false
    }

    @Override
    void transform(Context context, Collection<TransformInput> inputs, Collection<TransformInput> referencedInputs, TransformOutputProvider outputProvider, boolean isIncremental) throws IOException, TransformException, InterruptedException {

        // inputs traversing transfrom
        // There are two types of inputs: directory and jar, which need to be traversed separately.
        inputs.each {TransformInput input ->
            input.directoryInputs.each {DirectoryInput directoryInput->

                //TODO Injection Code
                Inject.injectDir(directoryInput.file.absolutePath)

                def dest = outputProvider.getContentLocation(directoryInput.name,
                        directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY)
                // Copy the input directory to the output specified directory
                FileUtils.copyDirectory(directoryInput.file, dest)
            }

            input.jarInputs.each {JarInput jarInput->


                //TODO Injection Code
                String jarPath = jarInput.file.absolutePath;
                String projectName = project.rootProject.name;
                if(jarPath.endsWith("classes.jar")
                        && jarPath.contains("exploded-aar/"+projectName)
                        // hotpatch module is used to load dex without injecting code
                        && !jarPath.contains("exploded-aar/"+projectName+"/hotpatch")) {
                    Inject.injectJar(jarPath)
                }

                // Rename the output file (which conflicts with the directory copyFile)
                def jarName = jarInput.name
                def md5Name = DigestUtils.md5Hex(jarInput.file.getAbsolutePath())
                if(jarName.endsWith(".jar")) {
                    jarName = jarName.substring(0,jarName.length()-4)
                }
                def dest = outputProvider.getContentLocation(jarName+md5Name, jarInput.contentTypes, jarInput.scopes, Format.JAR)
                FileUtils.copyFile(jarInput.file, dest)
            }
        }
    }
}
  • 8.Inject.groovy, JarZipUtil.groovy
package com.hotpatch.plugin

import javassist.ClassPool
import javassist.CtClass
import org.apache.commons.io.FileUtils

public class Inject {

    private static ClassPool pool = ClassPool.getDefault()

    /**
     * Add classPath to ClassPool
     * @param libPath
     */
    public static void appendClassPath(String libPath) {
        pool.appendClassPath(libPath)
    }

    /**
     * Travel through all classes in the directory and inject code into all classes.
     * The following class es do not require code injection:
     * --- 1. R Document-related
     * --- 2. Build Config
     * --- 3. Application
     * @param path Directory path
     */
    public static void injectDir(String path) {
        pool.appendClassPath(path)
        File dir = new File(path)
        if(dir.isDirectory()) {
            dir.eachFileRecurse { File file ->

                String filePath = file.absolutePath
                if (filePath.endsWith(".class")
                        && !filePath.contains('R$')
                        && !filePath.contains('R.class')
                        && !filePath.contains("BuildConfig.class")
                        // Here is the name of the application, which can be configured by itself.
                        && !filePath.contains("HotPatchApplication.class")) {
                    // Application package name, self-configurable
                    int index = filePath.indexOf("com/hotpatch/plugin")
                    if (index != -1) {
                        int end = filePath.length() - 6 // .class = 6
                        String className = filePath.substring(index, end).replace('\\', '.').replace('/','.')
                        injectClass(className, path)
                    }
                }
            }
        }
    }

    /**
     * Here you need to decompress the jar package, inject the code, and then regenerate the jar package
     * @path jar Absolute Path of Packet
     */
    public static void injectJar(String path) {
        if (path.endsWith(".jar")) {
            File jarFile = new File(path)


            // Save path of jar package after decompression
            String jarZipDir = jarFile.getParent() +"/"+jarFile.getName().replace('.jar','')

            // Unzip the jar package and return the complete set of class names for all classes in the jar package (with the. class suffix)
            List classNameList = JarZipUtil.unzipJar(path, jarZipDir)

            // Delete the original jar package
            jarFile.delete()

            // Injection code
            pool.appendClassPath(jarZipDir)
            for(String className : classNameList) {
                if (className.endsWith(".class")
                        && !className.contains('R$')
                        && !className.contains('R.class')
                        && !className.contains("BuildConfig.class")) {
                    className = className.substring(0, className.length()-6)
                    injectClass(className, jarZipDir)
                }
            }

            // Pack jar from scratch
            JarZipUtil.zipJar(jarZipDir, path)

            // Delete directories
            FileUtils.deleteDirectory(new File(jarZipDir))
        }
    }

    private static void injectClass(String className, String path) {
        CtClass c = pool.getCtClass(className)
        if (c.isFrozen()) {
            c.defrost()
        }
        def constructor = c.getConstructors()[0];
        constructor.insertAfter("System.out.println(com.hotpatch.hack.AntilazyLoad.class);")
        c.writeFile(path)
    }

}
package com.hotpatch.plugin

import java.util.jar.JarEntry
import java.util.jar.JarFile
import java.util.jar.JarOutputStream
import java.util.zip.ZipEntry

/**
 * Created by hp on 2016/4/13.
 */
public class JarZipUtil {

    /**
     * Unzip the jar package to the specified directory
     * @param jarPath jar Absolute Path of Packet
     * @param destDirPath jar Save Path after Packet Unzipping
     * @return Returns the complete set of class names for all classes contained in the jar package, including one data such as com.aitski.hotpatch.Xxxx.class
     */
    public static List unzipJar(String jarPath, String destDirPath) {

        List list = new ArrayList()
        if (jarPath.endsWith('.jar')) {

            JarFile jarFile = new JarFile(jarPath)
            Enumeration<JarEntry> jarEntrys = jarFile.entries()
            while (jarEntrys.hasMoreElements()) {
                JarEntry jarEntry = jarEntrys.nextElement()
                if (jarEntry.directory) {
                    continue
                }
                String entryName = jarEntry.getName()
                if (entryName.endsWith('.class')) {
                    String className = entryName.replace('\\', '.').replace('/', '.')
                    list.add(className)
                }
                String outFileName = destDirPath + "/" + entryName
                File outFile = new File(outFileName)
                outFile.getParentFile().mkdirs()
                InputStream inputStream = jarFile.getInputStream(jarEntry)
                FileOutputStream fileOutputStream = new FileOutputStream(outFile)
                fileOutputStream << inputStream
                fileOutputStream.close()
                inputStream.close()
            }
            jarFile.close()
        }
        return list
    }

    /**
     * Repackage jar
     * @param packagePath Pack all files in this directory into jar
     * @param destPath Absolute path of packaged jar package
     */
    public static void zipJar(String packagePath, String destPath) {

        File file = new File(packagePath)
        JarOutputStream outputStream = new JarOutputStream(new FileOutputStream(destPath))
        file.eachFileRecurse { File f ->
            String entryName = f.getAbsolutePath().substring(packagePath.length() + 1)
            outputStream.putNextEntry(new ZipEntry(entryName))
            if(!f.directory) {
                InputStream inputStream = new FileInputStream(f)
                outputStream << inputStream
                inputStream.close()
            }
        }
        outputStream.close()
    }
}
    1. Add a new plug-in to the build.gradle file under app module: apply plugin: com.hotpatch.plugin.Register

2. Create hack.jar

Create a separate module named com.hotpatch.plugin.AntilazyLoad:

package com.hotpatch.plugin
public class AntilazyLoad {
}

Pack hack.jar using the method described in the previous blog. Then copy hack.jar to the assets directory under app module. Also note: app module cannot rely on hack module. The reason for creating a hack module and artificially inserting dependencies on other hack.jar classes during dex packaging is to keep the apk file from being labeled CLASS_ISPREVERIFIED during installation.
In addition, because hack.jar is in assets, it is necessary to load hack.jar before loading patch_dex. In addition, since the DEX files loading other paths are executed in the Application.onCreate() method, hack.jar has not been loaded at this time, this is why it is not possible to insert piles in the Application when inserting piles in the previous chapter.

After the introduction of the process of pile insertion, the whole process of thermal repair is almost the same. Readers can refer to the complete code for demo trial. Hotpatch Demo

Posted by oQEDo on Sat, 06 Jul 2019 15:13:32 -0700