Preface
For java developers, it seems that they prefer to do things during compilation. For example, in order to achieve AOP programming, they like to use bytecode generation technology, commonly used traceless burial points, time-consuming statistics and so on. So how does Android do that? The so-called bytecode stuffing technology, in fact, is to modify the compiled class file, add its own bytecode to it, and then pack the modified class file when packaging. In order to easily modify and adapt the translated class files, Google Dad developed a set of gradle-related libraries, namely gradle-transform-api. With this tool, we can modify the class files ourselves. Let's look at the specific practices below.
1. Implementing a gradle Plugin
To use gradle-transform-api, we must first implement a gradle plug-in, and then register a Transform in the plug-in. There are three ways to implement the plug-in. Here is a brief introduction. See the details. Official Documents
1.1 is implemented directly in the build.gradle file:
class GreetingPlugin implements Plugin<Project> { void apply(Project project) { project.task('hello') { doLast { println 'Hello from the GreetingPlugin' } } } } // Apply the plugin apply plugin: GreetingPlugin
The key is to implement Plugin < Project > interface
1.2 Create a buildsrc module
The first way is not suitable for the development of complex plug-ins. If it is only the need of our own project and the plug-ins are more complex, we can create a buildsrc module, and then move the GreetingPlugin class above to this module. This is similar to the other way below. We will not introduce it in detail here. Interest can be seen in official documents.
1.3 Separate Works
Create a module or project of your own, which is the most common way to do this. You can see the directory structure.
├── pluginmodule │ ├── build.gradle │ └── src │ └── main │ ├── groovy │ │ └── com │ │ └── jianglei │ │ └── plugin │ │ ├── MethodTracePluginPlugin.groovy │ └── resources │ └── META-INF │ └── gradle-plugins │ └── com.jianglei.method-tracer.properties
Among them, JlLogPlugin is the plug-in implementer:
class MethodTracePlugin implements Plugin<Project> { @Override void apply(Project project) { project.getExtensions() .create("methodTrace", MethodTraceExtension.class) //Ensure that only build.gradle files with application s are introduced if (!project.plugins.hasPlugin('com.android.application')) { throw new GradleException('Android Application plugin required') } project.getExtensions().findByType(AppExtension.class) .registerTransform(new MethodTraceTransform(project)) } }
In addition, com.jianglei.method-tracer.properties is used to announce who is the plug-in implementer, and the name of the file is the name when you want to refer to it.
apply plugin: 'com.jianglei.jllog'
The document reads as follows:
implementation-class=com.jianglei.plugin.MethodTracePlugin
2. Implement a transform
In the first step, we register a transform, which can input the compiled class file, and then we process the class file and output the modified file.
The code is simple:
class MethodTraceTransform extends Transform { private Project project MethodTraceTransform(Project project) { this.project = project } @Override String getName() { return "MethodTrace" } @Override Set<QualifiedContent.ContentType> getInputTypes() { return TransformManager.CONTENT_CLASS } @Override Set<? super QualifiedContent.Scope> getScopes() { //This time, only com.android.application plug-ins are allowed in the main module (build.gradle) //So we need to modify all module s. return TransformManager.SCOPE_FULL_PROJECT } @Override boolean isIncremental() { return true } @Override void transform(TransformInvocation transformInvocation) throws TransformException, InterruptedException, IOException { } }
The core of transform ing is to override these methods. Let's explain them one by one.
2.1 getName()
This method is only used to define the name of the transform ation task, just choose one at will.
2.2 getInputTypes()
This is used to limit the type of file that can be processed by this transform. Generally speaking, all we need to deal with are class files. We will return to TransformManager.CONTENT_CLASS. Of course, if you want to deal with resource files, you can use TransformManager.CONTENT_RESOURCES. Here, as needed, you can see other configurations. Official website javadoc document Yes, there is a need for scientific Internet access.
2.3 getScopes()
In 2.2, we specify which files to process, so what files are we specifying here? For example, if we want to process a class file, but the class file can be either the current module, or a sub-module, or a third-party jar package, this is used to specify this. Let's see what options are available:
public static enum Scope implements QualifiedContent.ScopeType { PROJECT(1), SUB_PROJECTS(4), EXTERNAL_LIBRARIES(16), TESTED_CODE(32), PROVIDED_ONLY(64), /** @deprecated */ @Deprecated PROJECT_LOCAL_DEPS(2), /** @deprecated */ @Deprecated SUB_PROJECTS_LOCAL_DEPS(8); private final int value; private Scope(int value) { this.value = value; } public int getValue() { return this.value; } }
Basically from the name can also see the scope of action, of course, how to choose or pay attention to some specific, we will introduce later.
2.4 inIncremental()
Whether incremental compilation is supported or not, a qualified transform ation should reasonably support incremental compilation.
2.5 transform()
This method is the focus of our work. In this method, we get the input class file, then make some modifications, and finally output the modified class file. It is also divided into three steps:
2.5.1 Getting Input Files
transformInvocation.inputs.each {input -> transformInvocation.inputs .each { input -> transformSrc(transformInvocation, input) transformJar(transformInvocation, input) } }
There are two kinds of input files, one is the source code compiled class file under the src of this module, the other is the third-party jar package file, which we need to deal with separately.
2.5.2 Get Output Path
When the input file is available, we need to determine the output path first. Here we should pay attention to the fact that the output path must be obtained in a special way, and can not be specified at will. Otherwise, the next task will not be able to obtain your output file this time, and the compilation will fail.
For the class file output path of source code compilation, obtain as follows:
def outputDirFile = transformInvocation.outputProvider.getContentLocation( directoryInput.name, directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY )
For the output path of the jar package, the following is obtained:
def outputFile = transformInvocation.outputProvider.getContentLocation( jarInput.name, jarInput.contentTypes, jarInput.scopes, Format.JAR )
2.5.3 Processing Input Files
After the above steps, we can get the input files and determine the output path. Now we just need to process these files and then output them to the output path.
Firstly, the class file generated by source code compilation under src is processed.
private void transformSrc(TransformInput input){ input.directoryInputs.each { directoryInput -> //This is to store all directory files in a list collection. def allFiles = DirectoryUtils.getAllFiles(directoryInput.file) for (File file : allFiles) { //For example, the full path entered in the previous file is / A/B/com/jianglei/test/Test.class, and the output path obtained is / A/B/com/jianglei/test.class. // / transform/MethodTrace/debug, replaced by / transform/MethodTrance/debug/com/jianglei/test/Test.class def outputFullPath = file.absolutePath.replace(inputFilePath, outputFilePath) def outputFile = new File(outputFullPath) if (!outputFile.parentFile.exists()) { outputFile.parentFile.mkdirs() } //In this method, you can modify the class file as much as you like, and then output it to the output File. //If not, at least copy the original document. MethodTraceUtils.traceFile(file, outputFile) } } }
The comments are clearly written. Note here that even if you don't want to modify the class file, you should copy it as it is, otherwise the file will be lost.
Next, we process the jar file:
private void transformJar(TransformInvocation transformInvocation, TransformInput input, boolean isIncrement, boolean isConfigChange, Map<String, String> lastJarMap, Set<String> curJars, MethodTraceExtension extension) { for (JarInput jarInput : input.jarInputs) { def outputFile = transformInvocation.outputProvider.getContentLocation( jarInput.name, jarInput.contentTypes, jarInput.scopes, Format.JAR ) //This method is to process the jar file and then output the processed jar file to the output directory. MethodTraceUtils.traceJar(jarInput, outputFile) }
In fact, the above is to traverse every jar file to process, then how to deal with the specific?
public static void traceJar(JarInput jarInput, File outputFile) { def jar = jarInput.file LogUtils.i("Processing jar:" + jarInput.name) //Temporary location of jar package decompression def tmpDir = outputFile.parentFile.absolutePath + File.separator + outputFile .name.replace(".jar", File.separator) def tmpFile = new File(tmpDir) tmpFile.mkdirs() //Unzip to temporary directory first MyZipUtils.unzip(jar.absolutePath, tmpFile.absolutePath) //Collect all decompressed files def allFiles = new ArrayList() collectFiles(tmpFile, allFiles) allFiles.each { if (isNeedTraceClass(it)) { //Name the processed file as its original name - new form def tracedFile = new File(tmpFile.absolutePath + "-new") //To modify a single class file traceFile(it, tracedFile) //Replace the original file with a new one after processing it.delete() tracedFile.renameTo(it) } } MyZipUtils.zip(tmpFile.absolutePath, outputFile.absolutePath) tmpFile.deleteDir() }
jar files and ordinary class files know more about the compression process. After decompression, we can process them one by one according to the ordinary class files. Finally, we can compress the class folder after decompression to the output directory. Here, we should delete the decompression directory generated in the middle.
2.5.4 Summary
After the above steps, we can say that we have successfully implemented a gradle plug-in, which can intercept all the class files and modify them. We have successfully achieved AOP programming. Of course, if we modify the class files, this is not within the scope of this article, you can find ASM and other technologies by yourself.
3. Make plug-ins configurable
Now that our plug-in has been developed, is that enough? For example, what if you don't want to process third-party jar packages (copy them directly without handling them)? Or maybe I just want to deal with third-party jar packages at some point, and sometimes I don't want to, when we have to make our plug-ins configurable. Simply, there are two steps:
3.1 Define configuration classes
class MethodTraceExtension { /** * Whether to track third-party dependent methods to execute data */ boolean traceThirdLibrary = false boolean getTraceThirdLibrary() { return traceThirdLibrary void setTraceThirdLibrary(boolean traceThirdLibrary) { this.traceThirdLibrary = traceThirdLibrary } }
3.2 Registration Configuration
Here we need to register in the custom Plugin
class MethodTracePlugin implements Plugin<Project> { @Override void apply(Project project) { project.getExtensions() .create("methodTrace", MethodTraceExtension.class) ...... } }
After registration, we can configure the following configuration in the build file that introduced the plug-in
apply plugin: 'com.jianglei.method-tracer' ...... methodTrace{ traceThirdLibrary = false }
3.3 Access Configuration
Getting the configuration is simple, just use the following code:
//Get configuration information MethodTraceExtension extension = project.getExtensions().findByType(MethodTraceExtension.class)
The question is when do you get this configuration information? At first, I registered for this configuration and went directly to get:
@Override void apply(Project project) { //Registration Configuration project.getExtensions() .create("methodTrace", MethodTraceExtension.class) //Get configuration information MethodTraceExtension extension = project.getExtensions().findByType(MethodTraceExtension.class) ...... }
I want to get the configuration here and pass it into Transform, which is actually undesirable. The time for the application method to be invoked here is
When the application plugin code is called, at this time, our configuration code in build.gradle is not yet called, so we can not get the configuration we want, and we get the default values.
So how on earth should we get it? In fact, we just need to get it in the transform() method, at which time the code configured in build.gradle has been executed:
@Override void transform(TransformInvocation transformInvocation) throws TransformException, InterruptedException, IOException { //Get configuration information MethodTraceExtension extension = project.getExtensions().findByType(MethodTraceExtension.class) ...... }
4. Optimizing
Now, we have a configurable plug-in to modify all the class files. We have completed the functional requirements, but is the performance enough?
4.1 Should gradle plug-ins be introduced in application module or library module?
At present, our plug-ins are directly introduced in the application module, so how to do with so many modules? Should each module be introduced? Can it be introduced only in the main module? Should it be introduced only in the main module?
4.1.1 is only introduced in the main module
We know that butterknife needs to be introduced in every module. In fact, for multiple modules, we can only introduce plug-ins in the main application module. Here, we should pay attention to the getScopes() method in Transform:
@Override Set<? super QualifiedContent.Scope> getScopes() { //This time, only com.android.application plug-ins are allowed in the main module (build.gradle) //So we need to modify all module s. return TransformManager.SCOPE_FULL_PROJECT }
Here SCOPE_FULL_PROJECT is actually like this:
SCOPE_FULL_PROJECT = Sets.immutableEnumSet(Scope.PROJECT, new Scope[]{Scope.SUB_PROJECTS, Scope.EXTERNAL_LIBRARIES});
It shows that the modules processed here include this module, sub-module and third-party jar package, so that we can process all class files in the main module, so we can only introduce them in the main module. In this way, all sub-modules will be input in the form of jar package.
4.1.2 is introduced in each module
So what if you want to introduce each module?
First of all, the registration method should be modified:
@Override void apply(Project project) { project.getExtensions() .create("methodTrace", MethodTraceExtension.class) def extension = project.getExtensions().findByType(AppExtension.class) def isForApplication = true if (extension == null) { //Describe the current use in library extension = project.getExtensions().findByType(LibraryExtension.class) isForApplication = false } extension.registerTransform(new MethodTraceTransform(project,isForApplication)) }
The key is that we record in Transform whether it is currently applied to the main module or the sub-module.
In this mode, each module will execute its own transform() method, so the getScopes() method here needs some modifications:
@Override Set<? super QualifiedContent.Scope> getScopes() { def scopes = new HashSet() scopes.add(QualifiedContent.Scope.PROJECT) if (isForApplication) { //Add this to the application module to handle third-party jar packages scopes.add(QualifiedContent.Scope.EXTERNAL_LIBRARIES) } return scopes }
In the case of the main module, the third-party jar package should be processed extra, and the sub-module only needs to process its own project code.
In fact, through experiments, all the dependent third-party jar packages of sub-modules will only be input in the main module, in other words, sub-modules will never be able to handle third-party jar packages.
4.1.3 Summary
Both ways are possible, so which one should we choose? Is there any basis for choosing? From the introduction above, we can see that it is not, and it seems a little more complicated to write in the way of all sub-modules. Should we choose to introduce plug-ins only in the main module? In fact, otherwise, the biggest difference will be discussed below, when the natural outcome.
4.2 How to Incrementally Compile
With the introduction above, it's no longer a problem to complete a plug-in, but here's a problem. Every time we compile, the transform() method executes. We traverse all the class files, decompress all the jar files, and then recompress them into all the jar files. In fact, one compilation may only change. A class file, can we only modify this class file? gradle actually provides the method.
4.2.1 Incremental mechanism of gradle transformation
transform-api classifies input files into two categories:
- DirectoryInput, which wraps a class file corresponding to the source code, looks like this:
public interface DirectoryInput extends QualifiedContent { Map<File, Status> getChangedFiles(); }
In other words, we can get the changed class file in the following way:
input.directoryInputs.each { directoryInput -> directoryInput.changedFiles.each{changeFileEntry-> def status = changeFileEntry.value; } }
In this way, we can traverse all the changed files and get the status of each changed file. There are four kinds:
public enum Status { NOTCHANGED, ADDED, CHANGED, REMOVED; private Status() { } }
Recompiling directory.changedFiles after the first compilation or clean is empty and needs to be differentiated
After testing, delete a java file, the corresponding class file input will not appear REMOVED status, that is, the deleted file can not be retrieved from the changeFiles
- JarInput differs from DirectoryInput in that JarInput can only retrieve states, but also has four states:
public interface JarInput extends QualifiedContent { Status getStatus(); }
That is to say, if we want incremental compilation, we should deal with all jar packages in non-Status.NOTCHANGED state. If we remove a dependency, the jar package will never be entered again, and of course, there will be no jar packages in Status.REMOVED state.
Problems to be solved in incremental compilation of 4.2.2
With the above understanding of the incremental mechanism of gradle transform ation, I believe that everyone has a basic understanding of how to support incremental compilation, but there are still many problems to be solved to develop a robust plug-in that supports incremental compilation.
4.2.2.1 How to distinguish between uncompiled and unmodified
As mentioned earlier, for DirectoryInput, Directory.changedFiles are empty when it is not compiled or recompiled after cleaning, and empty when it is not modified. In the former state, we need to deal with all files, and in the latter state, we should not deal with any files. Similarly, JarInput also faces this problem, and it is very simple to solve it. In this paper, we present a simple scheme, which generates a tag file at the first compilation. If the tag file is empty at the next compilation, we can judge whether the tag file exists, that is, it is not modified, or it is recompiled after the first compilation or clean compilation. Of course, the last compilation will also have file output, we can directly take any output file to do this markup file.
4.2.2.2 How to Solve Incremental Compile-Time Packet Repetition
Generally speaking, if we rely on a third-party jar package, for example:
implementation "commons-io:commons-io:2.4"
The first compilation generates a file in the compiled output directory, such as:
/home/jianglei/AndroidStudioProjects/ASMStudy/app/build/intermediates/transforms/MethodTrace/debug/32.jar
Now we annotate the introduction of this package and recompile it. As we mentioned before, after deleting a reference to a jar package, we can't receive any information, and we can't process this package because it won't be input at all. Naturally, the 32.jar is still there. At this time, we are reintroducing the package we just received. Remove dependencies, and the generated files become:
/home/jianglei/AndroidStudioProjects/ASMStudy/app/build/intermediates/transforms/MethodTrace/debug/33.jar
At this time, the problem comes. 32.jar and 33.jar are actually jar packages. Class conflicts occur naturally when compiling, and this conflict is awkward and difficult to investigate, because there is no problem with gradle files. The simplest way is to recompile after cleaning. This problem naturally does not exist, but the general developers. There is no such awareness, it is too troublesome to do so. It is normal to delete a dependency and reintroduce it. Why do we have to clean first?
Now let's look at the solution:
The solution is simple. If we can find out which jar packages have been deleted during this compilation, can we solve the conflict problem by manually deleting the output file of the last compilation of the jar packages? So we can record for ourselves which jar packages are involved in the compilation at each compilation and where they are exported, as follows:
{ "commons-io:commons-io:2.4": "/home/jianglei/AndroidStudioProjects/ASMStudy/app/build/intermediates/transforms/MethodTrace/debug/5.jar", ...... }
Then when compiling this time, we read the last file and compare the jar packages that participated in the compilation twice. If there is deletion, we can delete the corresponding output file of the jar packages ourselves.
4.2.2.4 How to Judge Configuration File Change
Unlike the above class file change or jar change, configuration file change transform can't get any additional information, but you can't help it. For example, the last configuration file definition is as follows:
methodTrace{ traceThirdLibrary = false }
After compilation, the third-party jar package will not be processed naturally, but now it is changed to false, at this time, all the results of the last compilation will be restored, because this time the third-party jar package needs to be processed. The solution is also very simple. Since gradle did not inform us of the change of configuration file, we recorded the last configuration file ourselves. Compared with this compilation, if the change of configuration file is all over again, then the recorded file becomes like this:
{ "extension": { "traceThirdLibrary": true }, "jarMap": { "commons-io:commons-io:2.4": "/home/jianglei/AndroidStudioProjects/ASMStudy/app/build/intermediates/transforms/MethodTrace/debug/5.jar", ...... } }
There's a problem that hasn't been solved. Even if you compare the configuration files with or without changes, these codes are written in the transform() method. If this compilation only modifies the configuration files and nothing is changed, gradle thinks you haven't changed anything, and you don't call the transform() method directly, that means that. Given that incremental compilation of configuration files does not work, there is no good solution for the time being, only to re-CLEAN, or modify other java files, can trigger recompilation. If you have a better solution, I hope to point it out.
5. Search for missing and make up for missing
Before that, we still had a problem: should gradle plug-ins be introduced only in the main module or in all modules? In my opinion, the key point to measure is the compilation speed. If only introduced in the main module, the sub-module is actually handled as an input file in the form of a jar package, so that we can modify only one file in the sub-module. We all need to decompress the whole jar and then process all the class files in the jar. Finally, we have to compress it once and do more useless work; if we put it in all modules, we only need to deal with the changed class files, which can save a lot of time, so I recommend it in all modules.
6. Summary
With the above knowledge, I believe you should be able to develop a robust plug-in that supports incremental compilation, and then you can use bytecode stuffing technology to do whatever you want. You can click here for the above source code: https://github.com/FamliarMan/ASMStudy Of course, these are all my own speculation, the Internet does not seem to find relevant information, if there are errors, please correct, not very grateful!
Author: Daxian with low EQ
Link: https://www.jianshu.com/p/d84032b46b56
Source: Brief Book
The copyright of the brief book belongs to the author. For any form of reprinting, please contact the author for authorization and indicate the source.