stay Lombok is often used, but do you know how it works? , and Lombok is often used, but do you know how it works? (two) In the two articles, the underlying principle of Lombok is introduced. In fact, it is summed up as a sentence that is realized by changing the abstract syntax tree during compilation. The above two articles have talked about the knowledge of abstract syntax tree. If there is something unclear, you can have a look.

All the code involved in this article is on github

All the code involved in this article is on github

All the code involved in this article is on github

There are not many API documents about how to modify the abstract syntax tree of Java on the Internet, so this article records the relevant knowledge points for later reference.

JCTree introduction

JCTree is the base class of syntax tree elements, which contains an important field pos, which is used to indicate the position of the current syntax tree node (JCTree) in the syntax tree. Therefore, we can not directly use the new keyword to create the syntax tree node, even if it is created, it has no meaning. In addition, the data structure and data processing are decoupled in combination with the visitor mode. Some source codes are as follows:

public abstract class JCTree implements Tree, Cloneable, DiagnosticPosition {

    public int pos = -1;

    ...

    public abstract void accept(JCTree.Visitor visitor);

    ...
}

We can see that JCTree is an abstract class. Here we focus on several subclasses of JCTree

1. JCStatement: declare the syntax tree node. Common subclasses are as follows
   • JCBlock: statement block syntax tree node
   • JCReturn: return statement syntax tree node
   • JCClassDecl: class definition syntax tree node
   • JCVariableDecl: field / variable definition syntax tree node
2. JCMethodDecl: method definition syntax tree node
3. JCModifiers: access flag syntax tree node
4. JCExpression: an expression syntax tree node. Common subclasses are as follows
   • JCAssign: syntax tree node of assignment statement
   • JCIdent: identifier syntax tree node, which can be variable, type, keyword, etc

TreeMaker introduction

TreeMaker is used to create a series of syntax tree nodes. We said above that creating a JCTree cannot be created directly by using the new keyword. Therefore, Java provides us with a tool, namely TreeMaker, which will set the pos field for the JCTree object we create at the time of creation. Therefore, the context sensitive TreeMaker object must be used to create the syntax tree node.

For specific API introduction, please refer to, TreeMakerAPI Next, I will focus on several commonly used methods.

TreeMaker.Modifiers

The TreeMaker.Modifiers method is used to create an access flag syntax tree node (JCModifiers). The source code is as follows

public JCModifiers Modifiers(long flags) {
    return Modifiers(flags, List.< JCAnnotation >nil());
}

public JCModifiers Modifiers(long flags,
    List<JCAnnotation> annotations) {
        JCModifiers tree = new JCModifiers(flags, annotations);
        boolean noFlags = (flags & (Flags.ModifierFlags | Flags.ANNOTATION)) == 0;
        tree.pos = (noFlags && annotations.isEmpty()) ? Position.NOPOS : pos;
        return tree;
}

1. Flags: access flags
2. annotations: annotation list

The flags can be represented by the enumeration class com.sun.tools.javac.code.Flags. For example, we can use this to generate the following access flags.

treeMaker.Modifiers(Flags.PUBLIC + Flags.STATIC + Flags.FINAL);

public static final

TreeMaker.ClassDef

TreeMaker.ClassDef is used to create a class definition syntax tree node (JCClassDecl). The source code is as follows:

public JCClassDecl ClassDef(JCModifiers mods,
    Name name,
    List<JCTypeParameter> typarams,
    JCExpression extending,
    List<JCExpression> implementing,
    List<JCTree> defs) {
        JCClassDecl tree = new JCClassDecl(mods,
                                     name,
                                     typarams,
                                     extending,
                                     implementing,
                                     defs,
                                     null);
        tree.pos = pos;
        return tree;
}

1. mods: access flag, which can be created through TreeMaker.Modifiers
2. Name: class name
3. typarams: generic parameter list
4. extending: parent class
5. implementing: implemented interface
6. defs: detailed statement of class definition, including definition of fields, methods, etc

TreeMaker.MethodDef

TreeMaker.MethodDef is used to create a method definition syntax tree node (JCMethodDecl). The source code is as follows

public JCMethodDecl MethodDef(JCModifiers mods,
    Name name,
    JCExpression restype,
    List<JCTypeParameter> typarams,
    List<JCVariableDecl> params,
    List<JCExpression> thrown,
    JCBlock body,
    JCExpression defaultValue) {
        JCMethodDecl tree = new JCMethodDecl(mods,
                                       name,
                                       restype,
                                       typarams,
                                       params,
                                       thrown,
                                       body,
                                       defaultValue,
                                       null);
        tree.pos = pos;
        return tree;
}

public JCMethodDecl MethodDef(MethodSymbol m,
    Type mtype,
    JCBlock body) {
        return (JCMethodDecl)
            new JCMethodDecl(
                Modifiers(m.flags(), Annotations(m.getAnnotationMirrors())),
                m.name,
                Type(mtype.getReturnType()),
                TypeParams(mtype.getTypeArguments()),
                Params(mtype.getParameterTypes(), m),
                Types(mtype.getThrownTypes()),
                body,
                null,
                m).setPos(pos).setType(mtype);
}

1. mods: access flag
2. Name: method name
3. restype: return type
4. typarams: generic parameter list
5. params: parameter list
6. Throw: exception declaration list
7. Body: method body
8. defaultValue: the default method (which may be the default in the interface)
9. m: Method symbols
10. mtype: method type. There are many types, including generic parameter type, method parameter type, exception parameter type and return parameter type.

TreeMaker.VarDef

TreeMaker.VarDef is used to create a field / variable definition syntax tree node (JCVariableDecl). The source code is as follows

public JCVariableDecl VarDef(JCModifiers mods,
    Name name,
    JCExpression vartype,
    JCExpression init) {
        JCVariableDecl tree = new JCVariableDecl(mods, name, vartype, init, null);
        tree.pos = pos;
        return tree;
}

public JCVariableDecl VarDef(VarSymbol v,
    JCExpression init) {
        return (JCVariableDecl)
            new JCVariableDecl(
                Modifiers(v.flags(), Annotations(v.getAnnotationMirrors())),
                v.name,
                Type(v.type),
                init,
                v).setPos(pos).setType(v.type);
}

1. mods: access flag
2. Name: parameter name
3. vartype: type
4. init: initialization statement
5. v: Variable symbols

TreeMaker.Ident

TreeMaker.Ident is used to create an identifier syntax tree node (JCIdent). The source code is as follows

public JCIdent Ident(Name name) {
        JCIdent tree = new JCIdent(name, null);
        tree.pos = pos;
        return tree;
}

public JCIdent Ident(Symbol sym) {
        return (JCIdent)new JCIdent((sym.name != names.empty)
                                ? sym.name
                                : sym.flatName(), sym)
            .setPos(pos)
            .setType(sym.type);
}

public JCExpression Ident(JCVariableDecl param) {
        return Ident(param.sym);
}

TreeMaker.Return

TreeMaker.Return is used to create a return statement (JCReturn). The source code is as follows

public JCReturn Return(JCExpression expr) {
        JCReturn tree = new JCReturn(expr);
        tree.pos = pos;
        return tree;
}

TreeMaker.Select

TreeMaker.Select is used to create domain access / method access (here, the method access is only named, and the method call needs to use TreeMaker.Apply) syntax tree node (JCFieldAccess). The source code is as follows

public JCFieldAccess Select(JCExpression selected,
    Name selector) 
{
        JCFieldAccess tree = new JCFieldAccess(selected, selector, null);
        tree.pos = pos;
        return tree;
}

public JCExpression Select(JCExpression base,
    Symbol sym) {
        return new JCFieldAccess(base, sym.name, sym).setPos(pos).setType(sym.type);
}

1. selected: the expression to the left of the. Operator
2. selector: the expression to the right of the. Operator

Here is an example. A Java statement generated by a statement is a two statement

1, TreeMaker.Select(treeMaker.Ident(names.fromString("this")), names.fromString("name"));

2, this.name

TreeMaker.NewClass

TreeMaker.NewClass is used to create a new statement syntax tree node (JCNewClass). The source code is as follows:

public JCNewClass NewClass(JCExpression encl,
    List<JCExpression> typeargs,
    JCExpression clazz,
    List<JCExpression> args,
    JCClassDecl def) {
        JCNewClass tree = new JCNewClass(encl, typeargs, clazz, args, def);
        tree.pos = pos;
        return tree;
}

1. encl: I don't quite understand the meaning of this parameter. I think in many examples, this parameter is set to null
2. typeargs: list of parameter types
3. clazz: type of object to be created
4. args: parameter list
5. def: class definition

TreeMaker.Apply

TreeMaker.Apply is used to create a method call syntax tree node (JCMethodInvocation). The source code is as follows:

public JCMethodInvocation Apply(List<JCExpression> typeargs,
    JCExpression fn,
    List<JCExpression> args) {
        JCMethodInvocation tree = new JCMethodInvocation(typeargs, fn, args);
        tree.pos = pos;
        return tree;
}

1. typeargs: list of parameter types
2. fn: call statement
3. args: parameter list

TreeMaker.Assign

TreeMaker.Assign the user creates the assignment statement syntax tree node (JCAssign). The source code is as follows:

ublic JCAssign Assign(JCExpression lhs,
    JCExpression rhs) {
        JCAssign tree = new JCAssign(lhs, rhs);
        tree.pos = pos;
        return tree;
}

1. lhs: left expression of assignment statement
2. rhs: right expression of assignment statement

TreeMaker.Exec

TreeMaker.Exec is used to create an executable statement syntax tree node (JCExpressionStatement). The source code is as follows:

public JCExpressionStatement Exec(JCExpression expr) {
        JCExpressionStatement tree = new JCExpressionStatement(expr);
        tree.pos = pos;
        return tree;
}

TreeMaker.Block

TreeMaker.Block is used to create the syntax tree node (JCBlock) of the combined statement. The source code is as follows:

public JCBlock Block(long flags,
    List<JCStatement> stats) {
        JCBlock tree = new JCBlock(flags, stats);
        tree.pos = pos;
        return tree;
}

1. Flags: access flags
2. stats: statement list

Introduction to com.sun.tools.javac.util.List

When we operate the abstract syntax tree, sometimes it involves the operation of List, but this List is not the java.util.List we often use, but com.sun.tools.javac.util.List. This List is a chain structure, with head node and tail node, but only the tail node is a List, so it's just for understanding.

public class List<A> extends AbstractCollection<A> implements java.util.List<A> {
    public A head;
    public List<A> tail;
    private static final List<?> EMPTY_LIST = new List<Object>((Object)null, (List)null) {
        public List<Object> setTail(List<Object> var1) {
            throw new UnsupportedOperationException();
        }

        public boolean isEmpty() {
            return true;
        }
    };

    List(A head, List<A> tail) {
        this.tail = tail;
        this.head = head;
    }

    public static <A> List<A> nil() {
        return EMPTY_LIST;
    }

    public List<A> prepend(A var1) {
        return new List(var1, this);
    }

    public List<A> append(A var1) {
        return of(var1).prependList(this);
    }

    public static <A> List<A> of(A var0) {
        return new List(var0, nil());
    }

    public static <A> List<A> of(A var0, A var1) {
        return new List(var0, of(var1));
    }

    public static <A> List<A> of(A var0, A var1, A var2) {
        return new List(var0, of(var1, var2));
    }

    public static <A> List<A> of(A var0, A var1, A var2, A... var3) {
        return new List(var0, new List(var1, new List(var2, from(var3))));
    }

    ...
}

com.sun.tools.javac.util.ListBuffer

Because com.sun.tools.javac.util.List is inconvenient to use, a layer is encapsulated on it. This encapsulation class is ListBuffer. This kind of operation is very similar to the usual java.util.List usage.

public class ListBuffer<A> extends AbstractQueue<A> {

    public static <T> ListBuffer<T> of(T x) {
        ListBuffer<T> lb = new ListBuffer<T>();
        lb.add(x);
        return lb;
    }

    /** The list of elements of this buffer.
     */
    private List<A> elems;

    /** A pointer pointing to the last element of 'elems' containing data,
     *  or null if the list is empty.
     */
    private List<A> last;

    /** The number of element in this buffer.
     */
    private int count;

    /** Has a list been created from this buffer yet?
     */
    private boolean shared;

    /** Create a new initially empty list buffer.
     */
    public ListBuffer() {
        clear();
    }

    /** Append an element to buffer.
     */
    public ListBuffer<A> append(A x) {
        x.getClass(); // null check
        if (shared) copy();
        List<A> newLast = List.<A>of(x);
        if (last != null) {
            last.tail = newLast;
            last = newLast;
        } else {
            elems = last = newLast;
        }
        count++;
        return this;
    }
    ........
}

Introduction to com.sun.tools.javac.util.Names

This is a tool class to create names for us. The names of classes, methods and parameters need to be created through this class. A method often used in it is fromString(), which is generally used as follows.

Names names  = new Names()
names. fromString("setName");

Practical drill

We have learned about how to operate the abstract syntax tree. Next, we will write several real cases to deepen our understanding.

Variable correlation

The parameter we often operate on in a class is a variable. How can we use the characteristics of the abstract syntax tree to operate variables for us? Next, we will do some operations on variables.

Generate variables

For example, generate private String age; for such a variable, use the VarDef method we mentioned above

// For example: private String age;
treeMaker.VarDef(treeMaker.Modifiers(Flags.PRIVATE), names.fromString("age"), treeMaker.Ident(names.fromString("String")), null);

Assign values to variables

For example, do we want to generate private String name = "BuXueWuShu", or use VarDef method

// private String name = "BuXueWuShu"
treeMaker.VarDef(treeMaker.Modifiers(Flags.PRIVATE),names.fromString("name"),treeMaker.Ident(names.fromString("String")),treeMaker.Literal("BuXueWuShu"))

Add two literal quantities

For example, we generate String add = "a" + "b"; borrow the Exec method and Assign method we mentioned above

// add = "a"+"b"
treeMaker.Exec(treeMaker.Assign(treeMaker.Ident(names.fromString("add")),treeMaker.Binary(JCTree.Tag.PLUS,treeMaker.Literal("a"),treeMaker.Literal("b"))))

+ = grammar

For example, if we want to generate add += "test", it's almost as literal as above.

// add+="test"
treeMaker.Exec(treeMaker.Assignop(JCTree.Tag.PLUS_ASG, treeMaker.Ident(names.fromString("add")), treeMaker.Literal("test")))

+ + syntax

For example, you want to generate + + i

treeMaker.Exec(treeMaker.Unary(JCTree.Tag.PREINC,treeMaker.Ident(names.fromString("i"))))

Method correlation

We operate on variables, so basically we need to generate methods, so how to generate and operate methods? Let's demonstrate the operation methods related to methods.

No parameter no return value

We can use the MethodDef method mentioned above to generate

/*
    Method generation without parameter and return value
    public void test(){

    }
 */
// Define method body
ListBuffer<JCTree.JCStatement> testStatement = new ListBuffer<>();
JCTree.JCBlock testBody = treeMaker.Block(0, testStatement.toList());
    
JCTree.JCMethodDecl test = treeMaker.MethodDef(
        treeMaker.Modifiers(Flags.PUBLIC), // Method limit
        names.fromString("test"), // Method name
        treeMaker.Type(new Type.JCVoidType()), // Return type
        com.sun.tools.javac.util.List.nil(),
        com.sun.tools.javac.util.List.nil(),
        com.sun.tools.javac.util.List.nil(),
        testBody,	// Method body
        null
);

Return value with parameter or not

We can use the MethodDef method mentioned above to generate

/*
    Method generation without parameter and return value
    public void test2(String name){
        name = "xxxx";
    }
 */
ListBuffer<JCTree.JCStatement> testStatement2 = new ListBuffer<>();
testStatement2.append(treeMaker.Exec(treeMaker.Assign(treeMaker.Ident(names.fromString("name")),treeMaker.Literal("xxxx"))));
JCTree.JCBlock testBody2 = treeMaker.Block(0, testStatement2.toList());

// Generating input
JCTree.JCVariableDecl param = treeMaker.VarDef(treeMaker.Modifiers(Flags.PARAMETER), names.fromString("name"),treeMaker.Ident(names.fromString("String")), null);
com.sun.tools.javac.util.List<JCTree.JCVariableDecl> parameters = com.sun.tools.javac.util.List.of(param);

JCTree.JCMethodDecl test2 = treeMaker.MethodDef(
        treeMaker.Modifiers(Flags.PUBLIC), // Method limit
        names.fromString("test2"), // Method name
        treeMaker.Type(new Type.JCVoidType()), // Return type
        com.sun.tools.javac.util.List.nil(),
        parameters, // Participation
        com.sun.tools.javac.util.List.nil(),
        testBody2,
        null
);

With parameter and return value

 /*
    With parameter and return value
    public String test3(String name){
       return name;
    }
 */

ListBuffer<JCTree.JCStatement> testStatement3 = new ListBuffer<>();
testStatement3.append(treeMaker.Return(treeMaker.Ident(names.fromString("name"))));
JCTree.JCBlock testBody3 = treeMaker.Block(0, testStatement3.toList());

// Generating input
JCTree.JCVariableDecl param3 = treeMaker.VarDef(treeMaker.Modifiers(Flags.PARAMETER), names.fromString("name"),treeMaker.Ident(names.fromString("String")), null);
com.sun.tools.javac.util.List<JCTree.JCVariableDecl> parameters3 = com.sun.tools.javac.util.List.of(param3);

JCTree.JCMethodDecl test3 = treeMaker.MethodDef(
        treeMaker.Modifiers(Flags.PUBLIC), // Method limit
        names.fromString("test4"), // Method name
        treeMaker.Ident(names.fromString("String")), // Return type
        com.sun.tools.javac.util.List.nil(),
        parameters3, // Participation
        com.sun.tools.javac.util.List.nil(),
        testBody3,
        null
);

Especial

We have learned how to define parameters and methods. In fact, there are many statements to learn, such as how to generate new statements, how to generate statements for method calls, and how to generate if statements. Next we will learn some special grammar.

new an object

// Create a new statement, CombatJCTreeMain combatJCTreeMain = new CombatJCTreeMain();
JCTree.JCNewClass combatJCTreeMain = treeMaker.NewClass(
        null,
        com.sun.tools.javac.util.List.nil(),
        treeMaker.Ident(names.fromString("CombatJCTreeMain")),
        com.sun.tools.javac.util.List.nil(),
        null
);
JCTree.JCVariableDecl jcVariableDecl1 = treeMaker.VarDef(
        treeMaker.Modifiers(Flags.PARAMETER),
        names.fromString("combatJCTreeMain"),
        treeMaker.Ident(names.fromString("CombatJCTreeMain")),
        combatJCTreeMain
);

Method call (no arguments)

JCTree.JCExpressionStatement exec = treeMaker.Exec(
        treeMaker.Apply(
                com.sun.tools.javac.util.List.nil(),
                treeMaker.Select(
                        treeMaker.Ident(names.fromString("combatJCTreeMain")), // . left
                        names.fromString("test") // . content on the right
                ),
                com.sun.tools.javac.util.List.nil()
        )
);

Method call (with parameters)

// Create a method call combatJCTreeMain.test2("hello world!");
JCTree.JCExpressionStatement exec2 = treeMaker.Exec(
        treeMaker.Apply(
                com.sun.tools.javac.util.List.nil(),
                treeMaker.Select(
                        treeMaker.Ident(names.fromString("combatJCTreeMain")), // . left
                        names.fromString("test2") // . content on the right
                ),
                com.sun.tools.javac.util.List.of(treeMaker.Literal("hello world!")) // Contents of method
        )
);

if statement

/*
    Create an if statement
    if("BuXueWuShu".equals(name)){
        add = "a" + "b";
    }else{
        add += "test";
    }
 */
// "BuXueWuShu".equals(name)
JCTree.JCMethodInvocation apply = treeMaker.Apply(
        com.sun.tools.javac.util.List.nil(),
        treeMaker.Select(
                treeMaker.Literal("BuXueWuShu"), // . left
                names.fromString("equals") // . content on the right
        ),
        com.sun.tools.javac.util.List.of(treeMaker.Ident(names.fromString("name")))
);
//  add = "a" + "b"
JCTree.JCExpressionStatement exec3 = treeMaker.Exec(treeMaker.Assign(treeMaker.Ident(names.fromString("add")), treeMaker.Binary(JCTree.Tag.PLUS, treeMaker.Literal("a"), treeMaker.Literal("b"))));
//  add += "test"
JCTree.JCExpressionStatement exec1 = treeMaker.Exec(treeMaker.Assignop(JCTree.Tag.PLUS_ASG, treeMaker.Ident(names.fromString("add")), treeMaker.Literal("test")));

JCTree.JCIf anIf = treeMaker.If(
        apply, // Judgment statement in if statement
        exec3, // Conditional statement
        exec1  // Statement with invalid condition
);

Source address

summary

The paper must be light at last, and we must do it. I hope you can test it on your own after reading this article. Set a few parameters by yourself, and learn to generate get and set methods by Lombok. Although this knowledge is not used in daily development, if it is used, others will not meet you, and the gap is gradually widened. All the code involved in this article is on github , after pulling it down, you can see the global search of the CombatJCTreeProcessor class.