Recently, I wrote a gadget for the convenience of work. The utility is very simple, which is to sift out the parts you want from a json string.

introduce

The background is like this. For the convenience of online debugging, a tool can simulate to launch a data request, and then display the results to the page in the form of json. But the problem is that this data contains a lot of information, thousands of lines at any time (as shown in the figure above), but every time when debugging, you only want to see specific fields in it. Usually, you can only rely on the browser search tool to search line by line, and the fields you want to see may be separated by several screens, which is inefficient or easy to leak. If I want to see the data of JsonArray, I used to copy it, and then use grep to screen out the fields, but I lost the hierarchical information..... If we want to use some fields and columns together for data analysis, it's more difficult. We can only filter records by human flesh...

My tool uses a very simple syntax to identify the hierarchy of the target json and the fields you want in each layer. The syntax is similar to the hierarchical result of yaml. The same reduction is used to identify the same layer. The key words of each layer are the field key you want, which is not case sensitive. For the convenience of use, regular expressions are also supported.
Of course, there are several special rules:
1. If the current level is a jsonArray, the field needs to be marked with a suffix: [] (I may support the range in brackets later).
2. In the first line, you must write a field casually. The purpose of keeping this field is to be afraid of a JsonArray.
3. Currently, it is not allowed to add blank lines, especially between multiple lines, which will cause problems in filtering.

For example, you can try demo.

json
  menu
    id
    popup
      menuitem:[]
        value

realization

If you know the json data format, you will know that it is a hierarchical nested structure, and the hierarchical nested structure is actually easy to convert into a tree structure. In fact, all of the json parsers on the market are actually storing these data in a tree structure. After we know that json is a tree structure, do we construct an isomorphic subtree? Each layer of the meaning tree of the isomorphic subtree contains fewer nodes, but some nodes are isomorphic with the nodes of the original tree.

How to construct or describe such an isomorphic tree structure? Here I use a yaml like description, which uses different indents to identify hierarchical relationships.

1
   2
     3
   4
     5
     6

For example, 24 is a child of 1, 3 is a child of 2, and 56 is a child of 4. With description language, the next step is to transform description language into abstract syntax tree. Here I use the recursive descent algorithm in the compilation principle to construct the child nodes of each node in a recursive way.

For convenience, I will first preprocess the syntax description, mainly converting the indentation to the level depth, and then parse recursively. The parsing code is as follows.

public class Node {
    public int type = 0; //jsonObject or jsonArray 
    Map<String, Node> children = new HashMap<>();

    public Node(String[] keys, int[] deeps, int cur) {  //Analytic logic is placed directly in the constructor
        // No child nodes
        if (cur == keys.length - 1 || deeps[cur] >= deeps[cur+1]) {
            this.type = 0; //No child nodes
            return;
        }
        int childDeep = deeps[cur+1];
        for (int i = cur+1; i < keys.length; i++) {
            if (deeps[i] < childDeep) {
                break;
            } else if (deeps[i] > childDeep) {
                continue;
            }
            String key = keys[i];
            Node child = new Node(keys, deeps, i);  // Recursively resolving child nodes 
            if (key.contains(":")) {
                key = key.split(":")[0];
                child.type = 1;  // ArrayList;
            }
            children.put(key, child);
        }
    }
}

After the whole parsing, it is an abstract syntax tree. I use fastjson to parse the json string, which is also a tree hierarchy. Because our newly generated syntax tree and json syntax tree are isomorphic, we can recursively traverse the new syntax tree and abstract syntax tree at the same time, and generate a filtered json string at the same time. In this way, we complete the process of matching and filtering. The code is as follows.

   public Object getSelected(Object object) {
        // No child nodes
        if (children.size() == 0) {
            return object;
        }

        JSONObject res = new JSONObject(true);
        JSONObject json = (JSONObject)object;
        for (Map.Entry<String, Object> entry : json.entrySet()) {
            Node child = getChild(entry.getKey());
            if (child == null) {
                continue;
            }
            // json
            if (child.type == 0) {
                res.put(entry.getKey(), child.getSelected(json.get(entry.getKey())));
            }
            // jsonArray
            if (child.type == 1) {
                JSONArray arr = (JSONArray)entry.getValue();
                JSONArray newArr = new JSONArray();
                for (int i = 0; i < arr.size(); i++) {
                    newArr.add(child.getSelected(arr.getJSONObject(i)));
                }
                res.put(entry.getKey(), newArr);
            }
        }
        return res;
    }

    public Node getChild(String content) {
        for (Map.Entry<String, Node> child : children.entrySet()) {
            // Here I add regular expression matching to make the function of selector more flexible  
            if (content.equalsIgnoreCase(child.getKey()) || Pattern.matches(child.getKey(), content)) {
                return child.getValue();
            }
        }
        return null;
    }

Finally, write a class to encapsulate all the API s.

public class JsonSelector {
    private Node startNode;
    private JsonSelector() {};
    // Compile build syntax tree 
    public static JsonSelector compile(String txt) {
        // Pretreatment  
        txt = txt.replace("\t", "    ");
        String[] arr = txt.split("\n");
        int[] deeps = new int[arr.length];
        String[] keys = new String[arr.length];
        for (int i = 0; i < arr.length; i++) {
            String str = arr[i];
            deeps[i] = getSpaceCnt(str);
            keys[i] = rmSpace(str);
        }
        JsonSelector selector = new JsonSelector();
        selector.startNode = new Node(keys, deeps, 0);
        return selector;
    }

    public String getSelectedString(String jsonStr) {
        JSONObject json = JSONObject.parseObject(jsonStr, Feature.OrderedField);
        JSONObject res = (JSONObject) startNode.getSelected(json);
        return res.toJSONString();
    }

    private static int getSpaceCnt(String str) {
        int cnt = 0;
        for (cnt = 0; cnt < str.length(); cnt++) {
            if (str.charAt(cnt) != ' ') {
                break;
            }
        }
        return cnt;
    }

    private static String rmSpace(String str) {
        String res = str.trim();
        int end = res.length();
        while(end > 0 && res.charAt(end - 1) == ' ') {
            end--;
        }
        return res.substring(0, end);
    }
}