Old GC problems caused by Groovy scripts

Keywords: Web Development jvm

Recently, a system was launched, and Groovy script was used for authentication. The example code is as follows

ScriptEngineManager factory = new ScriptEngineManager();
ScriptEngine engine = factory.getEngineByName("groovy");

String function = String.format("def getTargetParamValue(%s) {return \"%s\"}", "o", "$o");
engine.eval(function);
Invocable invocable = (Invocable) engine;

Object result = invocable.invokeFunction("getTargetParamValue", "test-string");
System.out.println(result);        

This code defines a Groovy method that returns the corresponding value according to the passed parameter.

This code is executed frequently due to the high traffic in the production environment. The test code is as follows

public class ScriptEngineTest {

    public static void main(String[] args) {
        ScriptEngineManager factory = new ScriptEngineManager();
        ScriptEngine engine = factory.getEngineByName("groovy");
        //Change to dead cycle during test
        for (int i = 0;; i++) {
            try {

                String function = String.format("def getTargetParamValue(%s) {return \"%s\"}", "o", "$o");
                engine.eval(function);
                Invocable invocable = (Invocable) engine;

                Object result = invocable.invokeFunction("getTargetParamValue", "test-string");
                System.out.println(result);

                TimeUnit.MICROSECONDS.sleep(100);

                System.out.println(new Date().toLocaleString());

            } catch (Exception e) {
                String errorMsg = String.format("Abnormal!%s", e.getMessage());
                System.out.println(errorMsg);
            }

        }
    }
}

Simulate the production environment 10 times per second. Observing the JVM through VusualVM

  • CPU usage. It can be seen that the CPU usage will increase significantly each time the heap memory is expanded

  • Heap memory usage

  • Metaface usage has been increasing

  • The total loaded classes has been increasing

  • thread

  • dump memory

It can be seen that the Groovy method generated in each cycle has not been released after the method execution is completed, resulting in the increasing usage of the metasface, and finally supporting the JVM

To solve the above problems, the solution is to cache the generated method each time, and take it from the cache the next time you want to execute it.

private final ConcurrentHashMap<String, Invocable> concurrentHashMap = new ConcurrentHashMap<>();

private Object getInvokeResult(Object targetParam, String paramName, String expression) throws Exception {
    //targetParamClassName="com.umgsai.web.home.vo.NodeVO"
    String targetParamClassName = targetParam.getClass().getName();
    //expression="$nodeVO.bizOwner"
    //paramName="nodeVO"
    String functionKey = String.format("%s_%s_%s", targetParamClassName, paramName, expression);
    functionKey = StringUtil.replaceChars(functionKey, "$", "");
    functionKey = StringUtil.replaceChars(functionKey, ".", "_");
    //functionKey is the name of the method and the key of concurrentHashMap. Special characters need to be removed here

    Invocable invocable = concurrentHashMap.get(functionKey);
    if (invocable != null) {
        //If there is one in the cache, call directly
        return invocable.invokeFunction(functionKey, targetParam);
    }

   //If not, generate the method and save it to the concurrentHashMap
    synchronized (lock) {
        invocable = concurrentHashMap.get(functionKey);
        if (invocable == null) {
            String function = String.format("def %s(%s) {return \"%s\"}", functionKey, paramName, expression);
            engine.eval(function);
            invocable = (Invocable) engine;
            concurrentHashMap.put(functionKey, invocable);
            if (log.isInfoEnabled()) {
                String msg = String.format("Create new Groovy function, functionKey=%s, paramName=%s, expression=%s",
                        functionKey,
                        paramName, expression);
                log.info(msg);
            }
        }
    }

    if (log.isInfoEnabled()) {
        log.info(String.format("Groovy function concurrentHashMap.size=%d", concurrentHashMap.size()));
    }

    return invocable.invokeFunction(functionKey, targetParam);
}

Reference resources http://www.zgxue.com/120/1204001.html
https://www.cnblogs.com/fourspirit/p/4332154.html
https://www.jianshu.com/p/b1a46cc02377

Reprint please indicate the source https://www.cnblogs.com/umgsai/p/10742271.html

Posted by joe2 on Mon, 25 Nov 2019 12:20:48 -0800