sorry, first buy a pass, haha.
Everyone is working hard to upgrade, enjoying all kinds of conveniences brought by JDK8, but sometimes the upgrade is not so smooth? For example, today's question. We all know that the biggest change in JDK8's memory model is to abandon Perm and usher in the era of Metaspace. If you are not familiar with Metaspace, I have written an article about Metaspace before. If you are interested, you can read my previous article.
We used to add parameters like - XX:PermSize=256M -XX:MaxPermSize=256M to JDK8, because Perm is gone, if there are any other parameters JVM will throw some warning information, so we will upgrade the parameters, such as directly changing PermSize to MetaspaceSize, MaxPermSize to MaxMetaSize, but later on we will upgrade the parameters. You will find a problem, and you will often see Metaspace OutOfMemory exceptions or GC logs prompting Full caused by Metaspace GC, at this time we have to adjust Max Metaspace Size and Metaspace Size to 512M or larger. Fortunately, we find that the problem has been solved, and there is no OOM after that, but sometimes unfortunately, OOM still appears. At this point, people are very confused, the code has not changed at all, but loading classes seems to require more memory?
I didn't think about it carefully before. When I came across this kind of OOM problem, I thought it was mainly Metaspace memory fragmentation. Because I helped people solve similar problems before. They built thousands of class loaders. It was really caused by Metsapce fragmentation, because Metaspace can't compress. The solution was to increase Metaspace Si. Ze and Max Metaspace Size, and set them equal. This time, the problem is not the same. There are not many classes loaded, but the OutOfMemory exception of Metaspace is thrown, and Full GC has been going on, and from jstat point of view, the use of Metaspace's GC is basically unchanged before and after, that is, GC basically does not recycle any memory before and after.
Through our memory analysis tools, we can see that the same class loader loads the same class many times and there are multiple class instances in memory. We can also verify this by adding - verbose:class parameters. To output the following log, it will only be output if a class is constantly defined. So we want to build this scenario, so we simply write the demo. To verify
[Loaded ResponseVO$JaxbAccessorM_getDescription_setDescription_java_lang_String from __JVM_DefineClass__]
[Loaded ResponseVO$JaxbAccessorM_getDescription_setDescription_java_lang_String from __JVM_DefineClass__]
[Loaded ResponseVO$JaxbAccessorM_getDescription_setDescription_java_lang_String from __JVM_DefineClass__]
Demo
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.lang.reflect.Method;
/**
* Created by nijiaben on 2017/3/7.
*/
public class B {
public static void main(String args[]) throws Throwable {
Method defineClass = ClassLoader.class.getDeclaredMethod("defineClass",
new Class[]{String.class, byte[].class, int.class, int.class});
defineClass.setAccessible(true);
File file = new File("/Users/nijiaben/BBBB.class");
byte[] bcs = new byte[(int) file.length()];
FileInputStream in = null;
try {
in = new FileInputStream(file);
while ((in.read(bcs)) != -1) {
}
} catch (Exception e) {
} finally {
if (in != null) {
try {
in.close();
} catch (IOException e) {
}
}
}
while (true) {
try {
defineClass.invoke(B.class.getClassLoader(), new Object[]{"BBBB", bcs, 0, bcs.length});
} catch (Throwable e) {
}
}
}
}
The code is very simple, that is to call the defineClass method of ClassLoader directly by reflection to redefine a class.
The JVM parameters for running under JDK7 are set as follows:
-Xmx100M -Xms100M -verbose:class -XX:+PrintGCDetails -XX:MaxPermSize=50M -XX:PermSize=50M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
The JVM parameters for running down JDK8 are:
-Xmx100M -Xms100M -verbose:class -XX:+PrintGCDetails -XX:MaxMetaspaceSize=50M -XX:MetaspaceSize=50M -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
You can see the difference between JDK7 and JDK8 through jstat-gcutil < PID > 1000. As a result, you will find that the use rate of Perm under JDK7 is changing with the development of FGC, but the use rate of Metsapce has not changed before and after GC.
Results under JDK7:
[Full GC[CMS: 0K->346K(68288K), 0.0267620 secs] 12607K->346K(99008K), [CMS Perm : 51199K->3122K(51200K)], 0.0269490 secs] [Times: user=0.03 sys=0.00, real=0.03 secs]
Results under JDK8:
[Full GC (Metadata GC Threshold) [CMS: 5308K->5308K(68288K), 0.0397720 secs] 5844K->5308K(99008K), [Metaspace: 49585K->49585K(1081344K)], 0.0398189 secs] [Times: user=0.04 sys=0.00, real=0.04 secs]
[Full GC (Last ditch collection) [CMS: 5308K->5308K(68288K), 0.0343949 secs] 5308K->5308K(99008K), [Metaspace: 49585K->49585K(1081344K)], 0.0344473 secs] [Times: user=0.03 sys=0
Duplicate Class Definition
Duplicate class definitions, as has been proved in Demo above. When we call ClassLoader's defineClass method many times, even if the same class loader loads the same class file, we will create multiple Klass structures in the corresponding Perm or Metaspace in JVM. Of course, we don't call them directly in general, but reflection provides such powerful power. Force, some people will still use this way of writing, in fact, I think people who use it directly do not fully understand the implementation mechanism of class loading, including the scene of this problem is actually absorbed into the JDK jaxp/jaxws, for example, it exists such code to achieve com.sun.xml.bind.v2.runtime.reflect.opt.Injector inject method there is a direct call situation:private synchronized Class inject(String className, byte[] image)
{
if (!this.loadable) {
return null;
}
Class c = (Class)this.classes.get(className);
if (c == null)
{
try
{
c = (Class)defineClass.invoke(this.parent, new Object[] { className.replace('/', '.'), image, Integer.valueOf(0), Integer.valueOf(image.length) });
resolveClass.invoke(this.parent, new Object[] { c });
}
catch (IllegalAccessException e)
{
logger.log(Level.FINE, "Unable to inject " + className, e);
return null;
}
catch (InvocationTargetException e)
{
logger.log(Level.FINE, "Unable to inject " + className, e);
return null;
}
catch (SecurityException e)
{
logger.log(Level.FINE, "Unable to inject " + className, e);
return null;
}
catch (LinkageError e)
{
logger.log(Level.FINE, "Unable to inject " + className, e);
return null;
}
this.classes.put(className, c);
}
return c;
}
However, this implementation has changed since version 2.2.2.
private Class inject(String className, byte[] image)
{
...
c = (Class)findLoadedClass.invoke(this.parent, new Object[] { className.replace('/', '.') });
...
if (c == null)
{
c = (Class)defineClass.invoke(this.parent, new Object[] { className.replace('/', '.'), image, Integer.valueOf(0), Integer.valueOf(image.length) });
resolveClass.invoke(this.parent, new Object[] { c })
...
}
}
So you still use the jaxb-impl-2.2.2 version below, please note that upgrade to JDK8 may have problems described in this article.
The impact of duplicate class definitions
What harm does duplicate class definitions do? Normal class loading will first go through the cache search to see if there is a corresponding class, if there is a direct return, if there is no definition, if the method defined by the class is called directly, several temporary class structure instances will be created in the JVM. These related structures exist in Perm or Metaspace, that is to say, they will consume the memory of Perm or Metaspace, but When these classes are defined, they will eventually be checked for constraints, and if they are found to have been defined, they will throw Linkage Error's exception directly.
void SystemDictionary::check_constraints(int d_index, unsigned int d_hash,
instanceKlassHandle k,
Handle class_loader, bool defining,
TRAPS) {
const char *linkage_error = NULL;
{
Symbol* name = k->name();
ClassLoaderData *loader_data = class_loader_data(class_loader);
MutexLocker mu(SystemDictionary_lock, THREAD);
Klass* check = find_class(d_index, d_hash, name, loader_data);
if (check != (Klass*)NULL) {
// if different InstanceKlass - duplicate class definition,
// else - ok, class loaded by a different thread in parallel,
// we should only have found it if it was done loading and ok to use
// system dictionary only holds instance classes, placeholders
// also holds array classes
assert(check->oop_is_instance(), "noninstance in systemdictionary");
if ((defining == true) || (k() != check)) {
linkage_error = "loader (instance of %s): attempted duplicate class "
"definition for name: \"%s\"";
} else {
return;
}
}
...
}So these temporary structures can only be recycled while waiting for GC, because they are not reachable, so they will be recycled when GC. The question arises, why can they be recycled normally under Perm, but not in Metaspace?
Differences in class unloading between Perm and Metaspace
Here I mainly take CMS GC, the most commonly used GC algorithm, as an example.
Under JDK7 CMS, Perm's structure is actually the same as Old's memory structure. If Perm is not enough, we will do Full GC once. By default, Full GC will compress all generations, including Perm. In this way, according to the accessibility of objects, any class will only be bound to a live class loader, and these classes will be marked alive at the markup stage. They are then computed and moved to compress new addresses, which were previously recycled during compression because they were not associated with any living class loader, such as a Hashtable structure called System Dictionary.
oid GenMarkSweep::mark_sweep_phase4() {
// All pointers are now adjusted, move objects accordingly
// It is imperative that we traverse perm_gen first in phase4. All
// classes must be allocated earlier than their instances, and traversing
// perm_gen first makes sure that all klassOops have moved to their new
// location before any instance does a dispatch through it's klass!
// The ValidateMarkSweep live oops tracking expects us to traverse spaces
// in the same order in phase2, phase3 and phase4. We don't quite do that
// here (perm_gen first rather than last), so we tell the validate code
// to use a higher index (saved from phase2) when verifying perm_gen.
GenCollectedHeap* gch = GenCollectedHeap::heap();
Generation* pg = gch->perm_gen();
GCTraceTime tm("phase 4", PrintGC && Verbose, true, _gc_timer);
trace("4");
VALIDATE_MARK_SWEEP_ONLY(reset_live_oop_tracking(true));
pg->compact();
VALIDATE_MARK_SWEEP_ONLY(reset_live_oop_tracking(false));
GenCompactClosure blk;
gch->generation_iterate(&blk, true);
VALIDATE_MARK_SWEEP_ONLY(compaction_complete());
pg->post_compact(); // Shared spaces verification.
}
Under JDK8, Metaspace is a completely independent and decentralized memory structure, composed of discrete memory. When Metaspace reaches the threshold of triggering GC (related to Max Metaspace Size and Metaspace Size), it will do a Full GC, but this Full GC will not compress Metaspace. The only case of unloading a class is that the corresponding class loader must be dead if the class is loaded. Loaders are alive, so there's no way to uninstall them.
void GenMarkSweep::mark_sweep_phase4() {
// All pointers are now adjusted, move objects accordingly
// It is imperative that we traverse perm_gen first in phase4. All
// classes must be allocated earlier than their instances, and traversing
// perm_gen first makes sure that all Klass*s have moved to their new
// location before any instance does a dispatch through it's klass!
// The ValidateMarkSweep live oops tracking expects us to traverse spaces
// in the same order in phase2, phase3 and phase4. We don't quite do that
// here (perm_gen first rather than last), so we tell the validate code
// to use a higher index (saved from phase2) when verifying perm_gen.
GenCollectedHeap* gch = GenCollectedHeap::heap();
GCTraceTime tm("phase 4", PrintGC && (Verbose || LogCMSParallelFullGC),
true, _gc_timer, _gc_tracer->gc_id());
trace("4");
GenCompactClosure blk;
gch->generation_iterate(&blk, true);
}Prove that compression can recover repetitive classes in Perm
Without looking at the GC source code, what is the way to prove that Perm's recycling under FGC is due to compression and that those duplicate classes are recycled? You can change the test case above and change the last dead cycle.
int i = 0;
while (i++ < 1000) {
try {
defineClass.invoke(B.class.getClassLoader(), new Object[]{"BBBB", bcs, 0, bcs.length});
} catch (Throwable e) {
}
}
System.gc();
Set a breakpoint at System.gc, and then see if Perm usage has changed through jstat - gcutil < PID > 1000. Add - XX:+ ExplicitGCInvokes Concurrent to repeat the above action, and see what the output looks like.