Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded

0
votes

I can't run my process. It gives the following exception: "Exception in thread "main" java.lang.OutOfMemoryError: Java heap space"

java -Xms32m -Xmx516m FilteringSNP_genus Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:515) at java.lang.StringBuffer.append(StringBuffer.java:306) at java.io.BufferedReader.readLine(BufferedReader.java:345) at java.io.BufferedReader.readLine(BufferedReader.java:362) at FilteringSNP_genus.main(FilteringSNP_genus.java:65)

I have tried different memory usage configurations like:

java -Xms32m -Xmx1024m FilteringSNP_genus

but it has not worked, and increasing the -XmxVALUE has given a GC overheadlimit exceeded exception:

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.lang.String.substring(String.java:1940) at java.util.StringTokenizer.nextToken(StringTokenizer.java:335) at FilteringSNP_genus.main(FilteringSNP_genus.java:77)

Could anyone provide some clue to fix this?

Thanks

2
javaout-of-memory
Some more info would be appreciate: mostly what does your process do (it is a "Hello world" gone bad or is it really a complex system that may need all of that memory) – SJuan76
Can you post code around FilteringSNP_genus.java:65 – Amir Afghani

2 Answers

7
votes

I'd hazard a guess that you're reading from a file or a socket and using readLine() for convenience without considering the implications. Try reading into a char[] buffer instead.

Alternately, you're reading lines in and storing hard references to them in memory, so you obviously will run out of room once you read enough.

As for the GC overhead error, according to an Oracle JVM article: "The parallel collector will throw an OutOfMemoryError if too much time is being spent in garbage collection: if more than 98% of the total time is spent in garbage collection and less than 2% of the heap is recovered, an OutOfMemoryError will be thrown."

1
votes

I have found the GC will give up if it takes too long to find memory, even if the memory is actually there. The simplest problem is when the memory is mostly virtual, which is vastly slower than real memory. Also if memory is too fragmented, the GC can take time to find the space it needs. And if you are allocating big chunks of memory, that can make the situation much worse. A problem like this can be intermittent, working fine when the GC has had time to clean house and keep everything organized and failing when it is overloaded. My guess is that in your case you either have a paging problem or you are using too much of available memory in chunks that are too big.

Solutions: get more real memory (if paging is the problem). Use less memory. Use smaller pieces of memory. Arrays are the fastest way to crunch numbers, but data structures with pointers make life easier for the GC. If you can figure a way to use smaller arrays (or no arrays), do that.

It ought to be possible to get a proper 64-bit system (computer and JVM) with 8GB or more of memory so you can ignore and forget this problem, but I've yet to hear of anybody doing that. (And memory use expands to fill the memory available...)