Default Java Heap Size and Memory Issues it can cause

Question

I have an application whose backend is built on Java 8. Recently I've been seeing that the application restarts by itself, because of OOM. I got to know its because of OOM, by looking at the exit status of the Docker Container. There was no Xmx and Xms values configured for the system and as such it should be using the default heap size, which in case of my server which has 4 gigs of memory should be around 512 MB (I read in some SO answers that deafult heap max size is 1/8th the memory present). After setting the Xms and Xmx values my app now is behaving correctly without any restarts. I also tried increasing the memory in the server to 7 gigs before setting the Xmx and Xms values, but it also didn't help. What i want to know is that, how come the system was going OOM, when there was sufficient memory available for the application.
Does the default heap size extend itself if memory requirement increases for the app ? If memory was getting full, why was GC not getting triggered ?

Answer 1

Does the default heap size extend itself if memory requirement increases for the app?

Yes, but only up to the value in -Xmx (or the default for that value that the JVM chose).

If memory was getting full, why was GC not getting triggered?

GC was getting triggered, but it could not collect any garbage. Everything in your heap was non-garbage.

Answer 2

"how come the system was going OOM, when there was sufficient memory available for the application."

There are two possible issues here:

1. You have not configured enough memory for the memory needs of your application so could you share the values of your configuration?

2. There is a memory leak problem caused by the application so you should keep in mind this

Source: http://blog.sysco.no/memory/leak/memory-leak/

One of the main advantages of Java is the use of a garbage collector that allows programmers to forget about the memory management. That is not the case of languages such as C++ where you have to allocate and free memory since your code. In my experience in a big company, I saw memory leak problems several times causing the unavailability or the bad performance of many applications.

In addition, a memory leak belongs to a class of situations called software aging problems. Software aging defines the loss of performance over the time because of the gradual accumulation of little problems. Other term within this kind of problem is the rejuvenation process. For example, rejuvenation happens when the system has to be restarted in order to free the memory accumulated by a Java Virtual Machine (JVM) process after a period of time (COTRONEO, D, et al., 2015).

Basically the garbage collector on the JVM uses a mechanism to reuse the space of objects, which does not have any reference from the root set so when there is a problem within the source code or configuration that keeps objects referenced for ever, little by little the application is going to fill the heap until reaching its limit.

For example, in the following figure just the unreachable objects inside the graph are going to be collected all the reachable objects (even those generated by a bug or bad configuration) are going to be kept in the heap

source http://blog.sysco.no/files/guides/JVMGarbageCollectionV1.1.pdf

"Does the default heap size extend itself if memory requirement increases for the app ?"

The answer is not, when tyour application fills the whole new generation, the JVM is going to execute a minor GC to release objects from the new generation and those who survive are moved into the survivor space for some cycles until they reach certain age. When they (objects in the survivor space) are mature enough and still are alive, these objects are promoted into the old generation and when the old generation is full, a full GC is performed. Thus, as it seems to be your application is keeping references (memory leak) at some point the whole heap will be filled up of alive objects so JVM is not able to release them and it does not have a flag to increase the memory in cases like this.

"If memory was getting full, why was GC not getting triggered ?"

In this case I have a question, how do you know whether the GC is getting triggered or not? To tune the garbage collection process you have to measure this so you can try these flags to get information about the GC execution.

-XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xloggc:/home/user666/gc.log

In addition, you can use this nice web to analyze the log file

http://gceasy.io/

Las but not least, you can use these flags

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/some/where/your/machine

With these flags you are going to produce a heap dump, which can be analyzed with Memory Analyzer Tool (MAT: https://www.eclipse.org/mat/downloads.php) to determine whether the application has a memory leak or not.

Answer 3

The default maximum heap size is 1/4 of main memory. This assumes the machine is not dedicated to this one process. If it is you might want to increase the maximum heap size.

Note: The heap size is only the size of the heap which is just one memory area the JVM uses. It is not the total memory consumption. How much additional memory your JVM needs depends on what it is doing.

The initial size can grow to the size of the maximum. The maximum size doesn't grow, it's the size of heap you would rather the process die than use any more.

Without more information, I would try a 3 GB heap on a 4 GB machine if that is all it is running and see how this goes. You can monitor how much virtual memory the process is trying to use in top

NOTE: Not all OutOfMemoryError(s) are the same and you need to read the error message to determine what is the actual cause of the problem. e.g. you could have run out of threads, perm gen/metaspace or some other low level memory problem.