Java: what determines the maximum max heap size possible in a linux machine
I have two linux machines (both are VMs), one having 12GB memory and other having 8GB memory.
I tried to start the same java program on both machines, with maximum max heap size possible (using -Xmx flag). Following are the results I got.
If I specify a max heap size beyond above limits, I get below error.
Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes
I checked the free memory in two systems (using free command), and I got following.
My question is, what determines the maximum max heap size a java program can be started with, which would not result in above mentioned error? (System had sufficient memory to allocate 1073741824 bytes of memory when the program gave above error)
I have found interesting comments from JDK bug ( The bug in JDK 9 version and not in 8. It says bug was fixed in 8.x version but does not tell minor build number.
If virtual memory has been limited with "ulimit -v", and the server has a lot of RAM, then the JVM can not start without extra command line arguments to the GC.
// After "ulimit -v" The jvm does not start with default command line.
$ ulimit -S -v 4194304
$ java -version
Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes
Comments:
The problem seems to be that we must specify MALLOC_ARENA_MAX.
If I set the environment variable MALLOC_ARENA_MAX=4, then the jvm can start without any extra arguments.
I guess that this is not something that can be fixed from the jvm. If so we can close this bug.
When using "UseConcMarkSweepGC" then the command line above does not work. I have tried to add MaxMetaspaceSize=128m, but it does not help. I am sure there are an argument that makes it work, but I have not found one. Configuring the GC with limited virtual memory is not very user friendly.
Change parameters to as per your requirement and try this one.
ulimit -S -v 4194304
java -XX:MaxHeapSize=512m -XX:InitialHeapSize=512m -XX:CompressedClassSpaceSize=64m -XX:MaxMetaspaceSize=128m -XX:+UseConcMarkSweepGC -version
I did some experiments with the clues given by ravindra and found that the maximum max heap size has a direct relationship with the total virtual memory available in the system.
Total virtual memory in the system can be found (in KB) with:
ulimit-v
Total virtual memory can be altered with:
ulimit -v <new amount in KB>
The maximum max heap size possible was approximately 2GB less than the virtual memory. If you specify unlimited virtual memory using ulimit -v unlimited , You can specify any large value for the max heap size.
The memory you have available is the combination of free RAM plus swap space. It also depends on whether the system has overcommit enabled — if so, the kernel will allow programs to allocate more memory than is actually available (within reasonable limits), since programs often allocate more than they're really going to use.
Note that overcommit is enabled by default. To disable it, write 2 into /proc/sys/vm/overcommit_memory . (Strangely, a value of 0 does not mean "no overcommit".) But it's a good idea to read the overcommit documentation first.