Proper way to set Direct / Heap memories for distributed jar file?

Generally speaking, there is no difference between running from a batch file and the command prompt unless the command prompt environment has somehow been configured non-default. (ie: running with a different java home environment variable or something)

I think this may be the issue

I did some googling, and apparently there’s a common optimization app by Razer that changes system settings in an attempt to improve FPS for the system called “Razer Syanapse” and “Razer Cortex”, and 2 of my friends who own the devices giving me trouble said they have this application installed.

Apparently this Razer optimization app can cap java apps at 500mb because it alters the _JAVA_OPTIONS variable in your environment settings. But I haven’t tried first hand yet to confirm for sure, this just seems to be the consensus amongst a lot of minecraft users who had to uninstall this app to allow minecraft to allocate more than 499mb.

I’m waiting for one of my friends to be available for more troubleshooting over Teamviewer in the next few days, and then I can hopefully confirm first hand if Razer Synapse is indeed preventing JME games over 500mb from running. Its a shame that a Razer app designed to optimize games may be shunning out larger java games.

1 Like

I cannot tell you how many times I’ve been glad that Minecraft is so pervasive.

On the one hand, it does lead to a lot of screwy GLOBAL Java settings on some folks’ computers. On the other hand, 9 times out of 10, “Does Minecraft run?” is a good question to ask when trouble-shooting my apps on someone else’s box. :slight_smile:

3 Likes

This would be the place to put it,
https://wiki.jmonkeyengine.org/jme3.html#deployment

Do you know how to write wiki pages?

1 Like

No I actually haven’t ever edited the wiki before, but if it isn’t too difficult I could figure it out and start working on a documentation page titled “Distributing JME Games with Steam”

1 Like

Depends how far you want to take it. If you want to be able to preview and have shortcuts there is the Atom editor that has asciidoctor specific tools.

or for something as simple as it gets just click the + button at top of this page

https://wiki.jmonkeyengine.org/jme3/ios.html

and manually paste in text and preview using https://asciidoctor.org/docs/user-manual/#introduction-to-asciidoctor for a guid on syntax.

or, just gather what you want into one post in the thread and when ready and I can add it.

1 Like

Have any of you ever tried jlink?

What I’m doing on my tests is creating a custom runtime and packing it with my app. To illustrate what I’m doing a little better, I’m following these few steps:

  1. Add application plugin to build.gradle.
plugins { 
    id 'application'
}
  1. Run assembleDist task from that plugin. It creates two files – a zip and a tar (essentially the same) – under the build/distributions directory of your project containing what the app needs to run, including startup batch files (Windows .bat and shell script).

  2. Create a runtime image with jlink:

$ jlink --module-path <path-to-jdk>/jmods --add-modules <modules-names-you-need> --no-header-files --no-man-pages --compress=2 --strip-debug --vm=server --output <path-to-your-project>/build/distributions/runtime
  1. Add the generated runtime directory to the zip or tar (whichever you like) generated by assembleDist. The zip or tar file should have 3 dirs now: bin, lib, runtime.

  2. Change the batch files to use the Java executable located in the runtime directory jlink just created.

Pros:

  • Users don’t have to download Java themselves in order to run your game.
  • The JVM provided includes only essential modules, it’s not necessarily a full JRE.
  • You know what JRE version your app is running on.

Cons:

  • Once packaged and distributed, you can’t upgrade the JVM version (or at least I didn’t think on a way to do it).
  • The file is larger because now it has a JVM embedded.
3 Likes

i will need ask folks, but myself i didnt use it, but sounds promising. thanks.

Question:

  • is JVM as separated or packaged together? i wonder if steam would need update all again, or just part of files.
  • are there some problems when using OpenJDK (lets say AdoptOpenJDK) to compile? and what jre versions it support.

I had more time to troubleshoot the issue where the JVM memory params are ignored, and I made progress but have still not solved it entirley.

I have noticed that running the game on certain machines is extremely buggy if I’ve set the memory params in the SDK as well as in in the batch file/command prompt.

Increasing the JVM memory params in the batch file would only lead to more OutOfMemoryErrors (very strange, as it usually would throw an unable to allocate memory error on startup if the memory params are too high), until I finally erased all of the JVM params in the SDK before building the app. I guess some devices do not like when the jvm params are set in both places?

Additionally, I noticed that I cannot get the app to run without “buggy memory” issues on my friend’s device unless I set the DirectMemory param to the JVM, and the heap memory param to the App

In other words, I changed this
start javaw -XX:MaxDirectMemorySize=2250m -Xmx950m -jar The_Afflicted_Forests.jar
to this
start javaw -XX:MaxDirectMemorySize=2250m -jar The_Afflicted_Forests.jar -Xmx950m

and now I can successfully load all of the maps in my game with his device (as long as the map does not exceed ~1024 mb of DirectMemory for some reason. it appears this is a cap on his device despite the JVM param I’ve fianlly mangaged to set above a couple hundred mb. Beforehand when I was setting both params to the JVM, I could not even load a map that took up 600mb. I pretty much tried 100 different things with the batch file and JVM params by trial and error until he could load my main menu without an out of memory error… so I can’t really say i learned much here unfortunately, since half my maps still give an out of memory error).

And strangely, my own 3 devices will run every map in my game regardless of where I define the memory params. I am absolutely confused as to why my devices are all working seamelssly 100% of the time, but 3 other devices are consistently refusing to allocate the amount of memory as indicated by my JVM params .

I have the same java version and the same environment variables set on my own device, yet I cannot reproduce the same error I’m getting on a friend’s device.

Should i be looking at something else on their devices for java settings? I am still fairly new to this side of java and JVM tuning, so I do not know where else to look for device-specific settings aside from the Enviornemny Variables - and I already confirmed that the device causing problems does not have any java Environment Variables set that would be capping the memory.

I assume you’ve checked your friend’s global environment variables. Due to minecraft and other Java games (mostly minecraft) some folks have global Java heap parameters defined in their environment variables.

Note: when you put -Xetc. after the .jar then they are just passed as parameters to the application and are thus ignored by java. Probably if you print all of main()'s args you will see it… so it’s just being ignored in that case.

I initially tried settings the params this way (and i also tried setting params to both the JVM and app, with no luck) as I had read something that said I should be settings the params to the JVM, and not to the application. I essentially tried every combination of setting the DirectMemory / Heap memory to the app and/or JVM, and strangely enough I could only get his device to load my main menu when the heap was defined in the app, or not defined at all.

Settings the heap memory to the JVM on his device causes the DirectMemory to bug out and ignore my XXDirectMemory param on his device (as indicated by an OOM error while loading the main menu scenery thats only ~400mb) - but if I do not set it in the app, then his device gets a heap memory error from just a handful of NPCs being spawned at once, which i thought was indicative that the application’s heap memory param seems to be doing something on his device.

There were no java related environment variables set on his device.

I’m wondering if this issue may be related to java 8? (my project started in the SDK and I still use the SDK, so I’ve always been limited to java 8) And if so, should I consider upgrading to a newer java version and see if that may make a difference?

That’s the only other thing I can think of doing - aside from confirming that the global Environment Variables are not capping the memory, I do not know of any other place that could be changing the way java runs between devices.

I think maybe you misunderstand what I’m saying.

This:

Is the same as this:

`start javaw -XX:MaxDirectMemorySize=2250m  -jar The_Afflicted_Forests.jar` -fobiddyBigBadVoodooDaddy

Anything after the jar in this case is passed to the application as command line arguments… and if the main class in The_Afflicted_Forest is not looking at them then they just sit there in the String variable in main.

In your main() method, add this code:

for( String s : args ) {
    log.info("arg:" + s);  (or whatever)
}

…and you should see -Xmx950m in the log file if you are configured to capture logging.

I must have mistakenly thought that setting the heap to the application was doing something. But even without the heap setting defined to the JVM, the app runs - it just can’t handle a lot of NPCs since thats what typically drives up the heap memory in my game.

The strange thing is that setting the heap memory to the JVM will bug out the Direct Memory setting on my friends device.

So if I ignore the heap memory for now, I can run the gam e without a Direct Memory OOM error on his device using

start javaw -XX:MaxDirectMemorySize=2250m -jar The_Afflicted_Forests.jar
(however a heap memory eventually happens as more NPCs spawn, and maps that use more than ~1024mb of direct memory also cause an OOM, despite the memory being set over 2000mb on a device with 16gb ram)

but if I add the heap param, then the main menu crashes with a direct memory error.

start javaw -XX:MaxDirectMemorySize=2250m -Xmx950m -jar The_Afflicted_Forests.jar

What version of java does your friend run?

Java has changed the relationship of direct memory and heap memory many times. It could be that 950m is not enough to hold 2250m of memory and it’s trying to keep the direct memory as part of the heap.

I’m not up on the latest rules. I just know that locally my java 1.7 apps where I used to have to set direct memory, I don’t have to do it in 1.8 as it seems to ‘just work’. Also, no matter what I set it to, the JVM will report nothing interesting for max available direct memory (unlike on 1.7 where you got real numbers for that). To me, this (maybe incorrectly) indicates that the JVM is doing more to manage the direct heap than it used to and so a “max available” value is sort of meaningless now (or at least hard to calculate).

1 Like

This can be helpful:
https://developer.ibm.com/answers/questions/398383/summary-of-understanding-xxmaxdirectmemorysize-set/

What happens if you don’t set XX:MaxDirectMemorySize? According to Oracle, by default, the size [of XX:MaxDirectMemorySize] is set to 0, meaning that the JVM chooses the size for NIO direct-buffer allocations automatically.

I may be wrong (probably I’m wrong), but I think there is a chance that your app can work if you set only heap values.

What if you try this:
start javaw -Xms2g -Xmx2g The_Afflicted_Forests.jar

1 Like

This unfortunately doesn’t work on his device, although it does work on my device.
On his device, If I reduce the values to 1g instead of 2g, then his computer will launch the game.
When the heap is set to 2g it won’t even launch the game, and does not report any errors.

If I set the heap to 1g and the direct memory to anything more than 1g, then the game goes back to crashing without an error.

So it seems like there is a cap on the heap + direct memory size, which caps out around 2g

I’ve also noticed that the SDK runs my game without requiring any memory params, on both my device and my friends device, so it does seem like a lot has chanced since I last checked.

And this is leading me to wonder how the SDK is able to run my app with no memory errors, and can also load large maps in the scene editor without memory errors. I was also able to increase the heap and direct memory in the SDK config successfuly onmy friend’s device, but as soon as I’m out of the SDK the app apparently runs differently.

He is using version 8, specifically 1.8.0_241-b07 which is the same exact version I’m running on my device that is not having any trouble.

To take things further, I also copied the exact JRE from my computer to his, to ensure there was no discrepancy between versions, and the results are the same.

However I should note, that on my device, it says the JRE architecture is “x86_64” wheras the version on his device says just “x86” . Eevn when I copied my x64 JRE to his device, the java configuration labeled it as a “x86” architecture, although his device is indeed 64 bit.

It seems like his device is possibly running java as 32 bit? Although I don’t know why this would be happening, but I cannot get his device to run with a heap value higher than 2gb, and I know that java 32 bit is capped at 2gb for the heap. Unless it is just a cooincidence that something else is causing the memory to cap at around the same amount…

So to summarize where I am at with this issue on his device:

  • start javaw -Xms2g -Xmx2g The_Afflicted_Forests.jar
    this causes an immediate crash with no error reports when the JVM attempts to start on his device

  • start javaw -Xms1g -Xmx1g The_Afflicted_Forests.jar
    this will reach the main menu, but causes an OutOfMemory error for DirectMemory when the menu and menu-scenery is loaded (the app currently takes up ~700mb of direct memory at this point)

  • start javaw -XX:MaxDirectMemorySize=1400m -Xmx1024m -Xms512 -jar The_Afflicted_Forests.jar
    this will cause a crash on startup with no error as well. this is the memory setting that I believe should be working - as my game requires ~1400 mb for the biggest maps when not debugging, and slightly under 1gb of heap

  • start javaw -XX:MaxDirectMemorySize=2250m -jar The_Afflicted_Forests.jar
    this works to load the main menu, and any maps that are under 1000mb (this is an estimate) of Direct Memory. But it causes an out of memory exception for a map that currently takes up ~1200mb of Direct Memory

So it seems like setting the direct memory past 2gb does not cause a start-up crash if that is the only memory param set - but the game will still have out of memory errors when loading maps that eventually require more than 1g of direct memory.

And setting the heap just seems to cause more problems on his device. The higher the heap, the lower I must set the DirectMemory in order to avoid a JVM startup crash with no error - once the DirecrMemory is high enough in conjunction with a Heap size greater than 0, then the JVM will also crash with no errors, the same way it does when the heap is set anywhere past 2gb.

Edit: I also tested all of the same JVM params on a small java app that was not related to JME at all, and got the same results, so this does not seem to be exclusive to JME or my app, but I cannot find anyone having any similar issues when I try googling the problem :confused:

Without being able to poke at this myself, it certainly does seem like he’s running 32 bit java. Perhaps there is a 32 bit Java on the path ahead of it or there is some DLL magic or something going on.

You’ve been very thorough, though… so these are just shots in the dark. 64 bit java will report itself as such… but 32 bit Java will tend to error for larger heap values. At least it used to on JDK7.

1 Like

What are the exact operating-system versions involved?

I expect that a 32-bit operating system running on 64-bit hardware would show up as “x86”.

2 Likes

I also think he is running some 32 bit version of Java. Check the value of the PATH variable on his system. It probably points to a Java x86 somewhere.

1 Like

Yes it does indeed appear that his 64 bit machine has somehow decided to run 32 bit java - and it also turned the 64 bit version of java into 32 bit when I tried to copy the JRE that is recognized as 64 bit on my device onto his device.

So I also ran this line in the command prompt to confirm, and it is definitely reporting as 32 bit java
java -XshowSettings:properties -version

I was also the one who downloaded java on his device a few months ago to run my game, and I was fairly certain I downloaded 64 bit, because I have always been aware of the 32 bit memory limitation, and have already determined my game will never run on 32 bit machines

I also had 2 other friends download java around the same time, as they also did not have java prior to downloading my game. And for some reason, all of my friends with 64 bit machines somehow downloaded 32 bit java from the oracle download page… so I am wondering if java’s download page is messed up, otherwise it was an unlucky cooincidence that everyone somehow got 32 bit java except for me - but I’ve also always had java on all of my devices for longer than I can remember, whereas my 3 friends who got 32 bit java just installed it a few months ago.

So would the solution be for me to just go ahead and package the correct JRE with my game, and stop having people download java on their own? I’m guessing that is why the SDK has been able to run my game on his device despite having 32 bit java, since (if I recall correctly) the SDK comes with a bundled JRE of its own.
This is a requirement by Steam anyways (as I mentioned in one of my very early posts), but I suppose I stubbornly tried to fix this launch error before packaging a JRE into my build and reuploading to Steam, and it turns out that was the problem all along :laughing:

1 Like