Java9 incompability

Empire_Phoenix · July 5, 2016, 9:59am

The current JME3 is not capable of running in a Java9 environment.

The reason is, that the access to the cleaner methods in Bufferutils is not possible anymore, since they are now hidden.

This thread is intended for discussion how to workaround/solve this. See end of this post for current ideas (i will add more as they are proposed):

Jul 05, 2016 11:54:30 AM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.2-5781
 * Branch: master
 * Git Hash: e2b6c51
 * Build Date: 2016-06-23
Jul 05, 2016 11:54:31 AM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: LWJGL 2.9.3 context running on thread jME3 Main
 * Graphics Adapter: null
 * Driver Version: null
 * Scaling Factor: 1
Jul 05, 2016 11:54:31 AM com.jme3.app.LegacyApplication handleError
SEVERE: Uncaught exception thrown in Thread[jME3 Main,5,main]
java.lang.ExceptionInInitializerError
	at com.jme3.renderer.opengl.GLRenderer.<init>(GLRenderer.java:80)
	at com.jme3.system.lwjgl.LwjglContext.initContextFirstTime(LwjglContext.java:236)
	at com.jme3.system.lwjgl.LwjglContext.internalCreate(LwjglContext.java:377)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.initInThread(LwjglAbstractDisplay.java:117)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:211)
	at java.lang.Thread.run(java.base@9-ea/Thread.java:843)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make member of interface sun.nio.ch.DirectBuffer accessible:  module java.base does not export sun.nio.ch to unnamed module @76fb893d
	at jdk.internal.reflect.Reflection.throwInaccessibleObjectException(java.base@9-ea/Reflection.java:414)
	at java.lang.reflect.AccessibleObject.checkCanSetAccessible(java.base@9-ea/AccessibleObject.java:174)
	at java.lang.reflect.Method.checkCanSetAccessible(java.base@9-ea/Method.java:191)
	at java.lang.reflect.Method.setAccessible(java.base@9-ea/Method.java:185)
	at com.jme3.util.BufferUtils.loadMethod(BufferUtils.java:1278)
	at com.jme3.util.BufferUtils.<clinit>(BufferUtils.java:1291)
	... 6 more


Exception: java.lang.NullPointerException thrown from the UncaughtExceptionHandler in thread "jME3 Main"

1.) The simple way
Just remove the dispose method, and hope that the jdk9 is now finally capable of properly cleaning buffers

2.) The native way
Create a ultra small malloc/free wrapper that allows to abitrarly request and delete directbuffers.

3.) The selfmade memoryManagement way
At startup request a fixed size of directmemory (user specifiable). Then only give out subbuffers of this. Internally ensure that no part is given out twice without a further delete by manageing the free space. (Kinda similar how malloc itself works in C)

pspeed · July 5, 2016, 1:25pm

I thought it used to be written that it would just ignore the call if it couldn’t find the methods. I may be misremembering. That seems like the safest fallback to me. Then we still get the nice behavior on Java 8 and under and can tread water and hope for the best for JDK 9 while we work out a solution there.

The other options are kind of not good. (2) means we bypass any kind of knowledge the VM might have wanted about that memory. (3) means we have to deal with heap fragmentation and all kinds of other nasty things that presumably smarter people have already thought about.

zissis · July 6, 2016, 4:14am

This is not a solution. The jira for fixing the broken cleaning of buffers and the broken unmap implementation of memory map files in java is about 18 years old and STILL not addressed.

Empire_Phoenix · July 6, 2016, 6:35am

I agree,

personally I’m kinda in favour of the native way, that utility is pretty small, and with bullet we already have knowledge/example how to create native stuff.

And about pspeeds argument, the problem here is that the damn jvm does not even want to know, else it would work okay.

tonihele · July 6, 2016, 7:42am

This is not a solution. The jira for fixing the broken cleaning of
buffers and the broken unmap implementation of memory map files in java
is about 18 years old and STILL not addressed.

Doesn’t necessarily mean it is not fixed. Just not documented. Happens sometimes Just a thought,

Pesegato · July 6, 2016, 8:04am

and

You sure are bleeding edge… but isn’t it a bit early to worry about this?

zissis · July 6, 2016, 9:19am

It’s not fixed. I still have to use my workarounds in my commercial projects. I already tested the memory leaks and unreleased file handles.

FrozenShade · July 6, 2016, 9:51am

I found something interesting: truly unusual experience of revolution / Code / [r1369] /pre_beta/src/main/java/engine/misc/DeallocationHelper.java

Empire_Phoenix · July 6, 2016, 11:48am

Surely not,
we have ~ 200 days to find a acceptable solution and maybee programm a native part.

about the deallocation helper, it uses the same concept we use, so tuer is broken as well then

(You can no longer see/use the corresponding sun classes at all, not even via reflection)

Another way might be to use this
http://javadoc.lwjgl.org/org/lwjgl/system/MemoryUtil.html#memFree-java.nio.Buffer-
if a similar solution for jogl does exist.

zzuegg · July 6, 2016, 12:28pm

lwjgl3 has bindings to http://www.canonware.com/jemalloc/ for buffer management… might be worth a shot

zissis · July 7, 2016, 6:52pm

Whatever you guys do please do a at runtime to see if it’s running under java 8 or below and make sure you are still using the same code we have now for those versions.

Empire_Phoenix · July 8, 2016, 9:13am

No,

whatever I do, I will try to reduce reflection usage as much as possible,
my personal favourite would be to put a interface between the bufferstuff,
and allow different implementations for buffers, as a solution working great on desktop does not necessarly make sense on android ect.

One of those solution could of course be the current implementation

Anyway this won’t affect 3.0 or even 3.1 most likely it will only affect 3.2

zissis · July 9, 2016, 2:52am

Unless you use native libraries there is no way to avoid reflection with jdk8 unless you want leaks

Empire_Phoenix · July 9, 2016, 10:08am

Take a look:

nehon · July 9, 2016, 10:11am

the diff is a bit hard to read

Empire_Phoenix · July 9, 2016, 10:11am

I noticed basically it just replaces every call with allocateDirect to allocator.allocate and psuhes the reflection magic to a new class.

Empire_Phoenix · July 9, 2016, 10:17am

Extracted BufferAllocation to be replaceable and allow java9 to run by empirephoenix · Pull Request #526 · jMonkeyEngine/jmonkeyengine · GitHub try with ?w=1

nehon · July 9, 2016, 10:50am

oh I didn’t know this trick! nice!

nehon · July 9, 2016, 10:52am

alright.
Are you completely sure that those packages will be accessible through reflection though?
Seems weird to me…

Dokthar · July 9, 2016, 11:17am

interesting post from @gouessej