Native Compilation - GraalVM Native Image

Hi, I was investigating the possibility of compiling a JME game to native using the GraalVM. I am also curious about possible performance degradation when compiling to native, but for now I simply attempted to compile. I was able to compile without any real issues, but upon attempting to run the executable, I received the following error:

Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
        at com.jme3.system.AppSettings.<clinit>(AppSettings.java:263)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        at MainKt.main(main.kt:45)
        at MainKt.main(main.kt)
Caused by: java.lang.NullPointerException: inStream parameter is null
        at java.util.Objects.requireNonNull(Objects.java:246)
        at java.util.Properties.load(Properties.java:403)
        at com.jme3.system.JmeVersion.<clinit>(JmeVersion.java:51)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        ... 5 more

So, I am wondering if anyone knows why I might be getting this error?

I just started doing the JME tutorials (though I’m doing them in kotlin :slightly_smiling_face:) , and native compilation probably won’t change much for me, but I am curious, as I understand from this post that there shouldn’t be much in the way of native compilation.

As I am new here, please feel free to correct me if this is in the wrong category, or if I made some other mistake :slightly_smiling_face:.

Hi

Have you configured the reflections list?

Have not tried Graalvm native image myself but as far as I know all the classes that are used reflectively should be included in a reflection configuration file. The same goes for JNI classes as well.

I think they have added a new tool that will automatically do that for you. See this article for more details

Regarding the NullPointerException it seems it can not find version.properties file in this line

I guess you need to configure resources that need to be included in the native image as well.

These might be related:

Thanks. Actually, once I ran the tracing agent, everything, including resources, seemed to be configured automatically. But I faced further issues, I’ll report back later when I have more time. If I am successful, I’ll also post here the steps, of course.

2 Likes

I have been fighting this issue on getting one of my applications to compile: https://github.com/oracle/graal/issues/1979

It seems that many loggers do not like graalvm, and unfortunately many jme community built libraries used these loggers.

@tlf30 have you tried with the tracing agent?

I didn’t face the issue @tlf30 is facing, as I am compiling a simple hello world. But it appears that there is an outstanding issue preventing compilation. I will document my steps here:

  1. After using the tracing agent, one of the bundles in the resource-config.json file was {"name":"com.jme3.app/SettingsDialog"}, I don’t know why a / was used here instead of a . (different than all the other bundles, as well), which failed the build. Simple solution: switch to {"name":"com.jme3.app.SettingsDialog"},.

  2. As per this issue, I added the following file to be compiled with the JAR. Note, this requires the org.graalvm.nativeimage:svm:20.2.0 dependency.

import org.lwjgl.*;
import com.oracle.svm.core.annotate.TargetClass;
import com.oracle.svm.core.annotate.RecomputeFieldValue;
import com.oracle.svm.core.annotate.RecomputeFieldValue.Kind;
import com.oracle.svm.core.annotate.Alias;

@TargetClass(className = "org.lwjgl.MemoryUtilSun$AccessorUnsafe")
final class Target_org_lwjgl_MemoryUtilSun_AccessorUnsafe {
    @Alias @RecomputeFieldValue(kind = Kind.FieldOffset, declClass = java.nio.Buffer.class, name = "address") long address;
}

/** Dummy class to have a class with the file's name. */
public class MySubstitutions {
}
  1. I then ran native-image -jar ./build/libs/hello_world-0.1-SNAPSHOT-all.jar which resulted in a fallback image requiring a JDK for execution, so I added -H:+ReportExceptionStackTraces --no-fallback flags to determine the cause. It turned out that there were a number of unresolved types from lwjgl.

  2. So I added the --allow-incomplete-classpath flag to ignore the issue (perhaps it would run regardless if compiled?).

  3. This resulted in the linker error in this issue, apparently it is fixed on master, but for now you can just add the -H:NativeLinkerOption=prefs.lib flag.

  4. This successfully compiles! No problems. However, upon running the resulting executable, I get the following error: java.lang.UnsatisfiedLinkError: no awt in java.library.path. Apparrently there is an open issue related to this. I don’t know any more than that.

So, for now, it seems that I cannot compile to native. Need to wait for that last issue mentioned. Unless @tlf30 knows a way around this?

2 Likes

Seems there is a simmilar report here

Can you disable JME settings window and see if that error happens again?

Edit:
Set this to false

1 Like

No, I no longer get that error!
However, I now get a NullPointerException. I think that was referenced in one of the issues I linked.

May you put the whole stack trace?

Yes, sorry.

Nov 01, 2020 2:49:20 PM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.3.2-stable
 * Branch: HEAD
 * Git Hash: 1a05e3f
 * Build Date: 2020-04-27
Exception in thread "jME3 Main" java.lang.ExceptionInInitializerError
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
        at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:212)
        at java.lang.Thread.run(Thread.java:834)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
        at com.oracle.svm.core.windows.WindowsJavaThreads.osThreadStartRoutine(WindowsJavaThreads.java:138)
Caused by: java.lang.NullPointerException
        at com.oracle.svm.jni.functions.JNIFunctions$Support.getMethodID(JNIFunctions.java:1084)
        at com.oracle.svm.jni.functions.JNIFunctions.GetMethodID(JNIFunctions.java:391)
        at com.oracle.svm.jni.JNIOnLoadFunctionPointer.invoke(JNILibraryInitializer.java)
        at com.oracle.svm.jni.JNILibraryInitializer.callOnLoadFunction(JNILibraryInitializer.java:72)
        at com.oracle.svm.jni.JNILibraryInitializer.initialize(JNILibraryInitializer.java:127)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.addLibrary(NativeLibrarySupport.java:185)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibrary0(NativeLibrarySupport.java:141)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibrary(NativeLibrarySupport.java:102)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:228)
        at java.lang.Runtime.load0(Runtime.java:768)
        at java.lang.Runtime.load(Runtime.java:244)
        at java.lang.System.load(System.java:358)
        at org.lwjgl.Sys$1.run(Sys.java:70)
        at java.security.AccessController.doPrivileged(AccessController.java:83)
        at org.lwjgl.Sys.doLoadLibrary(Sys.java:66)
        at org.lwjgl.Sys.loadLibrary(Sys.java:87)
        at org.lwjgl.Sys.<clinit>(Sys.java:117)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        ... 4 more
1 Like

Are you using --initialize-at-build-time flag?

No, my full command is native-image -jar ./build/libs/hello_world-0.1-SNAPSHOT-all.jar --allow-incomplete-classpath -H:NativeLinkerOption=prefs.lib. However, I believe that by default all classes are initialized at build-time. I don’t know enough about JME to start modifying that behavior.

Edit:
I get the NPE at runtime though…

Do not know about it but based on the doc

The whole class hierarchy can be initialized at build time by passing --initialize-at-build-time on the command line.

Can you add that flag and see if anything changes?

After adding that flag, it unfortunately now fails with:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000000000000, pid=10220, tid=14228
#
# JRE version: OpenJDK Runtime Environment GraalVM CE 20.2.0 (11.0.8+10) (build 11.0.8+10-jvmci-20.2-b03)
# Java VM: OpenJDK 64-Bit Server VM GraalVM CE 20.2.0 (11.0.8+10-jvmci-20.2-b03, mixed mode, tiered, jvmci, jvmci compiler, compressed oops, parallel gc, windows-amd64)
# Problematic frame:
# C  0x0000000000000000
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\moshe\Desktop\Foolish_Wisdom\kotlin\jmonkey\hello_world\hs_err_pid10220.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/oracle/graal/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Error: Image build request failed with exit status 1

EDIT:
The graal bug tracker suggests building on the latest dev distribution of graal before filing a report. Will try that now.

EDIT 2:
The filed issue is here.

2 Likes

By the way, with regards to awt, it appears there was a commit to add support for it two days ago:
[GR-25755] Add AWT and Swing to JDK 11.

3 Likes

Just a blind guess

Looking at the error log looks like it has something to do with OpenCL.

Can you add this to delay OpenCL initializing

--initialize-at-run-time=org.lwjgl.opencl

I added this flag --initialize-at-run-time=org.lwjgl.opencl,org.lwjgl.openal,org.lwjgl.opengl as the native-image utility was throwing errors about initialization of these classes. I also tested together with the --initialize-at-build-time flag (combined with the other flag, it did not fail as it did before). It compiled successfully, but resulted in the same NPE at runtime.

EDIT: actually, using --initialize-at-build-time results in various awt issues (there stil hasn’t been a dev release since the commit with awt support).

EDIT 2: the awt issues (mentioned in the previous edit) are despite the fact that settings are set to disabled in the code, as suggested above in this thread.

I think you also can’t cross compile with graal. Which is pretty annoying too, that’s a reason many use java. See https://github.com/oracle/graal/issues/407

OK, just attempting a small test jme app:

[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] Exception in thread "main" java.lang.ExceptionInInitializerError
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.system.AppSettings.<clinit>(AppSettings.java:263)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.app.SimpleApplication.start(SimpleApplication.java:115)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at io.tlf.jme.test.Main.main(Main.java:90)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] Caused by: java.lang.NullPointerException: inStream parameter is null
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at java.util.Objects.requireNonNull(Objects.java:246)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at java.util.Properties.load(Properties.java:403)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.system.JmeVersion.<clinit>(JmeVersion.java:51)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	... 5 more

Same as OP NPE
Still playing with it.