Native Compilation - GraalVM Native Image

mosheduminer · November 1, 2020, 1:51pm

Hi, I was investigating the possibility of compiling a JME game to native using the GraalVM. I am also curious about possible performance degradation when compiling to native, but for now I simply attempted to compile. I was able to compile without any real issues, but upon attempting to run the executable, I received the following error:

Exception in thread "main" java.lang.ExceptionInInitializerError
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
        at com.jme3.system.AppSettings.<clinit>(AppSettings.java:263)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        at MainKt.main(main.kt:45)
        at MainKt.main(main.kt)
Caused by: java.lang.NullPointerException: inStream parameter is null
        at java.util.Objects.requireNonNull(Objects.java:246)
        at java.util.Properties.load(Properties.java:403)
        at com.jme3.system.JmeVersion.<clinit>(JmeVersion.java:51)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        ... 5 more

So, I am wondering if anyone knows why I might be getting this error?

I just started doing the JME tutorials (though I’m doing them in kotlin ) , and native compilation probably won’t change much for me, but I am curious, as I understand from this post that there shouldn’t be much in the way of native compilation.

As I am new here, please feel free to correct me if this is in the wrong category, or if I made some other mistake .

Ali_RS · November 1, 2020, 2:52pm

Hi

Have you configured the reflections list?

Have not tried Graalvm native image myself but as far as I know all the classes that are used reflectively should be included in a reflection configuration file. The same goes for JNI classes as well.

I think they have added a new tool that will automatically do that for you. See this article for more details

Ali_RS · November 1, 2020, 4:38pm

Regarding the NullPointerException it seems it can not find version.properties file in this line

github.com

jMonkeyEngine/jmonkeyengine/blob/ab96460853b0aa98027a9fb03d879c265f2d77a1/jme3-core/src/main/java/com/jme3/system/JmeVersion.java#L51


* 
* @author Kirill Vainer
*/
public class JmeVersion {
   
   private static final Logger logger = Logger.getLogger(JmeVersion.class.getName());
   private static final Properties props = new Properties();
   
   static {
       try {
           props.load(JmeVersion.class.getResourceAsStream("version.properties"));
       } catch (IOException | NullPointerException ex) {
           logger.log(Level.WARNING, "Unable to read version info!", ex);
       }
   }
   
   public static final String BUILD_DATE       = props.getProperty("build.date", "1900-01-01");
   public static final String BRANCH_NAME      = props.getProperty("git.branch", "unknown");
   public static final String GIT_HASH         = props.getProperty("git.hash", "");
   public static final String GIT_SHORT_HASH   = props.getProperty("git.hash.short", "");
   public static final String GIT_TAG          = props.getProperty("git.tag", "");

I guess you need to configure resources that need to be included in the native image as well.

These might be related:

mosheduminer · November 1, 2020, 4:42pm

Thanks. Actually, once I ran the tracing agent, everything, including resources, seemed to be configured automatically. But I faced further issues, I’ll report back later when I have more time. If I am successful, I’ll also post here the steps, of course.

tlf30 · November 1, 2020, 5:54pm

I have been fighting this issue on getting one of my applications to compile: Micronaut app native image generation fails with NoClassDefFoundError: Lorg/apache/logging/log4j/spi/LoggerContext; · Issue #1979 · oracle/graal · GitHub

It seems that many loggers do not like graalvm, and unfortunately many jme community built libraries used these loggers.

Ali_RS · November 1, 2020, 6:48pm

@tlf30 have you tried with the tracing agent?

mosheduminer · November 1, 2020, 7:08pm

I didn’t face the issue @tlf30 is facing, as I am compiling a simple hello world. But it appears that there is an outstanding issue preventing compilation. I will document my steps here:

After using the tracing agent, one of the bundles in the resource-config.json file was {"name":"com.jme3.app/SettingsDialog"}, I don’t know why a / was used here instead of a . (different than all the other bundles, as well), which failed the build. Simple solution: switch to {"name":"com.jme3.app.SettingsDialog"},.
As per this issue, I added the following file to be compiled with the JAR. Note, this requires the org.graalvm.nativeimage:svm:20.2.0 dependency.

import org.lwjgl.*;
import com.oracle.svm.core.annotate.TargetClass;
import com.oracle.svm.core.annotate.RecomputeFieldValue;
import com.oracle.svm.core.annotate.RecomputeFieldValue.Kind;
import com.oracle.svm.core.annotate.Alias;

@TargetClass(className = "org.lwjgl.MemoryUtilSun$AccessorUnsafe")
final class Target_org_lwjgl_MemoryUtilSun_AccessorUnsafe {
    @Alias @RecomputeFieldValue(kind = Kind.FieldOffset, declClass = java.nio.Buffer.class, name = "address") long address;
}

/** Dummy class to have a class with the file's name. */
public class MySubstitutions {
}

I then ran native-image -jar ./build/libs/hello_world-0.1-SNAPSHOT-all.jar which resulted in a fallback image requiring a JDK for execution, so I added -H:+ReportExceptionStackTraces --no-fallback flags to determine the cause. It turned out that there were a number of unresolved types from lwjgl.
So I added the --allow-incomplete-classpath flag to ignore the issue (perhaps it would run regardless if compiled?).
This resulted in the linker error in this issue, apparently it is fixed on master, but for now you can just add the -H:NativeLinkerOption=prefs.lib flag.
This successfully compiles! No problems. However, upon running the resulting executable, I get the following error: java.lang.UnsatisfiedLinkError: no awt in java.library.path. Apparrently there is an open issue related to this. I don’t know any more than that.

So, for now, it seems that I cannot compile to native. Need to wait for that last issue mentioned. Unless @tlf30 knows a way around this?

Ali_RS · November 1, 2020, 7:34pm

Seems there is a simmilar report here

github.com/oracle/graal

[native-image] Windows with a swing application

opened 12:24AM - 24 May 19 UTC

Skhmt

feature native-image

Running: `C:\graalvm-ce-19.0.0\bin\native-image.cmd -jar jPatcher.jar --no-fallb…ack` I get the following: ``` Error: Unsupported features in 2 methods Detailed message: Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time. The object was probably created by a class initializer and is reachable from a static field. By default, all class initialization is done during native image building.You can manually delay class initialization to image run time by using the option -H:ClassInitialization=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point. Trace: object sun.awt.AWTAutoShutdown method sun.awt.AWTAutoShutdown.getInstance() Call path from entry point to sun.awt.AWTAutoShutdown.getInstance(): at sun.awt.AWTAutoShutdown.getInstance(AWTAutoShutdown.java:133) at java.awt.EventQueue.detachDispatchThread(EventQueue.java:1137) at java.awt.EventDispatchThread.run(EventDispatchThread.java:88) at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:473) at com.oracle.svm.core.windows.WindowsJavaThreads.osThreadStartRoutine(WindowsJavaThreads.java:137) at com.oracle.svm.core.code.IsolateEnterStub.WindowsJavaThreads_osThreadStartRoutine_4bc03aa26f8cdfc97ebd54050e8ae4bce1023851(generated:0) Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image run time. The object was probably created by a class initializer and is reachable from a static field. By default, all class initialization is done during native image building.You can manually delay class initialization to image run time by using the option -H:ClassInitialization=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point. Trace: object sun.java2d.opengl.OGLRenderQueue field sun.java2d.opengl.OGLRenderQueue.theInstance Error: Use -H:+ReportExceptionStackTraces to print stacktrace of underlying exception Error: Image build request failed with exit status 1 ``` So then I run: `C:\graalvm-ce-19.0.0\bin\native-image.cmd -jar jPatcher.jar --no-fallback --initialize-at-run-time=sun.awt.AWTAutoShutdown,sun.java2d.opengl.OGLRenderQueue` Now I get: ``` Error: Unsupported features in 4 methods Detailed message: Error: No instances are allowed in the image heap for a class that is initialized or reinitialized at image runtime: sun.java2d.opengl.OGLRenderQueue. Try marking this class for build-time initialization with --initialize-at-build-time=sun.java2d.opengl.OGLRenderQueue Trace: object sun.java2d.opengl.OGLMaskBlit object sun.java2d.loops.GraphicsPrimitive[] field sun.java2d.loops.GraphicsPrimitiveMgr.primitives Error: No instances are allowed in the image heap for a class that is initialized or reinitialized at image runtime: sun.java2d.opengl.OGLRenderQueue. Try marking this class for build-time initialization with --initialize-at-build-time=sun.java2d.opengl.OGLRenderQueue Trace: object sun.java2d.opengl.OGLMaskFill object sun.java2d.loops.GraphicsPrimitive[] field sun.java2d.loops.GraphicsPrimitiveMgr.primitives Error: No instances are allowed in the image heap for a class that is initialized or reinitialized at image runtime: sun.java2d.opengl.OGLRenderQueue. Try marking this class for build-time initialization with --initialize-at-build-time=sun.java2d.opengl.OGLRenderQueue Trace: object sun.java2d.opengl.OGLRenderer field sun.java2d.opengl.OGLSurfaceData.oglRenderPipe Error: No instances are allowed in the image heap for a class that is initialized or reinitialized at image runtime: sun.java2d.opengl.OGLRenderQueue. Try marking this class for build-time initialization with --initialize-at-build-time=sun.java2d.opengl.OGLRenderQueue Trace: object sun.java2d.opengl.OGLTextRenderer field sun.java2d.opengl.OGLSurfaceData.oglTextPipe ``` What do I do now? It's saying to mark `sun.java2d.opengl.OGLRenderQueue` for initialization at run time when it's marked for initialization at build time and it says to mark it for initialization at build time when it's marked for initialization at run time.

Can you disable JME settings window and see if that error happens again?

Edit:
Set this to false

github.com

jMonkeyEngine/jmonkeyengine/blob/0fd70b81c93adde86a3590db49725a0c68fc3618/jme3-core/src/main/java/com/jme3/app/SimpleApplication.java#L184


      
          
          public boolean isShowSettings() {
              return showSettings;
          }
          
          /**
           * Toggles settings window to display at start-up
           * @param showSettings Sets true/false
           *
           */
          public void setShowSettings(boolean showSettings) {
              this.showSettings = showSettings;
          }
          
          /**
           *  Creates the font that will be set to the guiFont field
           *  and subsequently set as the font for the stats text.
           */
          protected BitmapFont loadGuiFont() {
              return assetManager.loadFont("Interface/Fonts/Default.fnt");
          }

mosheduminer · November 1, 2020, 7:50pm

No, I no longer get that error!
However, I now get a NullPointerException. I think that was referenced in one of the issues I linked.

Ali_RS · November 1, 2020, 7:54pm

May you put the whole stack trace?

mosheduminer · November 1, 2020, 7:55pm

Yes, sorry.

Nov 01, 2020 2:49:20 PM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.3.2-stable
 * Branch: HEAD
 * Git Hash: 1a05e3f
 * Build Date: 2020-04-27
Exception in thread "jME3 Main" java.lang.ExceptionInInitializerError
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
        at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:212)
        at java.lang.Thread.run(Thread.java:834)
        at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:517)
        at com.oracle.svm.core.windows.WindowsJavaThreads.osThreadStartRoutine(WindowsJavaThreads.java:138)
Caused by: java.lang.NullPointerException
        at com.oracle.svm.jni.functions.JNIFunctions$Support.getMethodID(JNIFunctions.java:1084)
        at com.oracle.svm.jni.functions.JNIFunctions.GetMethodID(JNIFunctions.java:391)
        at com.oracle.svm.jni.JNIOnLoadFunctionPointer.invoke(JNILibraryInitializer.java)
        at com.oracle.svm.jni.JNILibraryInitializer.callOnLoadFunction(JNILibraryInitializer.java:72)
        at com.oracle.svm.jni.JNILibraryInitializer.initialize(JNILibraryInitializer.java:127)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.addLibrary(NativeLibrarySupport.java:185)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibrary0(NativeLibrarySupport.java:141)
        at com.oracle.svm.core.jdk.NativeLibrarySupport.loadLibrary(NativeLibrarySupport.java:102)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:228)
        at java.lang.Runtime.load0(Runtime.java:768)
        at java.lang.Runtime.load(Runtime.java:244)
        at java.lang.System.load(System.java:358)
        at org.lwjgl.Sys$1.run(Sys.java:70)
        at java.security.AccessController.doPrivileged(AccessController.java:83)
        at org.lwjgl.Sys.doLoadLibrary(Sys.java:66)
        at org.lwjgl.Sys.loadLibrary(Sys.java:87)
        at org.lwjgl.Sys.<clinit>(Sys.java:117)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
        at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
        ... 4 more

Ali_RS · November 1, 2020, 8:02pm

Are you using --initialize-at-build-time flag?

mosheduminer · November 1, 2020, 8:08pm

No, my full command is native-image -jar ./build/libs/hello_world-0.1-SNAPSHOT-all.jar --allow-incomplete-classpath -H:NativeLinkerOption=prefs.lib. However, I believe that by default all classes are initialized at build-time. I don’t know enough about JME to start modifying that behavior.

Edit:
I get the NPE at runtime though…

Ali_RS · November 1, 2020, 8:13pm

Do not know about it but based on the doc

The whole class hierarchy can be initialized at build time by passing --initialize-at-build-time on the command line.

Can you add that flag and see if anything changes?

mosheduminer · November 1, 2020, 8:18pm

After adding that flag, it unfortunately now fails with:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x0000000000000000, pid=10220, tid=14228
#
# JRE version: OpenJDK Runtime Environment GraalVM CE 20.2.0 (11.0.8+10) (build 11.0.8+10-jvmci-20.2-b03)
# Java VM: OpenJDK 64-Bit Server VM GraalVM CE 20.2.0 (11.0.8+10-jvmci-20.2-b03, mixed mode, tiered, jvmci, jvmci compiler, compressed oops, parallel gc, windows-amd64)
# Problematic frame:
# C  0x0000000000000000
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\moshe\Desktop\Foolish_Wisdom\kotlin\jmonkey\hello_world\hs_err_pid10220.log
#
# If you would like to submit a bug report, please visit:
#   https://github.com/oracle/graal/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Error: Image build request failed with exit status 1

EDIT:
The graal bug tracker suggests building on the latest dev distribution of graal before filing a report. Will try that now.

EDIT 2:
The filed issue is here.

mosheduminer · November 1, 2020, 9:01pm

By the way, with regards to awt, it appears there was a commit to add support for it two days ago:
[GR-25755] Add AWT and Swing to JDK 11.

Ali_RS · November 2, 2020, 5:18am

Just a blind guess

Looking at the error log looks like it has something to do with OpenCL.

Can you add this to delay OpenCL initializing

--initialize-at-run-time=org.lwjgl.opencl

mosheduminer · November 2, 2020, 9:35am

I added this flag --initialize-at-run-time=org.lwjgl.opencl,org.lwjgl.openal,org.lwjgl.opengl as the native-image utility was throwing errors about initialization of these classes. I also tested together with the --initialize-at-build-time flag (combined with the other flag, it did not fail as it did before). It compiled successfully, but resulted in the same NPE at runtime.

EDIT: actually, using --initialize-at-build-time results in various awt issues (there stil hasn’t been a dev release since the commit with awt support).

EDIT 2: the awt issues (mentioned in the previous edit) are despite the fact that settings are set to disabled in the code, as suggested above in this thread.

8Keep123 · November 3, 2020, 12:22am

I think you also can’t cross compile with graal. Which is pretty annoying too, that’s a reason many use java. See [native-image] Cross compilation support? · Issue #407 · oracle/graal · GitHub

tlf30 · November 15, 2020, 1:08am

OK, just attempting a small test jme app:

[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] Exception in thread "main" java.lang.ExceptionInInitializerError
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:291)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.system.AppSettings.<clinit>(AppSettings.java:263)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.app.SimpleApplication.start(SimpleApplication.java:115)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at io.tlf.jme.test.Main.main(Main.java:90)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] Caused by: java.lang.NullPointerException: inStream parameter is null
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at java.util.Objects.requireNonNull(Objects.java:246)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at java.util.Properties.load(Properties.java:403)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.jme3.system.JmeVersion.<clinit>(JmeVersion.java:51)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:351)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	at com.oracle.svm.core.classinitialization.ClassInitializationInfo.initialize(ClassInitializationInfo.java:271)
[Sat Nov 14 16:05:41 AKST 2020][INFO] [SUB] 	... 5 more

Same as OP NPE
Still playing with it.