Successes and Challenges using JME3 with OSGi - Classpaths and Native Libraries

As discussed elsewhere we now have a snapshot of the JME3 nightly build (currently based on 2011-12-15) uploaded to the Sonatype “snapshots” fork of the Maven Central repository. My

team is now retesting all our connections to JMonkey using this latest snapshot. In this effort

we are collecting information and producing artifacts which may be helpful to others, and we are also

facing some challenges which others may be able to help with. Because it is difficult to explain

the situation super-concisely, I will spread the information over two or three posts.



We develop and test using JME3 for three different target environments, using Netbeans + Maven (currently 7.0.1).

  1. Standalone java applications - easiest to develop and test with to get basic features working.

  2. OSGi bundles - for formal deployment into containers with other components, in our case, robot software components. We generally use the OpenGL AWT canvas, wrapped in Swing, and test using the Apache Felix container.

  3. Netbeans Platform UI - (with Netigso) - used to build a comprehensive robotics studio app, in which JME3 is used to deliver an onscreen simulator for a humanoid robot. Here the OSGi component delivering the OpenGL AWT canvas is wrapped in Netbeans UI. Netigso is the Netbeans platform bridge to OSGi, which is discussed here http://wiki.netbeans.org/OSGiAndNetBeans and here: http://wiki.apidesign.org/wiki/Netigso



We have wrapped the JME3 code and dependencies into a single OSGi bundle that we are currently delivering as part of the Cogchar project, called ext.bundle.opengl.jmonkey
  • Pom.xml source: https://www.assembla.com/code/cogchar/subversion/nodes/trunk/maven
    /ext.bundle.opengl.jmonkey/pom.xml

  • Binary bundle .jar delivered to Sonatype-snapshots: https://oss.sonatype.org/content/repositories/snapshots/org/cogchar/ext.bundle.opengl.jmonkey/


For the most part, this is all working very well and is spectacular. Yay OpenGL! Yay Robots! Yay JMonkey!
But we have a few challenges, as I said. The big issues with running JME3 (+LWJGL, etc.) under OSGi fall into two basic categories: 1) Classpath issues, and 2) Native Library issues.
I will go into these issues in updates to this post.

Aww shoot, I thought I could edit my posts, but I guess it’s only the replies? Anyhoo, sorry about the lame formatting, but, onward. One more point of introduction is that Normen once mentioned “OSGi bundles will also be provided by us when jME3 is stable”, and I’m hoping that the following discussion is helpful to that effort.



1) Classpath issues

A) Current versions of LWJGL apparently require runtime access to sun.misc.Unsafe. We have marked this dependency as an explicit package import in the pom.xml for ext.bundle.opengl.jmonkey. This import works under Felix, as long as we supply permission for sun.misc to be exposed by the framework, using org.osgi.framework.system.packages.extra=sun.misc as shown at line 210 of this OSGi application pom.xml. However, this technique is not yet working for us under Netigso/Netbeans-Platform. I mention this last point 1) in case it is useful to others, and 2) because we actually had JMonkey+LWJGL (August 2011 vintage) working under Netigso (before some refactoring of our own, yes), and so I’m hoping that either someone can confirm yes/no “the LWJGL dependence on sun.misc.Unsafe is/is-not new since August” and/or point out some other useful configuration point we may have missed (or forgotten and refactored over ;0).



B) JME3 likes to load assets from the classpath, and it’s a beautiful thing! Some assets (e.g. the “Common” tree) are packaged with the JMonkey core libraries, and hence it is natural to export them from our ext.bundle.opengl.jmonkey bundle. We also have our own separate bundle of assets (called org.cogchar.bundle.render.resources). Under OSGi, each bundle gets assigned its own classloader, which basically has authority to load classes+resources only from that bundle. The tricky thing is that JME3 (in its default configuration) in several places uses the current thread’s ContextClassLoader (henceforth TCCL) to pull in assets, so we must make sure that “the right classloader” is set on our thread when we enter JMonkey code that might load assets. We have made a lot of these scenarios work by trial and error, but as we’re now trying to become more formal and tidy our own projects (not duplicating resources, not mixing code and resources without a good reason), we’re needing to understand better exactly what JME3 is doing, which decisions are more settled vs. more volatile, and how we might perhaps better use the configurable AssetManager features. Here is a recap of what I currently see in practice when creating a SimpleApplication to run in a Swing/AWT canvas under OSGi (on Windows 7, x64, JDK6). Note that “iii” below is where we currently are having trouble and would like help.



i) SimpleApplication.initialize() loads resources from the JME3 core library, using the TCCL. We facilitate this by overriding initialize() and setting the TCCL to point to ext.bundle.opengl.jmonkey before calling super.initialize().



ii) SimpleApplication.update() also loads resources from the JME3 core library, using the TCCL. Presumably this only happens on the first call to update(), or some other time that “resources are needed”. Currently we are overriding update() and switching the TCCL before every call to super.update(), which doesn’t feel real good, but it is working.



(Side note about i & ii: Strangely the TCCL during these callbacks does not seem to be the same as what is

on the parent thread when we call startCanvas(), although…I thought it used to be as of September).



iii) We override simpleInitApp(), and read in our own assets from org.cogchar.bundle.render.resources. We set the TCCL to point to the classloader for that bundle, and read in an Ogre mesh model (currently using a copy of Sinbad from 20110917 as well as our own models). Here is the trouble: The model itself is located and loaded, and is rendered OK. But its subordinate models and materials are not found. We see:



INFO [LWJGL Renderer Thread] (JmonkeyAssetLoader.java:55) - Setting thread class loader to local loader: 154.0

Dec 23, 2011 12:02:03 AM com.jme3.scene.plugins.ogre.MeshLoader load

WARNING: Cannot locate jme3dat/models_20110917/sinbad/Sinbad.material for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml

Dec 23, 2011 12:02:03 AM com.jme3.scene.plugins.ogre.MeshLoader applyMaterial WARNING: Cannot locate Sinbad/Eyes for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml

Dec 23, 2011 12:02:03 AM com.jme3.scene.plugins.ogre.MeshLoader applyMaterialWARNING: Cannot locate Sinbad/Body for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml



Dec 23, 2011 12:02:04 AM com.jme3.scene.plugins.ogre.MeshLoader applyMaterial WARNING: Cannot locate Sinbad/Clothes for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml

INFO [LWJGL Renderer Thread] (JmonkeyAssetLoader.java:74) - ********** Finished loading model, restored classLoader to: 129.0



Again, Sinbad.mesh.xml itself is loaded OK, and even displays on the canvas as a red outline, but his siblings/children are dead to him (though those files are in the same package). The same jar of assets (treating the same bundle as just a jar), and same Java code works fine outside OSGi, or if we just lump all JME3+app code and assets into one mega-bundle. But for some reason, the sub-asset-load under OSGi is not working, perhaps because something about the state of the single DesktopAssetManager is not compatible with the TCCL-switching skullduggery? What would be the right approach? Do we need to create a separate AssetManager for each classloader, or register some separate AssetLocator / AssetLoader instances?

You can set a classloader (list) to the assetManager. See that your issues are normal OSGi issues. The same is true for the SDK, the plugins also have their own classloader.

Ugh, I am still missing something(s).



The javadocs for com.jme3.asset.AssetManager.addClassLoader() say: “Adds a ClassLoader that is used to load Classes that are needed for Assets like j3o models. This does not allow loading assets from that classpath, use registerLocator for that.”



I tried adding both of my bundle classloaders with addClassLoader() anyway, but this did not change the results for situation 1.B.iii (can’t find materials for Ogre mesh file). I don’t understand why this particular error is occurring, because from what I can see the com.jme3.scene.plugins.ogre.MeshLoader.load() method class is just asking the assetManager to loadAsset(material-path), so it seems like my TCCL-hack should work for the materials well as the parent. I don’t see where any additional threads are being used (references to ThreadingManager in DesktopAssetManager are commented out). The most important error, it appears, is this one, but casual reading of the code does not reveal how this could be happening:



WARNING: Cannot locate jme3dat/models_20110917/sinbad/Sinbad.material for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml



I am willing to try to register my own AssetLocators, however, I don’t see a good class to use as an extension point, since com.jme3.asset.plugins.ClasspathLocator does not accept a classloader argument. Instead it just uses good old Thread.currentThread().getContextClassLoader() [line 82]. I think it would be useful to allow users to explicitly set classLoaders on the ClasspathLocator (which is what I am currently doing indirectly by hacking the TCCL).



So my questions (edit - added #3) are:

Q.1.B.iii.1) Why is the MeshLoader’s material sub-load failing? Where is that code using something other than TCCL, in the default Desktop.cfg setup?

Q.1.B..2) Where can additional ClassLoaders for assets actually be plugged in to the AssetManager system?

Q.1.B.
.3) Do we agree that com.jme3.asset.plugins.ClasspathLocator should allow client to set explicit ClassLoader(s), as alternative to TCCL?

You can also add all kinds of (even self-made) locators to the assetManager. Just add a jar locator to the url of the jar in the other module.

Thanks Normen. By “jar locator” you are suggesting what is now called com.jme3.asset.plugins.ZipLocator, right? That requires accessing the file system, which is kinda cumbersome (raising issues of security, portability, permissions) when I know that these assets are already available on my bundle classpaths, and when they already load fine with same code in non-OSGi environments (where the app may have a different file tree, but the same total set of relevant classpath entries). I am more inclined to write my own ExplicitClassLoaderLocator, unless+until we agree on Q.1.B._.3 above (which I added after the initial reply posting). I guess I am now torn between doing that and debugging the MeshLoader to find out where it is bypassing/confounding the TCCL.

You can use the url locator with jar urls.

It seems that com.jme3.asset.plugins.UrlLocator can in fact be used in a way that is tidier than switching the TCCL, at least under my current OSGi configuration. I think it would be cleaner still to provide more flexible Classloader-based loading, but I will go ahead with a UrlLocator approach unless/until I get stuck again.



On the Netigso vs. sun.misc.Unsafe front, (Classpath problem #1.A above), I summarized the problem for the Netigso mailing list, and I’m awaiting responses there.

Misleading warnings due to swallowed exceptions in loading implied sub-resources of materials.

By registering my own extended version of ClasspathAssetLocator I was able to discover why we couldn’t load “Sinbad.material” using the TCCL technique. The problem was that the Sinbad.material asset was being located and partially-loaded, despite this message:



WARNING: Cannot locate jme3dat/models_20110917/sinbad/Sinbad.material for model jme3dat/models_20110917/sinbad/Sinbad.mesh.xml



…but, in the process of loading the Sinbad.material, something in jME3 material-loading code stack was swallowing an exception while trying to load hardcoded resources (referred to in MaterialLoader.java, MTLLoader.java, etc.) like Common/MatDefs/Light/Lighting.j3md

…which is a resource we are supplying from a different bundle, which was not at that time on the TCCL-classpath. There was no error printed for that resource, but instead we got the misleading “Cannot locate” warning for the Sinbad.material itself. That’s what threw me off track, and led to question 1.B.iii above.



One instance per thread of each registered AssetLocator class

There is multiplexing of locators by thread going on in com.jme3.asset.ImplHandler, meaning that multiple instances of each registered locator class may be constructed, but this scheme seems to be working in a reasonable way and not compounding any of my other challenges, yet. I guess it might make the task of creating FlexibleClasspathLocator a little harder, since instance data in the locators is not automatically shared across threads. Each new instance of FlexibleClasspathLocator created by ImplHandler might have to ask some registry (which could be provided by an OSGi service) “what classloaders should I check with?”, or we could instead register one subclass (not instance) of FlexibleClasspathLocator for each bundle that supplies asset resources. Perhaps the latter is more naturally compatible with the existing DesktopAssetManager.class + Desktop.cfg in jME3.



Reviewing overall status of the problem tree:

Q.1.A - hoping for help from Netigso experts.

Q.1.B.iii.1 - is now answered by above pararaphs.

Q.1.B..2 - To plug in additional ClassLoaders for asset loading purposes (other than by hacking TCCL), one must create and register a custom locator, and duplicate the functionality of the com.jme3.asset.plugins.ClasspathLocator, because that class is not structured to support extension (the whole implementation is in the “locate()” method). Locator classes may be instantiated by multiple threads, leading to challenges covered in paragraph just above.

Q.1.B.
.3) I still feel the FlexibleClasspathLocator is desirable, but I can probably live without it for awhile, and just register a UrlLocator for each relevant URL returned by OSGi Bundle.getEntry("/"). The full set of implications of this approach is not yet clear.

Q.2 - Native Library issues under OSGi - we have not yet gotten into discussion of these issues.

@stub22:

Read the above paragraph take me a long time to understand, I’ve run in the same direction for a while now… but here is some thing I thought about it:



Your team want to design the SDK to be live with an “OSGi-container”:



Start with SDK and Netbean

— [Feature] Hot deploy classes and bundle things up

— [Use] Netigso solution

— [Problems] ??



Then come to the Engine

— [Feature] Hot asset deploy with help of AssetLoader

— [Use] Apache Felix

----[Problems]


> with the URLLocator ?
> Feel something not right with OSGi Bundle in case of a JME game?

I just want Hot Asset deploy with help of AssetLoader, and still feel kind of hard doing this in JME for now...

Beside, Hot deploy classes is far from complete
Reason:
1) Because of the lack of support for this feature in the Community ( in form of Open Source)..
2) Over-loaded for nowaday online game to have such an update-able system...
As in my POV are some MMOs of my game company: the hot deloy feature (update things) is supported in form of a separate java application which simply replace the current .jar,exe with lastest .jar,exe and run the game again ! :p ... Why we still do these ugly things in the era of OSGi, because you know, the update-able system (classes replacement is over-loaded)

P/s: Can you please provide more infomation about your project and at best the sample source code, it will save a lot of developer here to survive from so call "Challenges using JME3 with OSGi" :p

Thanks and really appreciated!

Read back a topic from 2 year ago, I have some ideas of WHY SHOULD and WHY SHOULDN’T for bundle things… Yes, it’s quite far to complete from my view :?



http://hub.jmonkeyengine.org/groups/features/forum/topic/osgi-bundles/

An update on the OSGi front: The cogchar sub-project with the following maven descriptor:

[xml] <dependency>

<artifactId>ext.bundle.opengl.jmonkey</artifactId>

<groupId>org.cogchar</groupId>

<version>1.0.4</version>

</dependency>[/xml]



…now provides an OSGi-bundled version of the main JME3 libraries, including the native libraries, which are exposed via a Bundle-NativeCode manifest entry. Libraries for all platforms (Windows x32, x64, Linux x32, x64, Apple) are included, but I am only testing with Windows x64.

On Win x64, the native library load is working fine for LWJGL, but failing for OpenAL (which I am not using). I’m not sure about JInput.



LWJGL native library load works well as long as we don’t set the org.lwjgl.librarypath system property!



When accessed in this manner, there is no need for JME3+LWJGL to unpack the libraries into the local filesystem. However, unless

Jme3System.isLowPermissions is set to true, then JME3 will run com.jme3.system.Natives.extractNativeLibs(), which

  1. unpacks libraries to local filesystem and
  2. Places a value into the property org.lwjgl.librarypath, telling LWJGL to load libraries from that location.



    So, when using this ext.bundle.opengl.jmonkey bundle, it is recommended to set Jme3System.isLowPermissions to true during the “createCanvas()” operation. I set the value back to false after the canvas is created and the native library is loaded. So, my user code looks like:



    JmeSystem.setLowPermissions(true);

    // do my create canvas stuff

    bvcApp.initCharPanelWithCanvas(vcp);

    JmeSystem.setLowPermissions(false);

Again, we will provide OSGI bundles when jME3 is released.