[SOLVED] EXCEPTION_ACCESS_VIOLATION after 64 KinematicRagdollControls

I think you hit issue 740:
TestRagdollCharacter crashes · Issue #740 · jMonkeyEngine/jmonkeyengine · GitHub

Could you try again with a recent build of JME from the master branch? If not, you can probably work around the issue by copying the setupSinbad() routine from jme3test.bullet.TestBoneRagdoll and invoking it right after constructing the control.

First, as per issue 740, using 3.2.1, TestBoneRagdoll works fine for me, but TestRagdollCharacter results in an EXCEPTION_ACCESS_VIOLATION. However, after upgrading to the 3.3.0-SNAPSHOT version of native Bullet, TestRagdollCharacter works fine. So, it certainly seems that 740 will be fixed in 3.3.0. :+1:

Second, upgrading to 3.3.0-SNAPSHOT doesn’t fix my Sinbad test app. I still get an EXCEPTION_ACCESS_VIOLATION with exactly the same internal exceptions. However, incorporating the setupSinbad method does fix it, even if I’m still using 3.2.1. So, that’s presumably caused by a different problem.

Third, I’ve managed to reproduce an EXCEPTION_ACCESS_VIOLATION in a similar way to how it happens in my game, but with far less code, as shown below.

    private int count;
    
    @Override
    public void simpleInitApp() {}
    
    @Override
    public void simpleUpdate(float tpf) {
        System.out.println(++count);
        
        BulletAppState bulletAppState = new BulletAppState();
        
        bulletAppState.setThreadingType(ThreadingType.PARALLEL);
        
        stateManager.attach(bulletAppState);
    }

So, it simply adds another BulletAppState with parallel threading every frame. For me, I get an EXCEPTION_ACCESS_VIOLATION after precisely 66 BulletAppStates have been added. Removing the BulletAppStates makes no difference. The internal exceptions (below) are exactly the same as I was getting with my Sinbad test app before. Whether that means it’s caused by the same problem, or whether it’s just coincidence, I wouldn’t know.

Event: 0.411 Thread 0x0000000002708000 Exception <a 'java/lang/NoSuchFieldError': method resolution failed> (0x00000000e04ef408) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\methodHandles.cpp, line 1167]
Event: 0.414 Thread 0x0000000002708000 Exception <a 'java/lang/NoSuchFieldError': method resolution failed> (0x00000000e04fc758) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\methodHandles.cpp, line 1167]
Event: 0.533 Thread 0x0000000059cd9000 Exception <a 'java/io/FileNotFoundException'> (0x00000000e0676fa8) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jni.cpp, line 709]
Event: 0.677 Thread 0x0000000059cd9000 Exception <a 'java/lang/NoClassDefFoundError'> (0x00000000e0716d18) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1394]
Event: 3.114 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e178d1c0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]
Event: 3.114 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e178ed30) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]
Event: 3.114 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e178fe50) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]
Event: 3.115 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e1794f70) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]
Event: 3.115 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e1795bc8) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]
Event: 3.115 Thread 0x0000000059fc4000 Exception <a 'java/security/PrivilegedActionException'> (0x00000000e17967f0) thrown at [C:\re\workspace\8-2-build-windows-amd64-cygwin\jdk8u161\10277\hotspot\src\share\vm\prims\jvm.cpp, line 1390]

Although my game gets different internal exceptions, if whatever is causing the simple test app above to crash could be fixed, it might also fix the issue with my game, because the behaviour is similar. It’s certainly the nearest I’ve been able to get to reproducing the problem with a standalone app.

I’d be interested to know if anyone else gets the same behaviour running the above app. In case it makes a difference, I’m using 64-bit Windows.

1 Like

James:

With the code shown, I was able to reproduce the crash on my 64-bit Windows machine.

OK, that’s good. Obviously, it’s not as easy to see what the issue is with this one. NoSuchFieldError, FileNotFoundException, and NoClassDefFoundError sound as if something’s missing, although I can’t easily tell what that is from looking at the error report file.

Should I raise it as an issue on GitHub?

1 Like

Sure, open an issue.

I did find this in btQuickprof.h:

BT_QUICKPROF_MAX_THREAD_COUNT = 64;

As I suspected, disabling Quickprof resolves the issue.

Interesting! And well spotted! But, how does a max thread count being exceeded result in a NoClassDefFoundError?

Anyway, a max thread count of 64 may well be a perfectly reasonable feature of Bullet, as long as it’s talking about concurrent threads. But, if I repeatedly attach and detach the same BulletAppState (see below), I get the same EXCEPTION_ACCESS_VIOLATION, even though in that case there will never be more than one concurrent BulletAppState.

    private int count;
    private int frame;
    private BulletAppState bulletAppState;
    
    @Override
    public void simpleInitApp() {}
    
    @Override
    public void simpleUpdate(float tpf) {
        if (frame % 4 == 0) {
            System.out.println(++count);
            
            bulletAppState = new BulletAppState();
            
            bulletAppState.setThreadingType(ThreadingType.PARALLEL);
            
            stateManager.attach(bulletAppState);
        } else if (frame % 4 == 2) {
            stateManager.detach(bulletAppState);
        }
        
        frame++;
    }

So, if my understanding of the issue is correct, when I create the issue, I’ll include the above code, and state that the issue is that if the total number of BulletAppStates ever attached exceeds 64, then the JVM crashes, as opposed to the total number of BulletAppStates that are currently attached.

If the total number of BulletAppStates currently attached exceeds 64, then it probably still shouldn’t cause the JVM to crash, but, that’s a separate issue.

I see btQuickprof.h is in Bullet. Does this mean that the issue is in Bullet? Or is it something you reckon we can fix in jMonkeyEngine?

Of course, key to this problem is that the threading type is set to parallel, and in the error report file, I see 66 Java threads: pool-1-thread-1 to pool-66-thread-1. Perhaps the problem is that jMonkeyEngine isn’t terminating the thread when the BulletAppState is detached from the AppStateManager?

1 Like

Issue opened: 64 attached or detached BulletAppStates with parallel threading causes JVM to crash · Issue #928 · jMonkeyEngine/jmonkeyengine · GitHub

1 Like

Given its function, I suspect QuickProf keeps data on every thread it sees, even after that thread is detached.

The planned fix involves recompiling Bullet with the #define that disables QuickProf. Since there’s currently no JNI interface to access QuickProf data, it’s no big loss.

1 Like

You should also add a link to this forum thread in your issue post.

Disregard, already in post, sorry about that.

I’ve just tried out the fix for 928, and I can confirm that my game no longer crashes after 64 ragdolls! :grinning:

Thanks for everything, Stephen. You’ve been a tremendous help. :+1:

1 Like

As you probably know, JME’s KinematicRagdollControl is an unfinished work in progress. It’s been that way for many years. I try to fix any issues I find in it. Even though it’ll be deprecated in JME 3.3, I think such fixes will add value to JME 3.2.2 .

My main project right now is to develop an improved kinematic ragdoll control that works with models other than Sinbad. For the moment I’m targeting JME 3.2.* . I’ll post a forum announcement when it’s ready for testing.

4 Likes

Yes, I noticed that KinematicRagdollControl is now deprecated in the master branch. Is there something somewhere explaining what it’s going to be replaced with?

I didn’t realise there was going to be a 3.2.2. I take it 3.2.2 will also have the fixes for 913 and 928 in it? And presumably those fixes will still be necessary in 3.3.0? PhysicsJoint will still need a native implementation of finalizeNative, and QuickProf will still need to be disabled in case you have more than 64 BulletAppStates.

Yes, when I first came to add ragdolls to my game, KinematicRagdollControl didn’t work with my model. It was falling over when it tried to create the CollisionShapes. I managed to work around it by supplying a separate j3o containing the model’s collision shapes. Took a bit of work to integrate that into KinematicRagdollControl, but I managed it eventually. The rest of KinematicRagdollControl's functionality works perfectly fine with my model though.

1 Like

If you haven’t already, read this topic:
Next release(s) of the Engine

I expect that 3.3.x releases (when they occur) will incorporate all changes currently in master, and that simple bug fixes currently in master (such as 913 and 928) will get back-ported to the v3.2 branch for inclusion in future 3.2.x releases (when they occur). However, I don’t expect to make those decisions and don’t know when those releases will occur.

I’m impressed that you got KRC to work with a model other than Sinbad.

I suspect the main reason KRC is deprecated in master is that it relies heavily on Bone, Skeleton, and SkeletonControl, which are also deprecated. If someone were to rewrite KRC to use Joint, Armature, and SkinningControl instead, there’d be less reason to deprecate it. But as I said, I’m focused on making it work, for me, with the old animation system; I’ve barely looked at the new animation system.