[SOLVED] jmeSurfaceView for android api-31+ not working properly

Troubleshooting android
21

I tried on API32,33 with .ogg and jme3-android-native version as you mentioned, NOPE its not working , still reverting back to old main activity.

2 Likes
22

Alright, here is what happens under the hood…

The GL thread (AudioNodes) through updates calls this stack, and the problem happens here on this ov_clear call which calls the libc int close(int fd) or int fclose(FILE*), in either cases, the final call is assigned to the internal low level int close(int), the GNU documentation states that if another thread has tried to access the resource while closing, an interruption error (similar to java InterruptedException) will be thrown EINTR.

This is the ov_clear function defined in the TremorAndroid zip file, in source file ivorbisfile.c:

/* clear out the OggVorbis_File struct */
int ov_clear(OggVorbis_File *vf){
  if(vf){
    vorbis_dsp_destroy(vf->vd);
    vf->vd=0;
    ogg_stream_destroy(vf->os);
    vorbis_info_clear(&vf->vi);
    vorbis_comment_clear(&vf->vc);
    if(vf->dataoffsets)_ogg_free(vf->dataoffsets);
    if(vf->pcmlengths)_ogg_free(vf->pcmlengths);
    if(vf->serialnos)_ogg_free(vf->serialnos);
    if(vf->offsets)_ogg_free(vf->offsets);
    ogg_sync_destroy(vf->oy);

    if(vf->datasource)(vf->callbacks.close_func)(vf->datasource);
    memset(vf,0,sizeof(*vf));
  }
#ifdef DEBUG_LEAKS
  _VDBG_dump();
#endif
  return 0;
}

Now, the function call is this (vf->callbacks.close_func)(vf->datasource), and we totally agree the syntax of this function call is not clear (as it’s how we call a function through a pointer to function), but i believe there are clearer ways to write this…anyway, the close_func is assigned to be a reference for the int fclose through:

int ov_open(FILE *f,OggVorbis_File *vf,char *initial,long ibytes){
  ov_callbacks callbacks = {
    (size_t (*)(void *, size_t, size_t, void *))  fread,
    (int (*)(void *, ogg_int64_t, int))              _fseek64_wrap,
    (int (*)(void *))                             fclose, 
    (long (*)(void *))                            ftell
  };

  return ov_open_callbacks((void *)f, vf, initial, ibytes, callbacks);
}

Where, the ov_callbacks is just a structure with a function table that is assigned when we open the vorbis file in the ov_open function, and this is the same way JNI-function-tables are assigned, this is the ov_open assigning the plain fclose from libc:

int ov_open(FILE *f,OggVorbis_File *vf,char *initial,long ibytes){
  ov_callbacks callbacks = {
    (size_t (*)(void *, size_t, size_t, void *))  fread,
    (int (*)(void *, ogg_int64_t, int))              _fseek64_wrap,
    (int (*)(void *))                             fclose,
    (long (*)(void *))                            ftell
  };

  return ov_open_callbacks((void *)f, vf, initial, ibytes, callbacks);
}

I think this problem may arise from the user code, @Anuj_Topiwala you may be using AudioNodes in multiple threads or in an async way and at some time, an object stops the audio while other threads are still playing it, so you might please check this again, but idk why it’s only manifested on high android apis only…

And, btw the ParcelFileDescriptor referred on the stacktrace, here:

And i think this is the part in jme code:

So, the problem may lie around this part, if the user-code is not the cause:

Here is a provisional testing strategy:

  1. Run a simple single threaded audio state test to exclude user-code problems.
  2. If we excluded the user code problems, the problem may be on the NativeVorbisLoader from closing the NativeVorbisFile before closing the AssetFileDescriptor, the AssetFileDescriptor just fetches the file descriptor (an integer number representing a file in Unix) and the offsets and length of the asset, so the code may not need the afd object after line 71, so it should be closed at this point; because the native code opens it again and the native code should be able to fully own this fd and it shouldn’t be owned by any other random resources…
2 Likes
23

This is the jMonkeyEngine stack which triggers the error…

And this is the code on your application @Anuj_Topiwala that triggers this issue, i am now more biased towards that the problem is from the NativeVorbisLoader, however, i still want to exclude user-code problems; because it is simpler than going around modifying the audio code, and anyway, we will need a testcase to test the changes (if the problem was really from the NativeVorbisLoader).

2 Likes
24

I have not tested in single thread yet will do it ASAP. But I am using kotlin and there are multiple coroutines block in my code so will need to make change in structure of code if I need to make audio nodes from single thread and hence what you said is about audio nodes trying to stop from another thread is more likely. Also the reason why only higher android api triggers this error is because android changed fdsan warning level to error Quoting official docs- “Android 11. fdsan now aborts upon detecting an error; the previous behavior was to log a warning and continue.” (Source: Behavior changes: all apps  |  Android Developers)

2 Likes
25

Okay So instead of changing existing audio nodes as it will require changing structure of code I have kept existing audio nodes as .wav which is working and just a new single test audio node with .ogg is created in simpleinit() of main game class that extends SimpleApplication. So here is structure

class gameclass():SimpleApplication() {

…
…
var test:AudioNode?=null
override simpleInitApp() {

…
…
test=AudioNode(assetManager,“Sounds/run.ogg”, AudioData.DataType.Buffer)
test!!.isLooping=true
test!!.play()

/// All coroutines launched post this
// and nowhere else the test node is being used again

}
}

The problem still persist. I replace above with .wav file it runs smoothly.
Please do tell me If I met your conditions of test case as I doubt there is any multi threading call issue in above.

2 Likes
26

Yeah, gj, thank you, i won’t ask for more, but if you would like to open an issue on jMonkeyEngine-github specifying the problem you discovered and the error log you caught, thanks for your contributions.

One more thing, i discovered something suspicious, the ogg native library open function, doesn’t actually open a file :joy:, it utilizes an already opened file passed through the first parameter, idk why their library has a close function (which really closes a file), anyway if that is the case then removing this file.close() call may resolve the problem, and we should modify the misleading names, and afaik while working with FILEs on the Serial4j project, if 2 file objects are pointing to the same file, closing one of them will close the other…so either call afd.close() or file.close(), but not both…and btw in this case OOP is totally misleading, they are just references to the same memory…

EDIT:
Btw, jme overrides the library function table, i previously mentioned, and this is the source code for the ogg library close function now:

It just closes the already opened file using the GNU low level close defined in unistd.h (unix standard library).

EDIT2:
I think the name open need to be refactored to create_vorbis_from_file() or get_vorbis() or get_vorbis_from_file().

EDIT3:
The jMonkeyEngine jni function close calls the ov_clear which destroys the OggVorbisFile data structure and closes the stream file through a call to the function table, so we should take care of releasing the OggVorbisFile structure without closing the actual stream; because it’s owned by the Android ParcelFileDescriptor, so a simple modification to the function table close function and to the name of the java and jni methods is needed…

For example:

  • Replace open() with getVorbisFromFile()
  • Replace close() with releaseResources() or clearResources

And leave the stream portals (open/close) for the Android ParcelFileDescriptor.

And add some documentation as well…

2 Likes
27

I will see if this workaround will work.

1 Like
28

Hello there,

I have prepared a simple ogg testcase on android api-33 and it works fine on my physical device (Redmi note 6 pro stock miui-10 android-9):

@Anuj_Topiwala Are you testing on a physical device or an emulator and what android version…?

Could you please send us your app/build.gradle file ?

EDIT:

The logs on android api-33 and jme-3.6.0-beta2, cannot even find the ParcelFileDescriptor warning ?:


02/16 17:37:10: Launching 'app' on Xiaomi Redmi Note 6 Pro.
Install successfully finished in 4 s 90 ms.
$ adb shell am start -n "com.scrappers.testjmevorbisloader/com.scrappers.testjmevorbisloader.MainActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER
Connected to process 3753 on device 'xiaomi-redmi_note_6_pro-9f06dcb0'.
Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
I/jmevorbisloade: Late-enabling -Xcheck:jni
I/Perf: Connecting to perf service.
W/jmevorbisloade: Accessing hidden field Landroid/os/Trace;->TRACE_TAG_APP:J (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/os/Trace;->isTagEnabled(J)Z (light greylist, reflection)
D/AppCompatDelegate: Checking for metadata for AppLocalesMetadataHolderService : Service not found
W/jmevorbisloade: Accessing hidden field Landroid/graphics/Insets;->left:I (light greylist, linking)
W/jmevorbisloade: Accessing hidden field Landroid/graphics/Insets;->top:I (light greylist, linking)
W/jmevorbisloade: Accessing hidden field Landroid/graphics/Insets;->right:I (light greylist, linking)
W/jmevorbisloade: Accessing hidden field Landroid/graphics/Insets;->bottom:I (light greylist, linking)
W/jmevorbisloade: Accessing hidden field Landroid/view/WindowInsets;->CONSUMED:Landroid/view/WindowInsets; (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (light greylist, reflection)
I/JmeSurfaceView: Starting a new Game State
W/JmeSystem: JmeDialogsFactory implementation not found.
I/JmeSystem: Running on jMonkeyEngine 3.6.0-beta2
     * Branch: HEAD
     * Git Hash: e71f506
     * Build Date: 2023-02-13
I/OGLESContext: Android Build Version: {0}
I/JmeSurfaceView: Game returns from the idle mode
I/Adreno: QUALCOMM build                   : cf57c9c, I1cb5c4d1cc
    Build Date                       : 09/23/18
    OpenGL ES Shader Compiler Version: EV031.25.03.01
    Local Branch                     : 
    Remote Branch                    : 
    Remote Branch                    : 
    Reconstruct Branch               : 
I/Adreno: Build Config                     : S L 6.0.7 AArch64
I/Adreno: PFP: 0x005ff112, ME: 0x005ff066
I/ConfigStore: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDisplay retrieved: 0
I/ConfigStore: android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasHDRDisplay retrieved: 0
I/OpenGLRenderer: Initialized EGL, version 1.4
D/OpenGLRenderer: Swap behavior 2
W/jmevorbisloade: Accessing hidden method Landroid/view/View;->getViewRootImpl()Landroid/view/ViewRootImpl; (light greylist, reflection)
W/jmevorbisloade: Accessing hidden field Landroid/view/View$AttachInfo;->mVisibleInsets:Landroid/graphics/Rect; (light greylist, reflection)
W/jmevorbisloade: Accessing hidden field Landroid/view/ViewRootImpl;->mAttachInfo:Landroid/view/View$AttachInfo; (light greylist, reflection)
I/JmeSurfaceView: JmeSurfaceView's joined the UI thread
W/jmevorbisloade: Accessing hidden method Landroid/os/Trace;->asyncTraceBegin(JLjava/lang/String;I)V (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/os/Trace;->asyncTraceEnd(JLjava/lang/String;I)V (light greylist, reflection)
I/GLRenderer: OpenGL Renderer Information
     * Vendor: {0}
     * Renderer: {1}
     * OpenGL Version: {2}
     * GLSL Version: {3}
     * Profile: {4}
W/jmevorbisloade: Accessing hidden method Landroid/os/Trace;->traceCounter(JLjava/lang/String;I)V (light greylist, reflection)
W/AssetConfig: Cannot find loader {0}
W/jmevorbisloade: Accessing hidden method Landroid/graphics/FontFamily;-><init>()V (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/graphics/FontFamily;->addFontFromAssetManager(Landroid/content/res/AssetManager;Ljava/lang/String;IZIII[Landroid/graphics/fonts/FontVariationAxis;)Z (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/graphics/FontFamily;->addFontFromBuffer(Ljava/nio/ByteBuffer;I[Landroid/graphics/fonts/FontVariationAxis;II)Z (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/graphics/FontFamily;->freeze()Z (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/graphics/FontFamily;->abortCreation()V (light greylist, reflection)
W/jmevorbisloade: Accessing hidden method Landroid/graphics/Typeface;->createFromFamiliesWithDefault([Landroid/graphics/FontFamily;Ljava/lang/String;II)Landroid/graphics/Typeface; (light greylist, reflection)
W/AssetConfig: Cannot find loader {0}
I/chatty: uid=10159(com.scrappers.testjmevorbisloader) GLThread 3368 identical 6 lines
W/AssetConfig: Cannot find loader {0}
I/AndroidJoyInput: loading joysticks for {0}
I/droidJoystickJoyInput14: loading Joystick devices
D/openal: [ALSOFT] (II) Initializing library v1.21.1-ae4eacf1 HEAD
D/openal: [ALSOFT] (II) Supported backends: opensl, null, wave
D/openal: [ALSOFT] (II) Loading config /etc/openal/alsoft.conf...
D/openal: [ALSOFT] (II) Loading config /etc/xdg/alsoft.conf...
D/openal: [ALSOFT] (II) Got binary: /system/bin, app_process64
D/openal: [ALSOFT] (II) Loading config /system/bin/alsoft.conf...
D/openal: [ALSOFT] (II) Key disable-cpu-exts not found
D/openal: [ALSOFT] (II) Extensions: -none-
D/openal: [ALSOFT] (II) Key rt-prio not found
D/openal: [ALSOFT] (II) Key resampler not found
D/openal: [ALSOFT] (II) Key trap-al-error not found
D/openal: [ALSOFT] (II) Key trap-alc-error not found
D/openal: [ALSOFT] (II) Key reverb/boost not found
D/openal: [ALSOFT] (II) Key drivers not found
D/openal: [ALSOFT] (II) Initialized backend "opensl"
D/openal: [ALSOFT] (II) Added "opensl" for playback
D/openal: [ALSOFT] (II) Added "opensl" for capture
D/openal: [ALSOFT] (II) Key excludefx not found
D/openal: [ALSOFT] (II) Key default-reverb not found
D/openal: [ALSOFT] (II) Key channels not found
D/openal: [ALSOFT] (II) Key sample-type not found
D/openal: [ALSOFT] (II) Key frequency not found
D/openal: [ALSOFT] (II) Key period_size not found
D/openal: [ALSOFT] (II) Key periods not found
D/openal: [ALSOFT] (II) Key sources not found
D/openal: [ALSOFT] (II) Key slots not found
D/openal: [ALSOFT] (II) Key sends not found
D/openal: [ALSOFT] (II) Key ambi-format not found
D/openal: [ALSOFT] (II) Created device 0x7a66fde490, "OpenSL"
D/openal: [ALSOFT] (II) Key hrtf not found
D/openal: [ALSOFT] (II) Pre-reset: Stereo, Float32, 44100hz, 882 / 2646 buffer
D/: PlayerBase::PlayerBase()
D/: TrackPlayerBase::TrackPlayerBase()
I/libOpenSLES: Emulating old channel mask behavior (ignoring positional mask 0x3, using default mask 0x3 based on channel count of 2)
V/AudioTrack: set(): streamType 3, sampleRate 44100, format 0x1, channelMask 0x3, frameCount 0, flags #104, notificationFrames -3, sessionId 1993, transferType 0, uid -1, pid -1
V/AudioTrack: set() streamType 3 frameCount 0 flags 0104
W/AudioTrack: AUDIO_OUTPUT_FLAG_FAST denied by server; frameCount 0 -> 1772
D/openal: [ALSOFT] (II) Post-reset: Stereo, Int16, 44100hz, 882 / 2646 buffer
D/openal: [ALSOFT] (II) Key stereo-mode not found
D/openal: [ALSOFT] (II) Key cf_level not found
D/openal: [ALSOFT] (II) Key stereo-encoding not found
D/openal: [ALSOFT] (II) Stereo rendering
D/openal: [ALSOFT] (II) Channel config, Main: 3, Real: 2
D/openal: [ALSOFT] (II) Allocating 5 channels, 20480 bytes
D/openal: [ALSOFT] (II) Enabling single-band first-order ambisonic decoder
D/openal: [ALSOFT] (II) Max sources: 256 (255 + 1), effect slots: 64, sends: 2
D/openal: [ALSOFT] (II) Key dither not found
D/openal: [ALSOFT] (II) Key dither-depth not found
D/openal: [ALSOFT] (II) Dithering enabled (16-bit, 32768)
D/openal: [ALSOFT] (II) Key output-limiter not found
D/openal: [ALSOFT] (II) Output limiter enabled, -0.0005dB limit
D/openal: [ALSOFT] (II) Fixed device latency: 997732ns
E/openal: [ALSOFT] (EE) Failed to set real-time priority for thread: Operation not permitted (1)
D/openal: [ALSOFT] (II) Increasing allocated voices to 256
D/openal: [ALSOFT] (II) Key volume-adjust not found
D/openal: [ALSOFT] (II) Created context 0x7a66d2b710
I/ALAudioRenderer: Audio Renderer Information
     * Device: {0}
     * Vendor: {1}
     * Renderer: {2}
     * Version: {3}
     * Supported channels: {4}
     * ALC extensions: {5}
     * AL extensions: {6}
I/ALAudioRenderer: Audio effect extension version: {0}.{1}
I/ALAudioRenderer: Audio max auxiliary sends: {0}
V/AudioTrack: obtainBuffer(590) returned 1772 = 590 + 1182 err 0
V/AudioTrack: obtainBuffer(590) returned 1182 = 590 + 592 err 0
1 Like
29

Could you test it with the API 33 emulator?

1 Like
30

I have tried it now and encountered the error, i did the workaround provided above, and it seems to work (the application runs), but i cannot test the sound, it seems there are some problems regarding the sound drivers on the qemu virtualization tool used by the avd…i will have to find a fix for the sound issues first…

1 Like
31

I am also trying to create an Android 13 emulator with AVD with android command-line tools to test this. (this is my first time using AVD;)

May I ask what system image ABI should I use for the emulator? x86, armeabi-v7a or arm64-v8a?

32

Any image will work, jme provides native binaries for all architectures through the LLVM…

1 Like
33

I see, thanks!

1 Like
34

I have only two physical device one with Android 9 mi which I guess is api 28 and moto g stock Android which is api 29 level. But first I test on AVD and cross check on actual device for frame rates. I don’t have higher api physical devices so that done on AVD on Ubuntu host machine.
So did you test on api 33 AVD or just on your physical device ? Because redmi 6 pro is api 29 I guess ?

1 Like
35

I tested on my physical device android 9 and it works fine.
I tested on an emulator with API-33 (android 13) and i can reproduce the error.

3 Likes
36

I could also reproduce the crash on AVD Android 13 emulator. However, it crashed only when I set the audio data type for ogg file to AudioData.DataType.Buffer. It worked when I changed it to AudioData.DataType.Stream.

1 Like
37

Filled an issue on GitHub

2 Likes
38

Thanks for opening the issue, i currently have no idea whether i will be able to fix the sound problems on the AVD, if i haven’t succeeded, i will rely on the system logs to test the proposed fix…

1 Like
39

No problem, I can test it for you. :slightly_smiling_face:

And thank you so much for proposing to work on this issue :+1:

1 Like
40

So, the sound is working fine when using the DataType.Stream ?

The stream code is also calling the suspicious file.close().

The difference between streams and buffers is the ability of the buffer to cache the data in a buffer in one call for a faster reload, while the stream relies on updating the system, is that correct ?

At some point in the audio update, does jme calls close explicitly ?

1 Like