Android Audio Issues

Hello there,



I tried to add simple audio effects to my app with minor success.

It plays but for an infinite loop (with playInstance() → weired) and after the second trigger of playInstance the app crashes. :frowning:



After ~ 2 h of debugging + looking at the source code of the android audio renderer, it seems to me that audio is not yet finished?

Is there a wiki page for android what is finished and what not (especially the audio part)?

I really would like to improve it if nobody is working on that now :slight_smile:



thx

Android support is generally “not finished”. If you come to fix anything, patches are welcome, yeah.

So i digged a little deeper the last days:



I fixed the Audio issues and it worked on my phone but i soon realized that this setup might not be the smartest on an android phone!

I dont know much about the internal setup of jme but as far as i can tell, the audio on a desktop is handled by openal and all these AudioStream/AudioBuffer start to make sense.

But on android there are just two options (as far as i can tell): SoundPool and MediaPlayer!

These two already behave like the AndroidAudioRenderer including AudioStream/AudioBuffer.



Additionally AudioData is completly useless cause Android manages the sound data already. Neither can you read from a streamed AudioStream nor can you get the Data from an AudioBuffer.

That is why the current impl. introduces another AudioData class called AndroidAudioData which only holds the AssetKey, so the SoundPool/MediaPlayer can get the name to the asset and read/buffer data themselves.



Maybe on Android it is much better to stick directly to SoundPool/MediaPlayer…



What do you think?

No, the API should be the same on desktop and phone.

So i made it working on my phone!



Limitations:

  • SoundPool does neither give any information about the duration of the audiotrack nor if the track finished playing. So if you playInstance() twice and the first did not finish playing you can only stop the last track reference
  • Playing long Sounds (like Ocean Waves.ogg from the test data) takes the SoundPool a long time (~4 secs) to load the first time! Playing then works quite smooth. It would be nice to be able to tell the audio node to precache it…



    patch



    Let me know what you think
1 Like

I created the patch in the engine directory

For long audio, its best to play it as streaming music (there are flags in the key to specify this).

If the design needs to be changed, perhaps you have any recommendations? We can always change the interface of AudioRenderer, as its never used directly by the user.

@prich



I agree that the SoundPool and MediaPlayer have to be used. Momoko_Fan, the music is always streamed. It just depends whether you use SoundPool(which plays for a coupled of seconds and stored in limited heap memory) or the MediaPlayer which streams to the audio hardware which means that we only read-in a small chunk at a time,

There seems to be a problem with the way jmonkey handles things. prich, Good job with the patch but still needs more work. I say that because of the fact that the AndroidHarness doesn’t handle the changes with phone states. If I turn the phone off, the music still plays.



This is what it shows before I turn it off.



02-12 20:20:33.374: D/PhoneWindow(6247): couldn’t save which view has focus because the focused view com.jme3.input.android.AndroidInput@40552df8 has no id.

02-12 20:20:33.634: I/AndroidHarness(6247): onPause

02-12 20:20:33.644: I/AndroidHarness(6247): onStop

02-12 20:20:33.644: I/AndroidHarness(6247): onDestroy



The AndroidHarness has to pause everything, but it goes on to destroy everything. Then when I turn it on and return to my game



02-12 20:20:36.187: I/AndroidHarness(6247): onStart

02-12 20:20:36.197: I/AndroidHarness(6247): onResume



the music replays and the graphics reset. Huge problem, especially if some one calls.



Thats a huge no no. On other engines I’ve seen that when a phone changes states, the graphics pause along with the music. Some how the AndroidHarness has to handle this better.



Just wanted to throw in my two cents.

Thx for feedback! I will take care of this annoying behavior…



Maybe also opensl es is another solution to improve audio on android even more (suggested by momoko_fan)

after some research this opensl es feature is only available for sdk version >= 9 (= platform version 2.3).



jme3 is aiming to support 2.2 and above? So it won’t be an option i think…

We could use it only on devices with Android 2.3 and higher, if it provides higher quality audio. For now I guess lets get the current system working without issues.

@prich



OpenCore is just fine. With it, the android can play ogg vorbis, mp3, H.264, and MPEG-4 fine. There’s no need for OpenSL ES. The behavior is when the phone executes destroy, you’ll want to pause all the MediaPlayer instances unless the application is not running anymore. On the Video side though, I saw a clever trick for me anyways. The graphics pauses in the pause method by calling in the object and making it wait( Object.wait() ), and make the wait(0) since graphics runs on time itself. Then on the resume, wait becomes false.



One more thing prich. I was just curious as to why the MAX_NUM_CHANNELS is a constant. Some people might want to restrict how simultaneously a sound should play.



soundPool = new SoundPool(MAX_NUM_CHANNELS, AudioManager.STREAM_MUSIC, 0);



I am just curious. Thanks guys for the response.

well about the constant:

I don’t really know why it is a constant, but i carried it with the old code from larynx.



To me it seemed alot to play 16 sounds simultaneous (not music, sounds like a shot of a gun or moving a stone…).



Well i can change it to a static variable but I don’t really see the benefit to control maximum simultaneous streams this way…

I fixed the issue with the app going into background.

It now pauses and restarts the sound again.

=> also when phone owner gets called sound pauses and resumes again after call.

  • I added “wav” to the AndroidAssetManager → android can handle wav.
  • I added a simple sound test into jme3test.android called ‘SimpleSoundTest’ :wink:



    patch
1 Like

@prich



Awesome, core changes that are applied with phone states in mind is a good thing. I would give more feedback but somehow the new nightlies broke my rendering and I can’t run my application. Are you running into any issues yourself. I just want to know if its my phone.

@prich: I applied your patch, thanks.

Yeah shadowlords patch broke it. I tried to track it down after the audio patch but it took me too long and i was tired :wink:



Ill check it out today

@prich i’m having trouble reading audio on Android. I have FileNotFound exceptions in the openFD method.

Where did you place your audio files? were they in the asset directory of your JME project or in android resources?

I placed all the files into my asset folder of the android project.



Assuming your file is monkey.wav in asset directory you load it with the identifier “monkey.wav”