3D Sound System

Ok, I think I know what may be causing this.  Looks like there is a potential thread synchronization conflict between the CommandThread and StreamThread when shutting down which could potentially cause this behavior.  I'll see if I can come up with an effective solution and see if it fixes the behavior you described.

Hi,

I'm currently using your sound library and I'm quite successful with it. I have notice only one problem so far. Sound library works fine, but after some time of running (can't say when, but usually 30 - 120 minutes) it just abruptly stops working and make no sound from that moment on until I kill the application and start it over. Moreover after this the application does not want to quit nicely - I must force kill it. It prints this:



SoundSystem shutting down…

Error in class 'LibraryLWJGLOpenAL'

    Stream thread did not die!

Ignoring errors… continuing clean-up.




This message is printed on the quitting of application - not when the abrupt stop occurs. And it is printed only if the abrupt stop occurs before - normally it closes itself nicely.

(Even though the message encourages me to wait, it does not close itself even after few minutes)



Does anyone encounter similar problem?

Thanks for advices

not yet, does this happen with both lwjg and java software sound device?

I've had an application using SoundSystem running for many many hours without issue.  It wasn't playing audio the entire time, however…



Did you track down that error in the code?

gorgor said:
Sound library works fine, but after some time of running (can't say when, but usually 30 - 120 minutes) it just abruptly stops working and make no sound from that moment on until I kill the application and start it over.

Sorry I haven't been on the forums in a while - I've been swamped with work and school.  I haven't experienced this particular problem before myself, and none of my testers or other users have reported it yet.  My initial analysis, however is that it sounds like a thread synchronization problem to me - like the StreamThread is waiting for the CommandThread and vice versa, or perhaps one is sleeping at the wrong time or needs to be interrupted.  I did find another potential synchronization issue in the code, as I mentioned in one of my previous posts, so perhaps it is also causing the behavior you described.  One of the things I've been fighting with since I started this project is the multi-threaded infrastructure.  The library has steadily improved over time, and it is quite stable now, but as people report rare bugs like this, I usually end up finding some little potential conflict that requires fixing, so I expect that is the case here as well.

Do you experience this behavior in only one application, or would it be possible to create a simple test-case that you could upload for me to analyze and use as a test-case for debugging?  I'm hoping to have a little more free time for programming in upcoming weeks (I just finished a really tough class that was taking up all my free time).  If I can't reproduce the problem myself from your test-case (or if you can't create a test-case), then I'll work with you one-on-one to track down and debug the problem before the next release.

Thanks for fast replies… (unlike me:))



not yet, does this happen with both lwjg and java software sound device?

that's good idea.. I've not tried it with java sound, only lwjgl.

I do not know how rare it is, but it can be reliably reproduced on my machine with my application by playing it properly for an hour or so.
It however may be bug on my part - or most probably is taking circumstances that I'm the only one encountering it:)
I myself uses few threads, so maybe I screw it up somewhere. However all calls of SoundSystem is done from one thread only.


Do you experience this behavior in only one application, or would it be possible to create a simple test-case that you could upload for me to analyze and use as a test-case for debugging?

I did not tried to make the test-case yet. And I do not uses SoundSystem in any other project yet (I have no other project yet:)). So I can't say if I'll be able to reproduce it elsewhere:(

I'll post another info once I found something..
Thanks so far to all

I haven't been able to reproduce the problem here.  Have you had any luck creating a test-case, gorgor?  I'll go through the code again with a fine-tooth comb to look for any potential thread synchronization issues I can see.  I'm just about ready to post the next release, but I'd like to at least attempt to address this bug first.

Hi Paul,

I was not able to reproduce it in a test-case.

I also tried the idea of trying java sound instead of lwjgl and as expected problem disappeared…



But I have another question…

When I use your library and I kill my application pressing Alt-F4 on Windows or Cmd-Q on Mac application gets killed (window is closed) however the process still runs (process can be found in TaskManager(Win)/Activity Monitor(OS X)) and the music is still played and can be heard.

Thanks for advices

There is a cleanup/Shutdown function in the SoundSystem, call that when you want to exit.

Yea I know about the SoundSystem.cleanup() function which I call when my exit button is clicked. But the problem is when no exit button is clicked and instead Alt-F4 is pressed for whatever reason…

gorgor said:

Yea I know about the SoundSystem.cleanup() function which I call when my exit button is clicked. But the problem is when no exit button is clicked and instead Alt-F4 is pressed for whatever reason..

I believe the reason for this is because of the link between Java and native code when dealing with OpenAL.  Killing Java before it can tell the native OpenAL to shut down results in artifacts that manifest themselves in a variety of ways, from a streaming source continuing to play what is left in its queue, to a half-second of a sound effect repeating indefinitely, to the "blue screen of death" in Windows.  This is an extreme annoyance for sure, but I have not been able to find any way around the necessary call to cleanup.  I'll bring it up with the LWJGL folks again to see if they have any new suggestions.  If there were some way to detect that Alt-F4 was pressed (or that the browser was closed) and to force the shut-down to wait for a couple seconds so I can safely run cleanup..  anyone know how to do that?

Also note, there is a similar bug when playing MIDI, where the last note may continue to play if the applet/application is shut down abruptly without a call to cleanup.

Thanks for clarification…

It seems that I found the solution… I found that each GameState has method cleanup() which is called when the game is closing, so I put cleaning the SoundSystem up in cleanup() method of my MainMenuState…

it works for me and it works right now…

Hi



I'd like to use this sound system in my game and I already read through all the pages of this thread. But I was unable to find a way to integrate it proberly into jme (jme2 in my case). I tried to use LibraryJavaSound, and at least it does not throw exceptions and it also plays the sounds, but it doesn't consider the 3D coordinates I give to it (I think it's not meant to, right?) Using LibraryLWJGLOpenAL doesn't work, I get this:

Exception in thread "Thread-12" java.lang.IllegalStateException: Only one OpenAL context may be instantiated at any one time.



I found the same exception posted here several pages ago, but I didn't find any solution to that (I hope I didn't miss it *g*). Is there a solution? Would LibraryLWJGLOpenAL use the 3D coordinates right?

Nice work though :)
Wasserleiche said:

I tried to use LibraryJavaSound, and at least it does not throw exceptions and it also plays the sounds, but it doesn't consider the 3D coordinates I give to it (I think it's not meant to, right?)

It should consider 3D coordinates and play with the proper volume+panning (unless it prints out a message like "Pan Control Not Available").  You may not be using the correct settings when creating your sources.  For example, there is no "standard scale" for any scene (the number 0.2f could represent a centimeter or a light-year), so you might have to play around with the attenuation to better match the scale of your 3D scene (the rolloff-factor if using logarithmic attenuation).  Could you write a simple test case and post it here?  I'll determine if there is a bug or what you need to change to make it work.

Wasserleiche said:
Using LibraryLWJGLOpenAL doesn't work, I get this:

Exception in thread "Thread-12" java.lang.IllegalStateException: Only one OpenAL context may be instantiated at any one time.


That either means you tried to create two instances of SoundSystem or that OpenAL has been instantiated somewhere else in your program (a call to "AL.create();")  There are many ways this could happen (generally using the library incorrectly can result in this, but you should also look into how OpenAL is instantiated by jME's sound methods and make sure you aren't doing that in addition to instantiating the SoundSystem.).  If you are having trouble tracking this down, one way might be to start with a simple test case that works and start adding in jME things to it until you get the exception.  That should help you find where else OpenAL is being instantiated.


Wasserleiche said:
Is there a solution? Would LibraryLWJGLOpenAL use the 3D coordinates right?

All the Library plug-ins should.

Thanks for your quick reply!

I tracked down the AL.create() call to initSystem() in StandardGame. So I simply disabled the music and SFX via StandardGame.getSettings(). Now AL is not initialized there, so the SoundSystem is loading properly. However, I now get this Exception when I try to play a sound in the game:


Checking if LWJGL OpenAL is compatible...
    ...yes

Starting up SoundSystem...
Initializing LWJGL OpenAL
    (The LWJGL binding of OpenAL.  For more information, see http://www.lwjgl.org)
OpenAL initialized.

Exception in thread "Thread-12" java.lang.IllegalArgumentException: ByteBuffer is not direct
   at org.lwjgl.BufferChecks.checkDirect(BufferChecks.java:98)
   at org.lwjgl.openal.AL10.alBufferData(AL10.java:1042)
   at paulscode.sound.libraries.LibraryLWJGLOpenAL.loadSound(LibraryLWJGLOpenAL.java:434)
   at paulscode.sound.libraries.LibraryLWJGLOpenAL.quickPlay(LibraryLWJGLOpenAL.java:612)
   at paulscode.sound.SoundSystem.CommandQuickPlay(SoundSystem.java:1761)
   at paulscode.sound.SoundSystem.CommandQueue(SoundSystem.java:2249)
   at paulscode.sound.CommandThread.run(CommandThread.java:121)



Above the stacktrace you can see that everything is set up correctly. Here is the code I'm using to play a sound:


public void loadSFXTrack(String file, Vector3f pos) {
   URL url = ModelImporter.getURL(file, ResourceLocatorTool.TYPE_AUDIO);
   system.quickPlay(false, url, file, false, pos.x, pos.y, pos.z, SFX_MODEL, SoundSystemConfig.getDefaultRolloff());
}



And here my initalization code:


Class<?> libraryType = Library.class;
      
if(SoundSystem.libraryCompatible(LibraryLWJGLOpenAL.class))
   libraryType = LibraryLWJGLOpenAL.class;
else if(SoundSystem.libraryCompatible(LibraryJavaSound.class))
   libraryType = LibraryJavaSound.class;
      
try {
   system = new SoundSystem(libraryType);
   SoundSystemConfig.setCodec("wav", CodecWav.class);
} catch(Exception e) {
   e.printStackTrace();
}



Of course the sound I want to play is a *.wav file. I was able to play it with LibraryJavaSound. Any idea whats going wrong?

Exception in thread "Thread-12" java.lang.IllegalArgumentException: ByteBuffer is not direct
   at org.lwjgl.BufferChecks.checkDirect(BufferChecks.java:98)
   at org.lwjgl.openal.AL10.alBufferData(AL10.java:1042)
   at paulscode.sound.libraries.LibraryLWJGLOpenAL.loadSound(LibraryLWJGLOpenAL.java:434)
   at paulscode.sound.libraries.LibraryLWJGLOpenAL.quickPlay(LibraryLWJGLOpenAL.java:612)
   at paulscode.sound.SoundSystem.CommandQuickPlay(SoundSystem.java:1761)
   at paulscode.sound.SoundSystem.CommandQueue(SoundSystem.java:2249)
   at paulscode.sound.CommandThread.run(CommandThread.java:121)



That is the incompatibility that was introduced after LWJGL decided to stop supporting indirect buffers in their binding of OpenAL.  This is fixed in the upcoming release, which I hope to have out sometime this weekend.

If you want to use a temporary fix in the mean time, make the following changes:

in LibraryLWJGLOpenAL.loadSound:

//        AL10.alBufferData( intBuffer.get( 0 ), soundFormat,
//                           ByteBuffer.wrap( buffer.audioData ),
//                           (int) audioFormat.getSampleRate() );
// CHANGE TO:
        AL10.alBufferData( intBuffer.get( 0 ), soundFormat,
                           (ByteBuffer) BufferUtils.createByteBuffer(
                                buffer.audioData.length ).put(
                                    buffer.audioData ).flip(),
                           (int) audioFormat.getSampleRate() );


in ChannelLWJGLOpenAL.preLoadBuffers:

            //byteBuffer = ByteBuffer.wrap( bufferList.get(i), 0,
            //                              bufferList.get(i).length );
            //CHANGE TO:
            byteBuffer = (ByteBuffer) BufferUtils.createByteBuffer(
                   bufferList.get(i).length ).put( bufferList.get( i ) ).flip();


in ChannelLWJGLOpenAL.queueBuffer:

        //ByteBuffer byteBuffer = ByteBuffer.wrap( buffer, 0, buffer.length );
        //CHANGE TO:
        ByteBuffer byteBuffer = (ByteBuffer) BufferUtils.createByteBuffer(
                                           buffer.length ).put( buffer ).flip();


in ChannelLWJGLOpenAL.feeRawAudioData:

        //ByteBuffer byteBuffer = ByteBuffer.wrap( buffer, 0, buffer.length );
        //CHANGE TO:
        ByteBuffer byteBuffer = (ByteBuffer) BufferUtils.createByteBuffer(
                                           buffer.length ).put( buffer ).flip();

I should have known since I read about it… Ok I'll just wait until the next release :slight_smile:

Thanks for your help anyway :slight_smile:

PaulLamb said:
This is fixed in the upcoming release, which I hope to have out sometime this weekend.

The final code clean-up is taking a bit longer than I anticipated.  I'll have this thing out soon..

Sound System, Major Update, Beta Release



Sound System

Sound System jPCT



JavaSound library pluggin

LWJGL OpenAL library pluggin

JOAL library pluggin



WAV codec pluggin

JOgg codec pluggin

JOrbis codec pluggin

IBXM codec pluggin



JavaDoc



I am calling this a beta release, because this has been one of the biggest overhauls of the library yet, and I fully expect there to be bugs.  I’ll post additional updates in the coming days and problems are discovered.  I’m working on updating the tutorials as well.



There have been extensive changes since the last release.  I’ll try and explain all of them quickly.



The big/important changes:



1) I fixed the incompatibility introduced back in LWJGL 2.3 when they stopped supporting indirect buffers.

2) I added a Doppler effect algorithm to LibraryJavaSound using sample-rate controls.  Note: normal sample-rate ranges in JavaSound are 4KHz - 48Khz, so this can effect how well certain sound effects do with the Doppler effect.

3) I implemented a standard interface for Doppler effect, which can now be used with all library plug-ins.

4) I implemented a new stream-listener interface, which enables listening for End of Stream events (most useful for playing a sequence of music clips)

5) I added in an intelligent auto-search for common MIDI synthesizers when the default is missing.  This makes MIDI possible on non-Sun Java versions (such as OpenJDK)

6) I added the ability to create normal sources from raw PCM data.  Most useful for playing procedurally-generated sound effects.

7) I added the ability to check the millisecond position of any playing source

8) I added a Mixer ranking system, and an intelligent Mixer-picking based on the functions compatible with each Mixer.  This allowed an intelligent work-around for the Java Sound “webcam chosen as default Mixer” bug, as well as the fact that non-Sun Java (such as OpenJDK) may not have the “Java Sound Audio Engine” mixer available.



Additional changes:



1) I fixed a reverse-byte-order bug in CodecIBXM which caused some sounds to be loud and scratchy.

2) I fixed a bug where switching libraries caused pre-loaded sounds not to re-load.

3) I fixed various bugs when running 64-bit Java plug-in for Firefox on Linux.  There are still more (see below).

4) I added an error message when attempting to load a missing file from the JAR.  Before it acted as if there were no problem if a file was not present.

5) I added in a workaround for the Java Sound bug where a random InterruptedException is sometimes thrown when trying to access the default MIDI sequencer.

6) I added a varriable to SoundSystemCongfig for specifying which MIDI synthesizer to try first

7) I fixed the message-flooding bug that would happen when Mixer controls are not available.

8) I fixed various potential thread-synchronization bugs.



There are still several know issues with 64-bit Java on Linux which I am currently working on, including:



1) No MIDI

2) Random blocking for a half-second every few seconds

3) No sound from Java Sound when switching from OpenAL to Java Sound on the fly

4) JFrame ‘paint’ method causes system graphics to go wacky when playing JavaSound, and can even result in a complete OS lock-up requiring reset.



Please let me know if you run into any problems, and I will work on correcting them right away.

Sound System, Bug Fixes, New Plug-in



Sound System



JavaSound library pluggin

LWJGL OpenAL library pluggin

JOAL library pluggin



WAV codec pluggin

JOgg codec pluggin

JOrbis codec pluggin

IBXM codec pluggin

JSpeex codec pluggin



JavaDoc



I fixed the new bug I introduced by trying to fix the IBXM reverse-byte-order bug.  While this is a relatively minor fix, I ended up having to make small changes to nearly every package, so be sure to download all the new versions (in other words, mix-and-match at your own peril).



Additionally, I finished the JSpeex codec plug-in.  Speex-encoded files are generally wrapped into a .ogg container file.  (This is what is produced by the binaries you can download from http://www.speex.org/downloads/).  While I was digging around, I noticed that there is an “experimental” capability for JSpeex to wrap the data into .wav files as well.  Since the .wav format is quite simple to read from (much easier to understand than .ogg), I went ahead and made this codec plug-in useable with either format.  By default, it assumes a .ogg container, but I added a method you can call to use speex-encoded .wav files if you prefer instead.  Just a reminder - these aren’t normal .ogg or .wav files (just the header information is), so obviously they will not work with the other codec plug-ins.  I would recommend using a different file extension (like .spx) so you don’t get them mixed up with any other non-speex files.



One more note: Speex can also save into “raw” format (i.e., no header information).  I didn’t make the codec plug-in support this format because it requires the user to define the header information manually, which doesn’t mesh well with SoundSystem.  However, if using the raw format is a requirement for you, let me know and I can look into it more closely to see if there is a way to support it in my codec plug-in.



As always, let me know if you run into any problems, and I will work on correcting them as soon as possible.