3D Sound System

[PaulLamb]

– This slot is reserved for the most recent working releases –



I have been working on a 3D game sound engine for Java which provides a common interface to various 3rd-party sound and codec libraries. I know JME has its own methods to link with OpenAL, so I hope my posting this here doesn’t offend the developers. At the very least, the sourcecode, which is included in the following links, might be useful for reference. This project and source code is free to use for any project comercial or otherwise.



Downloads:



Sound System Version date: October 23, 2010

The core SoundSystem library, independent from 3rd-party libraries. It is stripped down to the bare essentials, and designed to be easily customizable with various sound library and codec plug-ins. If memory is a concern (such as in an applet) this may be a good option, because it allows you to choose as many or as few plug-ins as you require for your project. NOTE: The core SoundSystem library without any plug-ins is only capable of playing MIDI files. Additional plug-ins should be added for more capabilities. The source code and license are included in the .zip file.



SoundSystem Utils Version date: August 9, 2009

Includes a SoundSystem loader, and an example XML file.





Plug-ins:



JavaSound library plug-in Version date: October 23, 2010

Interface to the Java Sound API. More compatable than OpenAL, but not as high quality and fewer features. This plug-in utilizes JavaSound’s panning and volume control methods to simulate an reasonable-quality 3D sound system. Known bug: quickPlaying sounds will begin playing them at full volume for a split second, before switching to the correct volume. This is a bug with the Java Sound API itself, and therefore beyond my control to correct. An easy workaround is to add 0.02 seconds of silence to the beginning of each sound effect (the free Audacity sound editor works well for this).



LWJGL OpenAL library plug-in Version date: August 24, 2010

Interface to the LWJGL binding of OpenAL. The LWJGL library (http://www.lwjgl.org) is required for this plug-in to work. This library sounds much better than Java Sound, but is not as compatable. I recommend using the JavaSound library plug-in as a backup option. NOTE: Please read the included LWJGL license.



JOAL library plug-in Version date: August 24, 2010

Interface to the JOAL binding of OpenAL. The JOAL library (https://joal.dev.java.net/) is required for this plug-in to work. As mentioned previously, this library sounds much better than Java Sound, but is not as compatable. I recommend using the JavaSound library plug-in as a backup option. NOTE: Please read the included JOAL license.



WAV codec plug-in Version date: October 23, 2010

Adds support for .wav files.



JOgg codec plug-in Version date: August 24, 2010

Adds support for .ogg files using the J-Ogg library. This codec is less compatible than the JOrbis codec, but the license is less restrictive. Sometimes running incompatable .ogg files through a converter will make them compatable. NOTE: Please read the included JOgg license.



JOrbis codec plug-in Version date: November 23, 2010

Adds support for .ogg files using the JOrbis library. More compatible than the JOgg codec, but reads data more slowly (it may not be possible to stream more than one file simultaneously when using this codec). This plug-in is licensed by the LGPL. NOTE: Please read the included LGPL document.



IBXM codec plug-in Version date: August 24, 2010

Adds support for Protracker, Fast Tracker 2, and Scream Tracker 3 (.s2m, .mod, and .xm) files using the IBXM library. File sizes for these formats tend to be quite small, so this may be a good option for background music. This plug-in is based on or using the IBXM library, which is bound by the BSD License. NOTE: Please read the included license document.



JSpeex codec plug-in Version date: August 24, 2010

Adds support for .ogg or .wav files encoded with Speex (a compression optimized for human voice). See http://www.speex.org/ for more information.





Documentation:



JavaDoc Version date: October 23, 2010

Also includes the JavaDocs for SoundSystemJPCT and all library and codec plug-ins, and the utils library.



3D Sound with SoundSystem PDF (download the example programs)

A tutorial-style guide to using the core SoundSystem library (last updated: April 14, 2009).





Demos:



Sound Effects Player (download the Source Code)

Demonstrates library switching on the fly, streaming background music, playing MIDI, and playing multiple sources simultaneously. Last updated August 21, 2010



Bullet / Target Collision (download the Source Code)

Demonstrates the LibraryJavaSound plug-in. Last updated 30 March, 2009



Holy Bouncing Helicopter Balls! (download the Source Code)

(Currently written in jPCT. In the process of porting to jME). Demonstrates moving through a world with multiple sources. Last updated August 21, 2010





What’s new?



- Fixed incompatibility introduced by LWJGL 2.3 not supporting indirect buffers

• Fixed reverse-byte-order bug in CodecIBXM
  
• Fixed bug where switching libraries caused pre-loaded sounds not to re-load
  
• Fixed various bugs in 64-bit Java plug-in for Firefox on Linux
  
• Added error message when attempting to load missing file from the JAR
  
• Added Doppler algorithm to LibraryJavaSound using sample-rate control
  
• Implemented standard interface methods for Doppler effect
  
• Added Doppler interface methods to SoundSystemJPCT with SimpleVector parameters
  
• Implemented new listner interface for End Of Stream events
  
• Added workaround for InterruptedException bug when accessing MIDI sequencer
  
• Added auto-search for common MIDI synthesizers when the default is missing
  
• Added variable to SoundSystemConfig for specifying MIDI synthesizer to try
  
• Added method for creating normal sources from raw PCM audio data
  
• Added method for checking millisecond position of playing sources
  
• Added Mixer ranking system and auto-selection of the most compatible Mixer
  
• Fixed message-flooding bug when Mixer controls are unavailable
  
• Added work-around for Java Sound “webcam chosen as default Mixer” bug
  
• Added work-around for non-Sun Java (eg. OpenJDK) missing “Java Sound” Mixer
  
• Fixed multiple potential thread-synchronization bugs
  
• Fixed a bug where certain types of .ogg files created in versions of Audacity were cut off just before the end of the sample

[Alric]

Interesting… jME's sound system is not a great strength. I'll be interested to see how this works.

[Tobias]

Yeah, this is real interesting… especially the ability to use JavaSound as a fallback. Can you  tell us more about the features?



EDIT: Looked a bit at the javadocs, and I couldn't find a way to set the pitch. Is that possible?

[PaulLamb]

Tobias said:

EDIT: Looked a bit at the javadocs, and I couldn't find a way to set the pitch. Is that possible?

No there is no pitch setting in the SoundSystem library yet, but I can certainly look into adding it.  What would this feature be most likely used for?

Tobias said:

Yeah, this is real interesting... especially the ability to use JavaSound as a fallback. Can you  tell us more about the features?

Sure.  There is also a tutorial guide, which explains things in a little bit more detail (with example programs), but here is a summary of some of the main features:
1) Customizable plug-in infrastructure.  Add as many or as few 3rd-party sound libraries or codecs as you like.
2) Common interface.  You don't have to learn a whole new system for each 3rd-party sound library or codec.
3) Automatic compatibility checking.  SoundSystem will automatically try to load the library pluggins in the order you added them, and if the first one fails, it will try the second, etc.
4) Switching between 3rd-party sound libraries on the fly.
5) Ability to either load an entire clip into memory (for "normal sources") or stream it in chunks ("streaming sources").  The first time you attempt to play a file for a "normal source", if it has not been loaded yet, SoundSystem will automatically load it for you.  As this might cause a slight pause the first time a sound effect is played, there is also a loadSound method.  Clips are loaded into a buffer which is reused for multiple "normal sources", so there are not multiple coppies of audio data floating around.
6) Automatic handling of too many simultaneuos sources.  SoundSystem will iterate through its "channels", and if there are no more available, it goes back and stops the oldest one to play the new one there.  A Source can be specified as a "priority source", which will not be overwritten by a new one.  Specifying all sources as "priority" ensures that all sounds play from beginning to end, but then some sounds might not be able to play if there are too many sounds playing at once.  This gives you control of how you want too many simultaneously playing sources to be handled in your program.
7) Customizable settings.  You can customize numerous things such as the number or size of streaming buffers, number of channels, maximum file size, default attenuation model, etc.
8) Audio transition methods for streaming sources and MIDI.  Lets you queue music sequences or fade out/in between music cuts.
9) Individual source and Master volume control.
10) QuickPlay method that automatically plays a source at the specified position in 3D, then automatically removes the source when it is finished playing.  Lets you "play it and forget it".

[Tobias]

Sounds like a lot of lovely features



I'm thinking of using this in Mad Skills Motocross, have to check its stability first. And find some time for it, as the release is just around the corner. I hate to have to make such a major change so close to release but OpenAL is just too unstable on some systems.



That would put your sound system through a lot of testing on different setups, I believe.



Can I define sounds as being non-3D, like music and other non-positional fx?

No there is no pitch setting in the SoundSystem library yet, but I can certainly look into adding it.  What would this feature be most likely used for?

The engine sound :) ... and I also pitch all sound effects slightly so they sound a bit differently every time they are played.

[Alric]

I also pitch all sound effects slightly so they sound a bit differently every time they are played.

Ditto. I find the suspension of disbelief is a bit better with some variation in there. If you play the exact same sound effect frequently (lots of explosions in my case) it is quickly noticeable.

[PaulLamb]

Tobias said:

Can I define sounds as being non-3D, like music and other non-positional fx?

Yes, sources can be either ambient or positional.

Tobias said:

I also pitch all sound effects slightly so they sound a bit differently every time they are played.

Alric said:

I find the suspension of disbelief is a bit better with some variation in there. If you play the exact same sound effect frequently (lots of explosions in my case) it is quickly noticeable.

Excellent point - I will definitely add this feature then.  I took a quick look at the controls available for JavaSound Clips and SourceDataLines, and it appears there is a SAMPLE_RATE float control type, which I believe could be used to affect the pitch (it would also affect the playback speed as well).  I am going to start coding this right now, so I'll post an update if/when I get it working properly.  For the JavaSound plug-in, I plan to simulate what OpenAL does with the AL_PITCH property, allowing the user to chose a value between 0.5f - 2.0f (where 1.0f is normal pitch), and multiply that number by the normal sample rate to get the new value.  I did some searching, but I don't see any straight-forward way to change the pitch of MIDI playback, so pitch changes will work for everything except MIDI (at least until I decide to write my own MIDI synthesizer).

Tobias said:

That would put your sound system through a lot of testing on different setups, I believe.

Excellent!  Just what I'm hoping for.

--UPDATE--
I finished adding in pitch-changing, and initial tests have gone beautifully.  Using the SAMPLE_RATE control for JavaSound as mentioned above results in exactly the same behavior as using the OpenAL AL_PITCH property.  I am going to run some more extreme tests to make sure this addition is stable, then I will post the updated SoundSystem core and library plug-ins.

[PaulLamb]

I uploaded the updated SoundSystem core and library pluggins.  The links in my initial post are still the same.



Since there are already several people using SoundSystem in their projects, I don't want to break the existing quickPlay method by adding in a parameter for initial pitch if it isn't necessary.  The following can be used instead:

soundSystem.setPitch( soundSystem.quickPlay( ... ), initialPitch );

This works fine on my computer, but if there winds up being latency problems with this method on slower machines, I will go back and change the quickPlay method.

[Tobias]

Using the SAMPLE_RATE control for JavaSound as mentioned above results in exactly the same behavior as using the OpenAL AL_PITCH property.

Sweeet.

[gouessej]

Ok you convinced me to use your library. Can you give me a small example of use? I would like to play a very short sound contained in a OGG file, it is a positional sound (an explosion). Should I create a separate thread to handle the sound?



Did you test your sound system under Mandriva 2009? Someone told me that the sound system of my game does not work at all on it whereas I use only plain JavaSound.



Could you include a fallback to openal-soft in JOAL when used under Linux? It would hugely increase the reliability of your system under Linux, you only need to build JOAL yourself but by replacing one single .so file.

[mulova]

I'd like to try this sound library, too.

[PaulLamb]

gouessej said:

Ok you convinced me to use your library. Can you give me a small example of use? I would like to play a very short sound contained in a OGG file, it is a positional sound (an explosion).

Ok, for short .ogg files, the JOrbis plug-in is the most compatible (the JOgg plug-in seems to be hit-and-miss with short .ogg files).  You would import the following:

import paulscode.sound.SoundSystem;
import paulscode.sound.SoundSystemConfig;
import paulscode.sound.libraries.LibraryJOAL;
import paulscode.sound.libraries.LibraryJavaSound;
import paulscode.sound.codecs.CodecJOrbis;

Next, at the top of your program, you would link with the plug-ins and instantiate the SoundSystem:

        try
        {
            SoundSystemConfig.addLibrary( LibraryJOAL.class );            // first choice
            SoundSystemConfig.addLibrary( LibraryJavaSound.class );  //fallback option
            SoundSystemConfig.setCodec( "ogg", CodecJOrbis.class );  // OGG support
        }
        catch( SoundSystemException e )
        {
            System.err.println("error linking with the pluggins" );
        }
        // Initialize the Sound System:
        mySoundSystem = new SoundSystem();

To play the sound effect to the right:

                mySoundSystem.quickPlay( false, "explosion.ogg", false,
                                         20, 0, 0,
                                         SoundSystemConfig.ATTENUATION_ROLLOFF,
                                         SoundSystemConfig.getDefaultRolloff()
                                        );

And finally, at the bottom of your program shut down the Sound System (important when using OpenAL on some machines):

        mySoundSystem.cleanup();

gouessej said:

Should I create a separate thread to handle the sound?

No, SoundSystem has its own "command queue" and "streaming" thread infrastructure.  I've spent a lot of time making SoundSystem thread safe, so you should be able to initialize, play, and shut down from whatever threads you are using in your game.  If you do notice any thread synchronization problems, be sure to let me know.

gouessej said:

Did you test your sound system under Mandriva 2009? Someone told me that the sound system of my game does not work at all on it :( whereas I use only plain JavaSound.

No, I do not believe I have (at least nobody has reported a problem from that system).

gouessej said:

Could you include a fallback to openal-soft in JOAL when used under Linux? It would hugely increase the reliability of your system under Linux, you only need to build JOAL yourself but by replacing one single .so file.

I would be glad to, but I thought that was done from the end-user's machine (perhaps I am mistaken about this?)  If the only thing changing is JOAL, shouldn't the user just swap out one version of JOAL for another (and I wouldn't change anything in my LibraryJOAL pluggin)?  If you've had some experience with this subject, perhaps you could help me create a new "OpenAL soft" library plug-in (I'd need a description of how to change JOAL to open-al soft, any source code you have related to this, etc.). 

[gouessej]

PaulLamb said:

Ok, for short .ogg files, the JOrbis plug-in is the most compatible (the JOgg plug-in seems to be hit-and-miss with short .ogg files).

Did you update OggInputStream to fix the bug with the short files? Some people on javagaming.org did it and it seems to work.

[PaulLamb]

gouessej said:

[Did you update OggInputStream to fix the bug with the short files? Some people on javagaming.org did it and it seems to work.

OggInputStream is not part of the JOgg library :?  Is there another JOgg library (besides the one at http://www.j-ogg.de), or where did those people obtain OggInputStream from?

[nymon]

PaulLamb said:

The only real drawback with this plug-in is that it is licensed by the LGPL, so if you use it in a project, you must provide the user with a way to switch JOrbis out for newer versions (The easiest way to do this is to keep the JOrbis JAR seperate from your project's jar, and distribute the JOrbis codec plug-in source code with your product.  That way, users can drop in new JOrbis sources and recompile an updated version of the JOrbis codec plug-in).

I didn't know that was specified by a LGPL license. Thats good to know!

[PaulLamb]

nymon said:

I didn't know that was specified by a LGPL license. Thats good to know!

The one thing I dislike about GPL and LGPL licensed stuff is that the language of those licenses is quite legal and very difficult for anyone who is not a lawyer to understand.

One of the best articles I found about the basic considerations for using LGPL libraries in Java can be found at http://www.gnu.org/licenses/lgpl-java.html.  I definitely recommend reading it.

[PaulLamb]

Ok, I added in the new getMixer and setMixer methods, and uploaded the updates.  Changes were applied to the LibraryJavaSound plug-in and to the SoundSystem core (changed a couple of 'private' varriables to 'protected' in the Library class).  These new methods are static, and should be thread safe.  It should also be possible to switch between Mixers on the fly.  I haven't run many tests yet, so let me know if you have any problems.  I'll have time to run more extensive tests this weekend.

[gouessej]

Ok I will use the JOrbis plugin, thanks

[gouessej]

Hi!



Can you add the method loadSound(URL url) in the class SoundSystem please? It would be better and more homogeneous with other Java API… and easier for me.



If I use loadSound, how can I know the source name in order to reuse it later?? Should I use newSource?



Do you use the best mixer with the bigger count of available channels?



I'm reading your tutorial. Thank you for all these stuffs.

[PaulLamb]

gouessej said:

Can you add the method loadSound(URL url) in the class SoundSystem please? It would be better and more homogeneous with other Java API... and easier for me.

I can't do this without breaking several methods.  Currently the filename is used in a hashmap to uniquely identify the audio data from each loaded file.  Allowing URL parameters eliminates this ability, since URL instance do not necessarily have a unique String identifier (the getPath and getQuery methods may return null depending on how the URL was instantiated).  Adding a loadSound(URL) method would require a different method of identifying unique audio data.  I can't think of any way to do this without requiring people currently using SoundSystem to change their code in some way, which I would prefer not to do unless necessary.

Is there a reason you are unable to use the String parameters (other than it being less convenient)?  SoundSystem allows you to load files locally (compiled into the JAR) or from an online location (filename starting with "http://").  If you know how all your URLs are created and that they have actual paths, a workaround would be to use the url.getFile() method as the parameter for SoundSystem.  If your online path uses a different protocol than "http://" ("ftp://", "udp://", "mms://" etc.), then you should be able to change the PREFIX_URL regular expression in the SoundSystemConfig class.

gouessej said:

If I use loadSound, how can I know the source name in order to reuse it later?? Should I use newSource?

Perhaps I should explain how the SoundSystem works here in a little more depth, since the terminology I am using is a bit different than OpenAL or JavaSound, which you may be more familiar with.  In the following description, I used italics for OpenAL/JavaSound terminology, and bold for SoundSystem terminology:

Whenever the SoundSystem initializes it grabs up all the sources that it can (as specified in SoundSystemConfig).  For obvious reasons, I didn't want to have two different entities called "source" (talk about confusing!), so decided refer to these permanant sources as channels instead.  As far as OpenAL/JavaSound is concerned, these are the only sources that exist.  When I talk about SoundSystem sources, I am not talking about the channels.  Instead, I am referring to the peripheral information (sound filename, position, etc) rather than an actual OpenAL/JavaSound source.  Hopefully you can understand the distinction.  As far as the underlying library is concerned, sources are not ever being deleted - there are 32 permanant sources the entire time which never go away (As I mentioned, I call them channels).  When a source is removed, all that is really being removed is the peripheral information: a 3D vector, two Strings, and some floats.  When a new source is created, new peripheral information is created, and as far as the underlying library is concerned, one of those 32 channels merely received new sample rate, volume, panning, etc.  Bytes from the correct sound file are attached/fed into whichever channel is supposed to play that sound, so there is very little overhead involved in switching between different sound effects on a single channel or playing a single sound effect on multiple channels.

So now that I have explained the terminology, to answer your question, loadSound does not create a source.  It loads a file's sound-bytes into memory to be used by sources.  The loadSound method is useful for loading a bunch of files in one place in your program to avoid a potential lag that happens the first time a particular file is encountered (the newSource and quickPlay methods will automatically call loadSound the first time a file is encountered if you haven't called it already).

To create the actual sources, use either quickPlay or newSource (or quickStream, newStreamingSource, or backgroundMusic for streaming music).  The easiest method to use is quickPlay (great for things like lasers, explosions, etc.).  It creates a temporary source, plays it, and automatically removes it when it finishes.  The newSource method is a bit more advanced, giving you more control over your sources (great for things like source management, far-source culling, etc.), and it requires you to remove your sources manually when you are finished with them.

gouessej said:

Do you use the best mixer with the bigger count of available channels?

No, for maximum compatibility (which is really the whole point of using JavaSound instead of OpenAL), I use the "Java Sound Audio Engine" mixer.  Not only is this the most compatible Mixer, it ensures there are volume and pan controls available for use in simulating 3D sound, and that there are up to 32 possible simultaneously open Clips/SourceDataLines.