[solved] Ogg streaming and memory problem

When I prepare about 300 ogg files with method call

     AudioSystem.createAudioTrack(URL, true);

Memory is consumed like it is not stream but just loaded all the sound file into the memory.

What should I do to play streamming ogg files?

This memory problem is too serious.

I'm developing sports game, and it uses many commentator sounds.

But streaming doesn't work at all.

And when a sound is ended, there is no memory release.

Whenever I play new sound, memory is constantly increasing.

Can anyone give me some insight to this problem?

Is it OpanAL Problem?

Can I solve this problem?

Can you post a SimpleGame test case?


Thank you for the attention.  :slight_smile:



Press Space bar, then next ogg file is played.

regardless of streaming or not, the memory is constantly increasing.



package test;

import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;

import com.jme.app.SimpleGame;
import com.jme.input.KeyBindingManager;
import com.jme.input.KeyInput;
import com.jmex.audio.AudioSystem;
import com.jmex.audio.AudioTrack;
import com.jmex.audio.AudioTrack.TrackType;

public class TestOggStreaming extends SimpleGame {
   private static final String   NEXT   = "next";
   private int c;
   private AudioSystem audio;
   private AudioTrack track;
   private String[] paths;
   
   @Override
   protected void simpleInitGame() {
      paths = new String[30];
      for (int i = 0; i < paths.length; i++) {
         paths[i] = "/test/"+i+").ogg";
      }
      
      KeyBindingManager.getKeyBindingManager().add(NEXT, KeyInput.KEY_SPACE);
        audio = AudioSystem.getSystem();
   }

   @Override
   protected void simpleUpdate() {
      if(KeyBindingManager.getKeyBindingManager().isValidCommand(NEXT, false)) {
         playNext();
      }
      
      audio.update();
   }

   @SuppressWarnings("deprecation")
   private void playNext() {
      c++;
      if (c >= paths.length) {
         c = 0;
      }
      URL url = TestOggStreaming.class.getResource(paths[c]);
      if (track != null) {
         track.stop();
         System.gc();
      }
      track = getSFX(url);
      track.play();
   }
   
    private AudioTrack getSFX(URL resource) {
       audio = AudioSystem.getSystem();
//       final boolean stream = FastMath.rand.nextBoolean();
       final boolean stream = true;
        AudioTrack sound = audio.createAudioTrack(resource, stream);
        sound.setType(TrackType.MUSIC);
        sound.setVolume(1f);
        sound.setRelative(false);
        sound.setLooping(false);
        return sound;
    }

   public static void main(String[] args) {
      TestOggStreaming app = new TestOggStreaming();
      app.start();
   }
}



the resource file is from jme/data/sound/test.ogg and it is duplicated 30 times
http://mulova.tistory.com/attachment/cfile1.uf@151DA01649D6DC94545009.zip

Thank you for the fantastic test case!

So, the leak was clearly observable from JConsole. I took a couple of snapshots of the JVM heap using jmap and analyzed the differences between those using jmhat with the -baseline option. I am thinking seriously about starting to use a decent profiler.

The leak chain came from buffers held by a chain of references that ended up at OggInputStream. There were circular references between OpenALStreamedAudioPlayer, OpenALAudioTrack, OpenALStreamedAudioPlayer$PlayerThread.

I think I have solved this with the following:

Added a cleanup() method to AudioTrack, so we can clean the player and null the reference to it:

    public void cleanup() {
       player.cleanup();
       player = null;
    }



To be on the safer side, I added the following to the OpenALStreamedAudioPlayer.cleanup() method. This allows us to null the reference to the stream:


            try {
               if (getStream() != null) getStream().close();
            } catch (IOException es) {
               logger.warning("Could not close audio stream: " + es);
            }
            setStream(null);



Finally, we add a call to the cleanup method of the track when we are done with it (on your test case). Also note that I moved the "c++" incrementing operator to the end of the method.

   private void playNext() {
      if (c >= paths.length) {
         c = 0;
      }
      URL url = TestOggStreaming.class.getResource(paths[c]);
      if (track != null) {
         track.stop();
         track.cleanup();
      }
      track = getSFX(url);
      track.play();
      c++;
   }



After doing these changes, memory doesn't grow without control with your test case. JConsole shows a ver reasonable usage pattern and performing a GC brings the VM to a stable low.

Please test and let me know if it helps. I don't know if I overlooked something. Maybe all this should be done automatically at the stop() method? I will double check and prepare a patch if it works for you too.

Hoep it helps.

Soooo helpful!!!  :-o

I can' t appreciate it enough.

I'll try it tomorrow.

Thanks one more time.

I tried the patch with jConsole.

It shows stable heap usage

but unfortunately, native memory is still increasing. ( shown from Windows Task Manager)

(The changes are that I misused the patch)

Have you checked process memory also?



Anyway, I appreciate it.  :slight_smile:

No, native memory increases for me too.



I had a look into this, as I suspected that AL buffers are not being freed (as it seems to me that memory increases in 1MB chunks?).



However, I have no idea about how to profile this on the native memory usage side :frowning: :(, and I got lost on debugging and profiling :/. And in fact, AL10.alDeleteBuffers is being called for them.



My understanding of JME and AL audio systems is almost nil. Ideally, we should now work on providing an AL test case, trying to reproduce the problem using just AL code from Java (no JME classes), finding what calls are done. More profiling/debugging may help.



I am sorry to fail you, but we need someone more knowledgeable here. If possible, I would be interested in knowing how to analyze this cleanly.


I know there's GLIntercept for OpenGL interception of calls… I wonder if there's something similar for AL that could be used. GLIntercept shows all calls made to OpenGL and it even tracks undestroyed IDs. Besides buffers, there's also alSources which can also increase memory.

I profiled with Yourkit and saw that once 64 Mb heap was reached, there was no increase anymore.



So it seems to me it just fills the heap, but once its full, the old instances will get GC'd.

But i'm not very experienced with profiling.

Doesn't look like a memory leak to me.


Momoko, thanks a lot. May I add that logging at library level (if means are provided) can be always useful and is often forgotten!

Core, did you profile before or after applying the changes I suggested?

I have profiled both versions and here are my findings. Charts are attached below.

Trunk unpatched version

Each new stream increases heap space. Running the Garbage Collector doesn't seem to free that memory. After a few (<50) strokes on the spacebar, heap space hits the JVM limit and the process dies.

Process monitoring shows that process starts using 45MB (non-shared memory) and has been allocating memory during all that time.

Modified version

Streams don't seem to increase heap space (actually it seems to grow a bit). I pressed space hundreds of times, and heap space was still below 6MB, and the process never used more than that.

Process monitoring, however, does show memory growing. It ended up using 600 MB and the program hadn't crashed. I requested a Garbage Collection a couple of times but it didn't help.


As memory grows but Java Heap doesn't, I reckon this should be memory leaked on native code? We may be missing a call to something?

It is also possible that the version we use of OpenAL has a bug (which could explain why Core seems to obtain different results than I do if we use different versions).

Mulova, maybe you could debug on your system and set a breakpoint on every interesting AL10 call. Check if something is not being freed.  Also, does this happen when using the MemoryAudioPlayer instead of the StreamedAudioPlayer?

Charts for the original unpatched version (which causes an OutOfMemory error).

Charts for the version that uses the modifications I suggested above. Doesn't crash, but seems to leak memory (process memory).

jjmontes said:

Mulova, maybe you could debug on your system and set a breakpoint on every interesting AL10 call. Check if something is not being freed.  Also, does this happen when using the MemoryAudioPlayer instead of the StreamedAudioPlayer?

MemoryAudioPlayer also increases the native memory also. And it increases about 10MB when it is not stream.
I'll debug Which AL call consumes the memory and learn about OpenAL  :)

strange the process never died for me

I don't know what the problem is, but just to give you a bit of a hint. Native buffers will almost never be freed by garbage collection since they reside in native memory and the GC don't know anything about that. They will be collected if their handles (in heap) must be garbage collected due to running out of heap space.



So, check for creation of native buffers (doesn't matter if they lose their reference later, they will not be collected). And replace that with a buffer pool, perhaps.

A consistent test of this issue would be to set memory limits to like 10 MB, then in a loop create and destroy AudioTracks of a small sound file. If the issue is indeed in the creation of too many AL objects then you will never get an OutOfMemory error while Task Manager will show the memory constantly increasing (way beyond 10 MB) for that test.

I spent some more time with this I think that all created AL buffers are destroyed. But as I I may have missed to count something, or maybe buffers are also created somewhere else where I didn't look.



Tests so far do show a Leak (except for Core-Dump).



Who wrote this in the first place? Someone must have a deeper knowledge of AL and JME sound system…


'twas Mojo if I remember correctly. It hasn't been changed much since he first checked it in, so it's bound to have some bugs. Works ok for Mad Skills Motocross so far, but I'm not streaming anything.

webkid90 said:

Process monitoring, however, does show memory growing. It ended up using 600 MB and the program hadn't crashed. I requested a Garbage Collection a couple of times but it didn't help.

If you are using StreamedAudioPlayer, you should call AudioPlayer.cleanup to release memory.