AudioNodes supporting sequential playback of multiple assets

Contribution Depot
1

My game relies heavily on speaking AI characters, and their sentence grammar is built by sequential playback of a series of assets with minimal latency between them.



In looking at AudioNode, there was a curious lack of any means of knowing when an asset’s playback was complete, and also an mildly inconvenient lack of a means to play a second asset from an AudioNode even if you guessed when the first asset had played.



I changed AudioNode minimally to add a Completed state and expose the means of assigning it an asset for play that had previously been buried within the constructors, and altered LwjglAudioRenderer to set this state in the appropriate places. This permits my own class to monitor its AudioNode and, each time it sees it reach a Completed state, stuff the next asset in for playback. It’s been working nicely so far.



So, in hopes that others might find this helpful, I offer these diffs. There seems to be no way to do this without altering JME, and I hope some variant of this capability is accepted to keep me from being forked.



I can gladly share my external class that leverages this capability. It also adds a nice free pool handling system to keep AudioNodes from being being part of the garbage collection cycle.



[java]



Index: src/lwjgl/com/jme3/audio/lwjgl/LwjglAudioRenderer.java

===================================================================

— src/lwjgl/com/jme3/audio/lwjgl/LwjglAudioRenderer.java (revision 9125)

+++ src/lwjgl/com/jme3/audio/lwjgl/LwjglAudioRenderer.java (working copy)

@@ -765,6 +765,8 @@

// And free the audio since it cannot be

// played again anyway.

deleteAudioData(stream);

+

  •                    src.setStatus(Status.Completed);<br />

}

}

}else if (!streaming){

@@ -779,7 +781,9 @@

src.setChannel(-1);

}

clearChannel(i);

  •                freeChannel(i);<br />

  •                freeChannel(i);<br />

+

  •                src.setStatus(Status.Completed);<br />

}

}

}

@@ -869,7 +873,7 @@



if (src.getStatus() == Status.Playing){

return;

  •        }else if (src.getStatus() == Status.Stopped){<br />

  •        }else if (src.getStatus() == Status.Stopped || src.getStatus() == Status.Completed){<br />

// allocate channel to this source
int index = newChannel();
@@ -928,7 +932,7 @@
if (audioDisabled)
return;

- if (src.getStatus() != Status.Stopped){
+ if (src.getStatus() != Status.Stopped && src.getStatus() != Status.Completed){
int chan = src.getChannel();
assert chan != -1; // if it's not stopped, must have id

Index: src/core/com/jme3/audio/AudioNode.java
===================================================================
--- src/core/com/jme3/audio/AudioNode.java (revision 9125)
+++ src/core/com/jme3/audio/AudioNode.java (working copy)
@@ -100,10 +100,14 @@

/**
* The audio node is currently stopped.
- * This will be set if {@link AudioNode#stop() } is called
- * or the audio has reached the end of the file.
+ * This will be set if {@link AudioNode#stop() } is called
*/
Stopped,
+
+ /**
+ * The audio node has finished playing its audio.
+ */
+ Completed,
}

/**
@@ -161,8 +165,7 @@
* @deprecated AudioRenderer parameter is ignored.
*/
public AudioNode(AudioRenderer audioRenderer, AssetManager assetManager, String name, boolean stream, boolean streamCache) {
- this.audioKey = new AudioKey(name, stream, streamCache);
- this.data = (AudioData) assetManager.loadAsset(audioKey);
+ loadAsset(assetManager, name, stream, streamCache);
}

/**
@@ -178,8 +181,7 @@
* seeking, looping and determining duration.
*/
public AudioNode(AssetManager assetManager, String name, boolean stream, boolean streamCache) {
- this.audioKey = new AudioKey(name, stream, streamCache);
- this.data = (AudioData) assetManager.loadAsset(audioKey);
+ loadAsset(assetManager, name, stream, streamCache);
}

/**
@@ -238,8 +240,37 @@
throw new IllegalStateException( "No audio renderer available, make sure call is being performed on render thread." );
return result;
}
-
+
/**
+ * Loads an audio asset with the given audio file.
+ *
+ * @param assetManager The asset manager to use to load the audio file
+ * @param name The filename of the audio file
+ * @param stream If true, the audio will be streamed gradually from disk,
+ * otherwise, it will be buffered.
+ * @param streamCache If stream is also true, then this specifies if
+ * the stream cache is used. When enabled, the audio stream will
+ * be read entirely but not decoded, allowing features such as
+ * seeking, looping and determining duration.
+ */
+ public void loadAsset(AssetManager assetManager, String name, boolean stream, boolean streamCache) {
+ audioKey = new AudioKey(name, stream, streamCache);
+ data = assetManager.loadAsset(audioKey);
+ }
+
+
+ /**
+ * Loads an audio asset with the given audio file, buffering the file fully for playback
+ *
+ * @param assetManager The asset manager to use to load the audio file
+ * @param name The filename of the audio file
+ */
+ public void loadAsset(AssetManager assetManager, String name) {
+ loadAsset(assetManager, name, false, false);
+ }
+
+
+ /**
* Start playing the audio.
*/
public void play(){
@@ -274,8 +305,8 @@
* Do not use.
*/
public final void setChannel(int channel) {
- if (status != Status.Stopped) {
- throw new IllegalStateException("Can only set source id when stopped");
+ if (status != Status.Stopped && status != Status.Completed) {
+ throw new IllegalStateException("Can only set source id when stopped or completed");
}

this.channel = channel;
[/java]
2

What’s the difference between Stopped and Completed? An audio node becomes stopped when the audio finishes playing, I don’t see the point of the Completed state. Also I don’t see how you solve the problem you mentioned of knowing when the audio finishes playing, as you still have to check the status value every frame with or without your patch

3

In fact, you would have to wait for update anyway because (as I discovered once) doing things on setStatus() is very dangerous since it is called directly from the audio thread and you can really mess up the audio renderer’s internal data structures.



My fear about the loadAsset() stuff is that it doesn’t check to see if one is already playing or not. When that other sound finishes playing then you will get a stop for an old audio data. Not sure it causes problems or not.

4

My understanding of AudioNode’s states may have been flawed.



I did not want to have to learn how many cases might cause “stopped” to become the state (calling stop() is an obvious one, but I wanted to not discover others). Indeed, I imagined that an AudioNode was stopped initially and only started some while later. It is true that loadAsset() needs to be called only when the state were Stopped or Completed.



Perhaps, if Completed were not needed, the loadAsset() methods would have value and could be fortified by an exception if called while state != Stopped.



tone