Offline voice recognition

We are busy creating an Android game where we want to use voice recognition for spell casting.



I wonder if you guys could recommend a library/engine for the voice recognition that could be used offline and during normal gameplay.



with kind regards,

Kirito

Voice recognition is tricky, but might work well for simple things…

The only Open Source option I know about is Sphinx, try googling that on the forums and on the web :slight_smile:

Sphinx-4 is written entirely in Java, try that :slight_smile:

http://hub.jmonkeyengine.org/groups/general/forum/topic/voice-recognition-proof-of-concept/ someone using sphinx here

Sadly enough no open source code it seems. Also don’t know if that is a run on a mobile device or on the pc. But I will see what I can do with sphinx in the hope it can use the build in mics.



Thank you guys for the suggestions

I found it really hard working with Sphinx4 (I wasn’t even able to build it, too complicatedn for me -.-)

I recommend voce (http://voce.sourceforge.net/) I just tried it out and it

!WORKS! :smiley:

(Voce uses Sphinx4)

@benkibitzer said:
I found it really hard working with Sphinx4 (I wasn't even able to build it, too complicatedn for me -.-)
I recommend voce (http://voce.sourceforge.net/) I just tried it out and it
!WORKS! :D
(Voce uses Sphinx4)

That actually looks like a nice setup, we should make a SDK plugin from this :)
@normen said:
That actually looks like a nice setup, we should make a SDK plugin from this :)

Added it to the repo now, you can install this via the update center now :) Tools->Plugins->"Voce Speech Recognition", then add the "Voce Speech Recognition" library to your project(s).

I also made a manual page (F1) but if somebody could check out the library and make a more concise example on how to get it to work when installing the plugin (what paths to set and where to put grammar files etc.) that would be awesome. I didn't check if one can read grammar files from the classpath but that would be preferable.
5 Likes

I checked this out and the voce developer apparently didn’t hear about URLs yet and uses Files for everything -.- I am in the process of adapting voce to work with URLs (and thus with files from the classpath) and will update our version.

2 Likes

Okay, changed the library to use URLs and updated the example in the manual, seems to work for me, you can update now :slight_smile:

2 Likes
@normen said:
Okay, changed the library to use URLs and updated the example in the manual, seems to work for me, you can update now :)


Terrifically badass. We should make a demo that turns on device cameras by saying "monkey see, monkey do" :)
1 Like

Thank you for the work Normen.



And even tho sphinx/voce are nice voice recognition tools and easy to use. It sadly enough doesn’t seem like it will accommodate what we are trying to accomplish sadly enough.



Even tho I have a feeling the idea our designers came up with might be a bit to much to handle offline on a mobile device either way.



The full idea is that the user can customize what they will be saying for the command and then able to repeat it at a later time to perform the action that belongs with the command.



Some feedback to the extend of this being accomplished would be nice as well. As I said I have slightly my doubts myself on this one. But if it is would you guys have a suggestion?

Well it certainly is a lot of computation work for a mobile device, thats why “Siri” and googles voice search do the heavy lifting on a server. This is your best bet for speech recognition in java, you can configure the grammar yourself as you see and the commands you issue then would also be configurable. Still, I don’t think it would work out anyway on a phone :slight_smile:

I’d switch it around a bit. Define simple “magic words” rather than trying to do english etc and then simplify down what you are looking for to something the mobile can cope with. Make sure there is a big difference between those magic words and then look for those differences rather than trying to do a full natural language parser.



The main thing is to never have similar sounding words. All the commands have to be very different. Then you can simplify down the parsing and just look for key markers to identify the command.

It seems Voce doesn’t play well together with JMonkey Android support. Adding the lib to a Android project gives me.



[java]UNEXPECTED TOP-LEVEL EXCEPTION:

java.lang.IllegalArgumentException: already added: Lcom/sun/speech/engine/BaseAudioManager;

at com.android.dx.dex.file.ClassDefsSection.add(ClassDefsSection.java:123)

at com.android.dx.dex.file.DexFile.add(DexFile.java:163)

at com.android.dx.command.dexer.Main.processClass(Main.java:486)

at com.android.dx.command.dexer.Main.processFileBytes(Main.java:455)

at com.android.dx.command.dexer.Main.access$400(Main.java:67)

at com.android.dx.command.dexer.Main$1.processFileBytes(Main.java:394)

at com.android.dx.cf.direct.ClassPathOpener.processArchive(ClassPathOpener.java:245)

at com.android.dx.cf.direct.ClassPathOpener.processOne(ClassPathOpener.java:131)

at com.android.dx.cf.direct.ClassPathOpener.process(ClassPathOpener.java:109)

at com.android.dx.command.dexer.Main.processOne(Main.java:418)

at com.android.dx.command.dexer.Main.processAllFiles(Main.java:329)

at com.android.dx.command.dexer.Main.run(Main.java:206)

at com.android.dx.command.dexer.Main.main(Main.java:174)

at com.android.dx.command.Main.main(Main.java:91)

1 error; aborting

C:UsersUsernameDocumentsJMEBasicGamenbprojectmobile-impl.xml:10: The following error occurred while executing this line:

C:Program Files (x86)Androidandroid-sdktoolsantbuild.xml:850: The following error occurred while executing this line:

C:Program Files (x86)Androidandroid-sdktoolsantbuild.xml:852: The following error occurred while executing this line:

C:Program Files (x86)Androidandroid-sdktoolsantbuild.xml:864: The following error occurred while executing this line:

C:Program Files (x86)Androidandroid-sdktoolsantbuild.xml:266: null returned: 1

BUILD FAILED (total time: 10 seconds)[/java]

mhh it looks like you added the lib to both the JME project and the Android project.

It should be only in your JME project and the build will do the rest.

Hmm odd since I only added it in the overall project properties once. But I will look into it. Thanks for the suggestion



Edit:

Even tho I only add it to the JME project when I compile it I get that error. I noticed when checking the mobile folder the libraries are added there as well (just for clearance where I added it. Rightclick project BasicGame->properties->libraries->add Jar/folder->voce speech recognition)

@normen is the fastest man on earth wtf

Just making sure you know that Android has a speech recognition API built right in.



With Jelly Bean it can be put into offline mode now.



Might work for your needs.

2 Likes

Even tho I know JB has an offline api from google themselves sadly enough that would limit the audience rather severely And the normal google api uses the online mode which we are mainly trying to prevent.



Sadly enough I am still running into the same issue as posted earlier and still not sure where it is coming from sadly enough.