For anyone who will look how to do it. Here simple free solution!
Found a nice app, that is generally under MIT.
It works on both Windows and Linux and seems like work very well.
You just do(Windows): .\rhubarb.exe -o output.txt .\tmp\guess-what-2.wav
or(Linux) ./rhubarb -o output.txt ./tmp/guess-what-2.wav
and you get letters where each letter mean some mouth shape(described on github page)
example:
| 0.00 | X |
|---|---|
| 0.05 | B |
| 0.19 | D |
| 0.40 | B |
| 0.54 | F |
| 1.03 | B |
| 1.31 | X |
| 1.46 | X |
so later just do some code that during sound play, will also change mouth shape keys or change bones of mouth to match letter shape described in tool documentation.
because X is not X letter, its some shape, it mean closed mouth.
here github: