Format conversion error

Hi



I tried to convert the packaged milkshape run.ms3d model to jme xml and save it to study the xml format and then to load it and convert back to binary.

It occurs to me that BinaryToXml writes out a Matrix3f according to the Data attribute in the binary array where XmlToBinary awaits a quaternion for attribute localrot in xml tag <joint … >



… please have a look





Stacktrace:



INFO: Child (FPS label) attached to this node (FPS node)

java.lang.RuntimeException: Unknown sax error: Unable to write start element: joint java.io.IOException: ilformated Quat:-0.9994293 0.021524467 0.026034879 0.02146365 0.99976623 -0.0026132448 -0.02608504 -0.00205295 -0.99965763

Seems to be still the same issue I had back here:

http://www.jmonkeyengine.com/jmeforum/index.php?topic=1450.0

Unfortunately, when Cep21 answered, I didn't have the xml file anymore, and I didn't have the time to reproduce it, either.

Cep, if you read this: the problem is really no way related to that specific file I used back then, it can easily be reproduced just the way winkman did: by loading the supplied run.ms3d that comes with jme, writing it out to xml, and trying to load it from the produced xml file. Back then, I  was exactly under the same impression: the xml output contains a matrix, as <joint>'s localrot protery, while a quaternion seems to be expected by the XMLtoBinary converter.

Here:

http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/skeletal.xml

is a simple example for such an xml document, produced by loading a ms3d model and writing it out to xml.

Well, looks like Cep's not going to show up anytime soon (http://www.jmonkeyengine.com/jmeforum/index.php?topic=2738.new#new), so I tried to have a look at this, because I would really love my blender exporter to be able to export skeletal animation into jME. The problem appears to be in XMLtoBinary, around line 187, where it says:

else if ("rotation".equals(att) || "rot".equals(att) || "localrot".equals(att)){
                writeQuat(value);}


But JmeBinaryReader clearly expects a matrix, which in fact can be found in the xml file. The logical approach seemed to be to do something like:

else if ("localrot".equals(att)){
                writeMatrix(value);}


But unfortunately, there is nothing like "writeMatrix" in XMLtoBinary.
So, I tried to fix this by making XMLtoBinary write out a Quaternion instead, and have JmeBinaryReader accept that instead of the rotation matrix. But apparently, JmeBinaryReader was not up to reading jointcontrollers (and default colors, and bounding spheres) at all, so I had to make quite some adjustments there, and fix a nasty bug in JmeBinaryReader, and voila, the xml file would load!
Something is still not quite right, however: When loading that guy from run.ms3d from xml, his right leg appears to have some bad joint indices. I will have a look at that as soon as possible, probably later today, and I will publish the modified files then, too.

That would be Great !

I looked into the joint format primarily to see if i could add a skeletal exporter into Your blender to jme script, but if You'd approach it i'll wait what You come up with

Bad news: I am not able to find out where those bad bone rotations or indices are coming from, because I can not produce any ms3d file with skeletal animation that will move anything in jME. The geometry shows up, apparently even the skeleton is loaded, but nothing moves… very frustrating. That run.ms3d guy is moving, however, but he has far too many joints and vertices to try and understand what's going on. So I will stop investigating, unless somebody else is able to produce a minimal ms3d file, preferably with only a quad face animated by two joints, that will load and animate in jME. If somebody can send me such a file, I might be able to find the problem, but as it is now, there is no way for me to find out what's wrong. I published the changed files anyway, I guess it is still a very small improvement being able to load jointcontrollers at all from xml without exceptions, even if soemthing is still not right.

The files are:

http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/JmeBinaryReader.java

http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/JmeBinaryWriter.java

http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/XMLtoBinary.java



On a second note, I had a quick look at the new blender 2.4 python API, and it seems like there is no way to get bone transformations currently, only the rest poses. Which makes it impossible to export skeletal animation from blender 2.4. It used to be possible with the 2.3 series, but the python doc for those versions seems to be no longer available from the blender site. If anybody has an offline version, they could make me very happy by sending it to me via email.

Winkman, it's great to hear that there is somebody out there willing to use and improve the blender exporter. If you are able to accomplish anything, please let me know!

do you mean animation specifically or jme xml in general, this may give you some answers to exports skeletal animation from blender http://www.doom3world.org/phpbb2/viewtopic.php?t=1711&start=60

Hevee,

Thank you winkman, that was exactly what I needed. It didn't help me identifying the problem with that run.ms3d guy, though, because with your file, everything went just fine. Loading from ms3d or xml both yielded the expected results. In the meantime, I have tested some more ms3d files (from psionic3d.co.uk), and all files I tested convert just fine to xml (well, there is some other problem with one of the files, some shared renderstate is referenced in the resulting xml file which is never defined, leading to an uncaught null pointer error, but that can easily be fixed by removing the bad references from the xml file). I think I will just leave that problem alone for now, assuming that it's specific to that very run.ms3d file, and (irrationally) hoping it will never occur in any other file. Which leads to the conclusion that the whole jointcontroller-from-xml-loading-problem is almost fixed.



On a second, and not quite unrelated note:

Thank you, mcbeth, that link was really useful, hopefully I will find enough time soon to have the blender exporter export skeletal animation.



Third:

Winkman, it's really good to hear that you care for that exporter at all and even thought about improving it, but that code was never intended to be a community project and is mostly quite obscure and uncommented, and probably clumsy, too (500+ lines is quite a lot for a python program, after all), so I don't think it would be very much fun for anybody to try and understand what's going on there. I will do my best to make skeletal animation export possible soon, and unless I seriously fail, I'd suggest you spare yourself the ugly details of my clumsy python coding.

Hey, for me total phyton noob that code seems not that ugly at all :slight_smile: but i would had a hard time to come up with anything useful.

So i'm really looking forward to that skeletal exporter.

oh no…not another phyton noob, being a python noob is bad enough. :-p



darkfrog

ah well, that was to underline my noobyness …

For me, this doesn't work. I'll have to look into this a little deeper, to make sure that I didn't produce the problems myself, but apparently, there are quite some attributes (and nodes) that XMLtoBinary still isn't able to process. One of those attributes would be "speed", which makes JmeBinaryReader unable to process that attribute, which appears in every jme exported jointcontroller, thus rendering JmeBinaryReader unable to load jointcontrollers. Those errors result in a ClassCastException: java.lang.String when trying to load a jointcontroller from xml.



Also, the fix you have added, Cep, seems to have broken ms3d loading for me. Is anybody else  having this problem, or did I just do something stupid breaking ms3d loading only for me?

This is what I get when trying to run TestMilkJmeWrite with the new cvs:


java.lang.ClassCastException: com.jme.math.Quaternion
        at com.jmex.model.XMLparser.JmeBinaryReader.readBegining(JmeBinaryReader.java:343)
        at com.jmex.model.XMLparser.JmeBinaryReader.loadBinaryFormat(JmeBinaryReader.java:159)
        at com.jmex.model.XMLparser.JmeBinaryReader.loadBinaryFormat(JmeBinaryReader.java:188)
        at jmetest.renderer.loader.TestMilkJmeWrite.simpleInitGame(Unknown Source)

I have tried to find any error on my side, but found nothing. Can anybody please run TestMilkJmeWrite with an up to date jME from cvs, and confirm or contradict that the fix in cvs from Cep has broken ms3d loading? Also, it would be great to hear if that fix is making problems with xml/jointcontroller loading for anybody else but me.

Cep, could you tell me how you tested your fix? Maybe that would point me to where my misconception lies.

TestMilkJmeWrite runs fine over here :expressionless:

I have ms3d loading working now, too, with latest cvs jME. Don't know what I have been missing… What's still not working, however, is xml jointmesh loading (and bounding spheres and areaclod meshes, see http://www.jmonkeyengine.com/jmeforum/index.php?topic=2667.0), all of which is fixed in my versions of the three xml/binary loading classes.

Next, I will try to replace my solution to the matrix/quaternion problem with Cep's, which was writing out a matrix for the localrot property instead of having XMLToBinary accept a quaternion (my approach). I think either way would work fine, but having not very much experience with quaternions, I guess it's better to have it Cep's way. Cep, is there a specific reason you chose to use matrices in the localrot property, but quaternions in the keyframes? I really would like to learn that reason, it might help me understanding quaternion rotations a bit better.

After trying to make matrices work as localrot property, I keep getting OutOfMemoryErrors… this is all getting quite frustrating. I had it all working allright using quaternions, so if Cep doesn't have any specific reason why he used matrices, I think it would be best to just use my solution with the quats, it's more consistent that way, too, with joints having quats in their keyframe rotations as well.



here: http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/TestXMLWriteRead.java is a test I've come up with, currently only testing jointmesh and areaclod classes. If my fixes end up in cvs (and thus prove to be any good), I will use this test to make more of the missing xml loading stuff available piece by piece.



The strange problem with that run.ms3d guy having faulty bone indices (or bone rotations?) for his right leg still persists, but for any other ms3d file I have, everything works just fine, so I don't really know where to start looking for that error.



One thing I took from Cep's fix and put it into XMLToBinary, however, is the previously mising writeMatrix method, which I renamed to writeMatrix3, because it seems to only be capable of writing 3x3 matrices.

Great! Now it works perfectly (except for the bounding spheres that get exported by default, but that's just two lines of code in XMLToBinary. Cep, I wrote you an email about trying to help with the missing attributes in XMLToBinary). Your solution is definitely better than mine, because it does not produce that strange "right-leg-problem" with the run.ms3d file. Thanks, Cep!

Tried the TestXMLWriteRead as is - gave me a class cast exception ???



java.lang.ClassCastException: java.lang.String

at com.jmex.model.XMLparser.JmeBinaryReader.processAreaClod(JmeBinaryReader.java:713)

at com.jmex.model.XMLparser.JmeBinaryReader.readBegining(JmeBinaryReader.java:234)

at com.jmex.model.XMLparser.JmeBinaryReader.loadBinaryFormat(JmeBinaryReader.java:159)

at com.jmex.model.XMLparser.JmeBinaryReader.loadBinaryFormat(JmeBinaryReader.java:188)

at tests.TestXMLWriteRead.loadFromXML(TestXMLWriteRead.java:99)

at tests.TestXMLWriteRead.simpleInitGame(TestXMLWriteRead.java:57)

at com.jme.app.BaseSimpleGame.initGame(BaseSimpleGame.java:414)

at com.jme.app.BaseGame.start(BaseGame.java:65)

at tests.TestXMLWriteRead.main(TestXMLWriteRead.java:72)





Do  i need other versions of the readers ??

Since Cep published his fix for the skeletal animation problem, which is a lot better than mine, I adjusted my version of XMLToBinary to work with Cep's fix instead of mine. All you need now to get areaclod meshes, skeletal animation, and bounding spheres (and probably most of the other stuff, but that's untested yet) to load from xml is a fresh cvs jME, then replace XMLToBinary by my new XMLToBinary class, which you can find here:

http://wwwhomes.uni-bielefeld.de/krabien/jmestuff/XMLtoBinary.java



Sorry for not pointing that out.

Hey - this works - can now get alot of models loaded into jme





Can this be added to the source