As promised we will share our results:
We replaced the existing functionality with Java built-in serialization and two alternative libraries: FST and Kryo.
Doing this we were able to remove about 1250 lines of code (with more possible) which means less maintaining. All three support streams, making refactoring from buffers easy. FST and Kryo support optional registration, meaning code does not break when you forget to register, but the gain in efficiency is possible. Small note: we were unable to test Kryo with registration.
In terms of serialization size, jME performs best (props for that), but FST and Kryo follow closely. For a message containing a few Vector3f and Matrix3f, jME fit it in 137 bytes, FST in 149. Java built-in serialization is exceptionally bad with 552 bytes.
We tested performance using a throughput test due to the lack of ‘real world’ example (MonkeyZone doesn’t work in the newest jME). Again, jME performed best closely followed by FST and Kryo. We are aware of the fact the throughput in a throughput test is limited by many other factors and we believe the difference in performance is unnoticeable in real games.
We think considering replacing the hand-made serialization system with either FST and Kryo is a good idea. It introduces a external dependency, but saves a lot of potential maintenance in the removal of over 1000 lines of code. We also believe the libraries are less buggy and the optional registration saves users and you lots of headaches. Performance is very similar and the differences are most likely unnoticeable.
If you want, we can help setting up a pull request. We also have written some unit tests that would be relatively easy to add. Our code can be found in this branch: https://github.com/rbottema/jmonkeyengine/tree/benchmarking