VRM humanoid format

General Discussion
1

I’ve surveyed about JME3 and CG for several days. And I think it is a good idea that JME3 support VRM + BVH. If JME3 supports VRM + BVH, various high quality humanoid models will be played with superb motion on JME3.
The following is a brief introduction to CG areas in Japan.
I am just a user of these software. Honestly, I don’t think I’m in a position to introduce it. Please think that it is just a chat. In addition, this sentence is through Google Machine Translation. Please check the details yourself.

VRM is an open humanoid model format based on GLTF.

Although VRM is described for VR, it can also be used on flat displays. Useful in all situations where a humanoid model is used.

Anyone can create a humanoid model freely with application called Vroid. With just one click. It’s a joke, but you don’t need specialized skills. Vroid is like terrain in a humanoid model. Vroid outputs VRM. It is available free of charge. This Twitter account is Vroid official.
https://twitter.com/vroid_pixiv/status/1030152633930080256

The humanoid model created by Vroid can be used in the game. According to the terms of Vroid, not only free games but also commercial use is possible. However, it is prohibited to redistribute Vroid Studio itself. It is possible to redistribute the output humanoid model. This page is in Japanese, so please machine translate.
https://vroid.pixiv.help/hc/en/articles/360008211814-VRoid-Studio% E3% 81% AE% 5% 95% 86% E 7% 94% A 8% E 5% 88% A 9% E 7% 94% A8% E3% 81% AF% E5% 8F% AF% E8% 83% BD% E3% 81% A 7% E3% 81% 99% E3% 81% 8B-

The impact of Vroid is tremendous. In order to introduce it into the JME3 ecosystem, it is necessary to support VRM.

About MMD MikuMikuDance. There are clusters of MMD, and a large number of motions, models and videos are produced. The humanoid model output by Vroid can be easily converted to PMX format. Many MMD videos use models created manually by Blender etc.
https://www.youtube.com/results?search_query=mmd
The format of MMD is vmd, pmd and pmx. vmd is motion data. pmd and pmx are model data. pmx is the successor of pmd.
I am not the author of this video, but I think that this video is a VRM model created by Vroid, converted to pmx, and applied vmd motion, and played in MMD.

vmd files are usually created by MikuMikuDance (MMD) or MikuMikuMoving. If you use the model or motion of MMD cluster in your own game, the author will get angry with you. Please stop. They basically publish the assets they created for video production. MMD clusters are horrible, so don’t use their work unless you understand the terms and conditions in detail.

There is one more important thing. liveanimation can convert vmd to BVH and edit BVH. In other words, there are two ways of creating motion data: MMD and live animation. The liveanimation’s vmd support is good and dances beautifully. Loading VMD, then exporting with BVH, importing that BVH, playing it completely succeeded. I confirmed that the combination of BVH motion created from vmd through live animation and the VRM model of Vroid makes it dance beautifully. This means that you can use MMD only for motion creation and you can not use MMD model format PMX.

a video production in game-like system. not jme3. I am not the author of this video.

The motion of this video was created by MMD. It is game-like motion. Being able to work with MMD is attractive for motion creation.

It seems that pmd and pmx can be converted to vrm through unity. But that is not necessary. It is possible to use MMD only for motion creation.

easy
Vroid> VRM> PMX
VMD <> BVH

little hard but not necessary
PMD, PMX> VRM

Motion data does not have a simple solution like Vroid or terrain, and needs to be produced by tools.

MMD has english UI.

For those who want to do Vroid + MMD, it may be a bit difficult, but I will show you the steps.
Open here with Google Machine Translation.

DL any vmd motion files.
Some DL pages require a password. The location where the password is written varies depending on the case, but it is written somewhere on the relevant web page. Please find it by machine translation. Some files do not require a password.

Create one of your humanoid models in Vroid.

Start liveanimation, select one of the character buttons arranged at the top of the screen, and place a model on the scene. Then import vmd and press the play button, the character will dance. Please export BVH file. Wrong export settings will break the motion. Please import the exported file again and confirm that it is not broken. I think that you can find the correct setting by trial and error.

You can DL the player on this page. The player plays with VRM model + BVH motion. You have VRM models by Vroid, and BVH motion files by VPVP wiki + live animation so far. The player is using Japanese. Machine translation does not arrive there. There are three tasks. Load model, load motion, press play button.

If JME supports VRM, the humanoid model of Vroid can be used on the game. And you can create that motion in MMD.

The spread of VRM, BVH and GLTF solves the 3DCG transportation problem.

Because Vroid supports VRM, it is worthwhile to support VRM.

That’s what I thought.
thank you for reading.

1 Like
2

@nehon wrote a BVH importer for jMonkeyEngine:
GitHub - Nehon/bvhretarget: test project for bvh loading and retargeting to arbitrary skeleton

I’m using it (with slight modifications) in Maud. If there’s enough interest, I’d be willing to integrate it into jMonkeyEngine.

1 Like
3

@lifeinwild I got to know about VRoid on gamefromscratch afew months ago, It’s cool for creating anime style fantasy characters.

VRoid Studio Revisited – GameFromScratch.com

Note that JME already supports gltf 2, I noticed in this tutorial the guy just exported the model from VRoid then renamed file format from .vrm to .glb (which is a gltf binary format) then successfully opened it in a gltf viewer.

2 Likes
4

Thank you for your reply.

@sgold
I already noticed that JME3 supports BVH. I forgot to write. However, I thought that there might be a problem when combining VRM and BVH, so I wrote VRM + BVH. I do not believe that there is the flexibility to combine data as I wish in the 3D domain. Because I had a lot of trouble.

@Ali_RS
I thought about the method of extension. But I am not sure if everything will work.
There is such a description on the official site. “Please change file extension from ~. Vrm to ~. Glb. However, custom settings for VRM model will not be reflected.”

The existence of this project may mean that the method is limited. Several commentary articles specify that this importer be used.

several people have said that the method fails by software.

I will try JME3 + Vroid + BVH.

5

In the 3D area, I think it is necessary to check the operation of each tool chain. I think a good new toolchain might be possible. I thought it would be easier to create a good quality game than before.
Vroid> VRM2PMXConverter> MMD> LiveAnimation> JME3
I confirmed that this tool chain works well except for the part of JME3. The JME 3 forum had no VRM topic and it was unclear if it would work well.

I experimented with it.
The Vroid model was displayed in JME. However, it takes a long time to read. Loading multiple Vroid models is likely to be a serious problem.

        long start = System.currentTimeMillis ();
        Spatial vroid = assetManager.loadModel ("test.glb"); // 17MB
        long end = System.currentTimeMillis ();
        System.out.println ((end-start) + "ms"); // 8877 ms

Asset manager seems to be thread safe. Application code can be multithreaded for asset loading.

How fast is j3o?
I created a j3o file with this code, but I was surprised at 113MB.

        BinaryExporter be = new BinaryExporter ();
        try {
            File f = new File ("./ testj3o.j3o");
            be.save (vroid, f);
        } catch (IOException e) {
            e.printStackTrace ();
        }

The loading time is 1 second.

        long start = System.currentTimeMillis();
        Spatial vroid = assetManager.loadModel ("testj3o.j3o"); // 113MB
        long end = System.currentTimeMillis ();
        System.out.println ((end-start) + "ms");//1153ms

Load time is about 8 times faster when handling Vroid model with j3o. But the file size has increased about 7 times. In recent years, in general, storage capacity may not have increased much. By change of HDD, SATA SSD, NVMe SSD, optane memory. It is difficult to choose either CPU time or storage capacity. Or is there a better way? I modified my game design a bit for this.

A large number of repeated alert logs. Perhaps this means that the result is different from the original display.

com.jme3.scene.plugins.gltf.CustomContentManager readExtension
WARN: Could not find loader for extension KHR_materials_unlit

JME3 does not bundle BVHLoader. I downloaded this project and made it possible to refer to class files.

It took 3677 ms to load 200 seconds of BVH motion with 10 MB. This will also speed up with j3o.

I failed to make the Vroid model dance. On the other hand, I succeeded in making this skeleton dance by BVH. This skeleton is neither VRM nor GLB.

jme_bvh_vroid

There is a software called VRM Live Viewer. This software can read Glb, but the motion fails when it reads a vrm file with the extension glb. This software bundles the default motion and loop plays the default motion until some motion is specified. However, the default motion did not play on Glb model.

In other words, if you change the VRM file extension to glb you may not be able to work with BVH motion. I guess this might be specification, not implementation problem.

The author said that VRM and BVH are not completely compatible. Hmmm. The idea of ​​creating models and motions in separate files or using different tools may have limitations.
https://connect.unity.com/p/vrm-live-viewer

Still, VRM + BVH is capable of considerable motion. attractive. Changing VRM extension to GLB makes it impossible to work with BVH.

thank you for reading.

1 Like
6

After some hard work, I got a little more familiar with VRM , VRoid , Gltf, BVH and bvhretarget project.

I considered various options for my app. I still want to use the BVH format and the retarget function.

The Maud project is updating the BVH loader. However, the BVH code is divided into Maud and Wes. it is little hard to catch up.

Please integrate BVH loader and retarget function into jME. It is good for jME to support the new open format.

By the way, I noticed some things. I just write it as information.
The bottleneck of GltfLoader is AWTLoader’s javax.imageio.ImageIO.read (). JSON parsing is not a bottleneck.

The humanoid model of Vroid is around 17MB in vrm format. that 78% is png image files. It hits the bottleneck of ImageIO.read (). Vroid can export models with low texture quality settings, and can halve import time. In many cases, the problem with ImageIO.read () seems to be solved by multithreading. java - ImageIO.read(...) - It is very slow, is there a better way? - Stack Overflow

1 Like
7

You should make an issue about it on the SDK project. Otherwise it probably will never be considered as it is lost here. You can however change the theme from the SDK. I’m happy with the NB default “white” theme.

1 Like
8

It was small for me, so I hesitated to set up a topic. But github issues are appropriate.

1 Like