JmeConvert tool

User Code & Projects
1

As part of another thread ([SOLVED] BinaryExporter and materials - #66 by mitm) I got the itch to see if I could make a command line “converter”. Or at least poke around at would make it hard or not.

Friday night I played around with some ideas and had some nice success so I’ve pushed up a prototype with basic functionality. You can find the latest downloadable release here:

…where I’ll keep updating that release until I start making “official versions”.

The tool is very basic but functional and so far works with every GLTF that I’ve tried from Sketchfab. I’ve been less successful with other formats that JME is supposed to support… but I don’t know if the SDK would read them either because I don’t have the SDK installed.

To use:

  1. download and unzip the zip somewhere
  2. run bin/jmec.bat or bin/jmec (on Linux) from that zip

…it will print command line help like:

Current command line help:
        _   __  __   ___    ___
     _ | | |  \/  | | __|  / __|
    | || | | |\/| | | _|  | (__
     \__/  |_|  |_| |___|  \___|

    JME3 Asset Converter v1.0.0

Usage: jmec [options] [models]

Where [models] are a list of model files.

Where [options] are:
 -sourceRoot <dir> : specifies the asset root for the models.
       Model paths will be evaluated relative to this root.

 -targetRoot <dir> : specifies the asset target root for writing
       converted assets and their dependencies.

 -targetPath <path> : a path string specifying where to place
       the copied/converted assets within the targetRoot.  Any
       internal asset keys will be 'rehomed' to this path.

Examples:

>jmec -sourceRoot C:\Downloads\CoolModel -targetRoot assets -targetPath Models/CoolModel C:\Downloads\CoolModel\AwesomeThing.gltf

 Converts C:\Downloads\CoolModel\AwesomeThing.gltf to a j3o and writes it
 to ./assets/Models/CoolModel/AwesomeThing.gltf.j3o

 Any dependent textures, etc. relative to C:\Downloads\CoolModel will
 be copied until the appropriate ./assets/Models/CoolModel/* subdirectory.

 For example:
    C:\Downloads\CoolModel\textures\awesome-sauce.jpg
 Gets copied to:
    ./assets/Models/CoolModel/textures/awesome-sauce.jpg

But basically the idea is that if you download some file or have saved a file from blender with its dependent textures and so on, you’d use -sourceRoot to tell it the relative root for that model (ie: where the textures, etc. should be resolved relative to). You give a -targetRoot that’s probably your project’s assets directory. Then specify a -targetPath within your assets directory for where you want to write the files. And then give it a file to convert.

It will write the j3o and all of the textures, etc. to that location.

As said, this is a super basic tool at the moment. It only allows straight convert + copy.

Next steps:

  1. add scripting support so that you can modify the models once loaded… say to set a proper scale or mark materials for j3m export. (Probably by giving it an asset key.)
  2. support selectively exporting j3m materials.

…that will be where I’ll cut a “version 1” because it will be a pretty powerful tool by then.

Edit: made a real release 1.0.0 with the above ‘next steps’ features. Read more below. Link here if you’re impatient: Release JMEC Version 1.0.0 · Simsilica/JmeConvert · GitHub

10 Likes
2

Yes, jme could use a tool which enables to import and easily reimport assets. (Eg. let’s say 10 models are to be created from single blend file, easy one click reimport if blend file is modified)

I did something similar here:
GitHub - TehLeo/JmeAssetImporter: A tool to automatically import assets from blender to jME

Thou I don’t have the time atm to make it into a super polished version.

3

Great tool, thanks so much for sharing. :slightly_smiling_face:

Edit:
Maybe we should pin this topic to be on top of forum page for a while ?

4

I think I’ve set it to “pinned” for two weeks now.

2 Likes
5

So I was able to finish adding scripting support and material generation. I also added some -probe options which changes how much output is generated when probing the asset.

The release is here:

Release notes:

  • Initial release, includes:
    • asset copying and rehoming
    • model script processors for groovy, javascript, etc.
    • model probing options
    • optional material generation

Here is some super basic documentation. If anyone wants to volunteer to flesh it out or has feature requests then let me know.

The command line help provides the basics:

        _   __  __   ___    ___
     _ | | |  \/  | | __|  / __|
    | || | | |\/| | | _|  | (__
     \__/  |_|  |_| |___|  \___|

    JME3 Asset Converter v1.0.0 build:2019-05-17T03:46:59-0400

03:47:00,290 INFO  [Convert] Max memory:3641.00 mb
Usage: jmec [options] [models]

Where [models] are a list of JME-compatible model files.

Where [options] are:
 -sourceRoot <dir> : specifies the asset root for the models.
       Model dependency paths will be evaluated relative to this root.

 -targetRoot <dir> : specifies the asset target root for writing
       converted assets and their dependencies.

 -targetPath <path> : a path string specifying where to place
       the copied/converted assets within the targetRoot.  Any
       internal asset keys will be 'rehomed' to this path.

 -script <path> : a script file that will be run against the model
       before writing out.  Any number of script files can be specified
       and they will be run in the order specified.
       Groovy and Javascript are supported 'out of the box' but any
       JSR 223 compatible scripting engine should work if on the classpath.

 -probe [probe options string] : configures the information that the probe
       will output.
       [probe options]:
       A : all options turned on, same as: btrscpd
       b : show bounding volumes
       t : show translations
       r : show rotation
       s : show scale
       c : show the list of controls
       p : show material parameters
       d : list asset dependencies

Examples:

>jmec -sourceRoot C:\Downloads\CoolModel -targetRoot assets -targetPath Models/CoolModel C:\Downloads\CoolModel\AwesomeThing.gltf

 Converts C:\Downloads\CoolModel\AwesomeThing.gltf to a j3o and writes it
 to ./assets/Models/CoolModel/AwesomeThing.gltf.j3o

 Any dependent textures, etc. relative to C:\Downloads\CoolModel will
 be copied until the appropriate ./assets/Models/CoolModel/* subdirectory.

 For example:
    C:\Downloads\CoolModel\textures\awesome-sauce.jpg
 Gets copied to:
    ./assets/Models/CoolModel/textures/awesome-sauce.jpg

The -probe option is useful for finding things out about the model that you might then use in a script.

The most interesting new option is probably the -script support. I’ve only used groovy scripts but in theory Javascript works also because it’s built into Java.

The bindings available to the script are:

  • ‘convert’ : the actual instance of the JmeConvert application.
  • ‘model’ : the ModelInfo wrapper around the asset being converted.

ModelInfo provides some findFirst()/findAll() methods for searching through the scene hierarchy. It can also provide the model name, etc…

The most common things that I find myself doing are resetting the scale of the model when I copy it. When downloading things from Sketchfab, a lot of the models are in 1 unit = 1 centimeter or 1 inch, etc… The “-probe b” option is useful here because it will tell you the world bounding volume and so you can calculate a good scale.

A simple script to rescale a Sketchfab model:

model.findFirst("RootNode (model correction matrix)").setLocalScale(0.05, 0.05, 0.05);

Sketchfab gltf models have a common structure so this should work with any of them.

The other thing I find myself doing is generating a j3m material… either for a specific geometry or all of them.

Here is an example for a specific geometry:

import com.jme3.asset.MaterialKey;
import com.jme3.scene.Geometry;
def geom = model.findFirst("SK_HPC_01_01_material_0_0", Geometry.class);
model.generateMaterial(geom.material, "materials/SK_HPC_01_01_material_0_0.j3m");

Here is an example that will generate a j3m for all of the materials. Note: if any of them have the same name then they will clobber each other… but it’s just an example.

import com.jme3.asset.MaterialKey;
import com.jme3.scene.Geometry;
model.findAll(Geometry.class).each { geom ->
    model.generateMaterial(geom.material, "materials/" + geom.material.name);
}

…change the output path to your tastes. It’s relative to the target path. Presumably, even “…/Materials” would work and put it in the common assets/Materials path but I haven’t tried it.

For those who care, my test model for the above was this one:

And here is it loaded into my app, rehomed and everything… all materials generated and the face material’s base color set to red:

I picked this one as my common example because it’s animated… and yes, the animation works fine in JME.

6 Likes
6

Thanks, looks cool.
As a feature, you may want to add an option to generate an icon from the converted model.

btw, did not know about the scripting API support in Java, thanks to your code I learned something new about Java today.:slightly_smiling_face:

2 Likes
7

It would be neat… but tricky, I think. Currently the code doesn’t require any lwjgl or anything to run. I think the apparatus just to generate an icon might be quite heavy-weight.

2 Likes
8

i did it before, but overall idk where icons would be required, because even in GUI 3d model can be used without any pregenerated images.

and the face material’s base color set to red

i always had problems when i export via .gltf to j3o(via SDK).

The problem was when i wanted set material via Code, not via Scene Explorer.

  • i never knew what Geom_0 or Geom_1 or Geom_2 will be geom i need set material, because each time i modified something in blender, this numbers were random. idk if its .gltf that make them random or blender(where i cant change this numbers by myself) or jme. Anyway it made me angry so i prefer use one material in blender for each file.

Do you maybe know some other way for apply material via code to exactly element you need?

what i want to say, for you it is SK_HPC_01_01_material_0_0 for this model, but try modify some materials and you will see this element might change to SK_HPC_01_01_material_0_1.

This is problem i seen with multiple materials used in blender.

9

Do you name your materials?

To be honest, I only have limited experience here… but hopefully someone else can help.

10

yes, i name 3 things always in blender:

  • Materials
  • Object name
  • Vertex Groups
    this last one is passed to geom names. Lets say “Car”.

if there is only one material, Geom will be named just “Car”

if there are more materials used, then there will be Car_0 and Car_1 etc. but i never know what is Car_0 because its totally random for me.(each time i modify something, it might change)

i wanted to let you know, because if you made this tool, you might want to look if its maybe JME related. because i really dont know what stage is it random.(it might be gltf or blender itself) anyway it would be good get rid of random names.

11

Another suggestion is an option to load UserData for gltf models. It can be done using gltf ExtraLoaders in JME.

This is the code snippet I am using. I borrowed it from someone else project.

/*
 * The MIT License
 *
 * Copyright 2019 .
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 *
 * MODELS/DUNE.J3O:
 * Converted from http://quadropolis.us/node/2584 [Public Domain according to the Tags of this Map]
 */

package otm.moona.core.common.util;

import com.google.gson.JsonElement;
import com.google.gson.JsonObject;
import com.jme3.scene.Spatial;
import com.jme3.scene.plugins.gltf.ExtrasLoader;
import com.jme3.scene.plugins.gltf.GltfLoader;
import java.util.Map.Entry;

/**
 * Gltf loader for Blender user data.
 * 
 * @author Robert
 */
public class GltfUserDataLoader implements ExtrasLoader {

    @Override
    public Object handleExtras(GltfLoader loader, String parentName, JsonElement parent, JsonElement extras, Object input) {
        // if its a spatial, we want to add all the spatial userdata 
        if (input instanceof Spatial) {
            if (extras.isJsonObject()) {
                JsonObject ext = extras.getAsJsonObject();
                for(Entry<String, JsonElement> element : ext.entrySet()) {
                    Spatial spatial = (Spatial) input;
                    spatial.setUserData(element.getKey(), element.getValue().getAsString());
                }
            }
        }
        return input;
    }
}
2 Likes
12

I’m considering adding this support to jmec tonight… are there some good gltf models I can use to test? Or are pretty much any gltf models likely to have this information?

13

In blender you add it through the object panel.

[Album] imgur.com

The thing to know though is that the custom properties can be float, boolean or Integer.

https://docs.blender.org/manual/en/dev/data_system/custom_properties.html

I had to rewrite the above code to this to get it to work.

public class GltfUserDataLoader implements ExtrasLoader {

    @Override
    public Object handleExtras(GltfLoader loader, String parentName, JsonElement parent, JsonElement extras, Object input) {
        // if its a geometry, we want to add all the geometry userdata 
        if (input instanceof Spatial) {
            if (extras.isJsonObject()) {
                JsonObject ext = extras.getAsJsonObject();
                for(Entry<String, JsonElement> element : ext.entrySet()) {
                    Spatial spatial = (Spatial) input;
                    if (element.getKey().equals("open")) {
                        spatial.setUserData(element.getKey(), element.getValue().getAsBoolean());
                    }
                }
            }
        }
        return input;
    }
}

I could not get it to grab a string because the ExtrasLoader would throw an exception when it hit one of the other uneditable blender default settings first.

Should be checking types or something but since I was time constrained I went with what I knew would work for that particular situation.

How other programs do userdata I do not know.

Edit: To clarify, most models may not have a default setting under custom data but there are some addons that do set these and these addons cause the problems with the gltf ExtrasLoader. if these werent present, the getAsString would probably work fine. As always, I could of been doing something wrong since was just learning about the gltf loader and was hacking it from someone elses project.

14

If someone could provide me a sample gltf model (or two) with various types of user-added data that would be super helpful. Not sure I will have time to flex my blender-fu tonight but I could definitely mess with code.

15

https://github.com/mitm001/gltf/tree/master/assets/Textures

Simple door.
Blend is a larger test area.
Uses gltf 2.79b exporter so if you hit problems I would suspect that.
Has animations.

Anyone can feel free to dump whatever gltf they want and I will accept the pr, just give a heads up here since I rarely check email.

Edit: Also contain some default custom data objects that are helpful to avoid so your loader will be more robust.
Edit: Uses gradle 5.4.1 but you can just grab the files from the zip if want.
Edit: went ahead and setup the environment in case you wanted a quick test project.

1 Like
16

I also made a simple blue cube test you can download here.

You should be able to add custom properties into any gltf file using a text editor without needing Blender.
gltf file is a JSON file, if you open the file testUserData.gltf from the test I provided, in a text editor, you will notice :

"nodes" : [
        {
            "mesh" : 0,
            "extras" : {
                "testString" : "I am string",
                "testFloat" : 1,
                "testBoolean" : "true"
            },
            "name" : "Cube"
        }
    ],

you can add your custome properties inside "extras".

17

That’s useful to know. I was mostly interested in handling the cases people would actually use in Blender, though.

…but knowing that I can just look in the file is useful, too.

18

New release jmec 1.1.0:

Includes support for GLTF extras as user-add data. Handles lists, maps, all primitives, etc…

Note: the examples provided in this thread would miss applying attributes to Geometry arrays which I guess turn out to be common from blender exports. For example, the “door” flag in the provided examples was actually not applied to anything.

Now in these cases, the “door” attribute is applied to each Geometry in the list. (2 in this case.) I felt this was the better option versus rolling it up to a parent… because rolling it up would lose information and potentially overwrite another attribute at that level. Plus, it seemed more likely to want these attributes with the “material holding object” and it would be trivial to roll them up in a model script anyway.

If you guys test this, let me know if you find any issues.

One quick way to see how it has been applied is to run jmec on the target that you just converted but just use the “-probe u” option to dump the user-added data.

Edit: by “handling lists and maps”, I mean if your json data has them:

"nodes" : [
        {
            "mesh" : 0,
            "extras" : {
                "testString" : "I am string",
                "testFloat" : 1,
                "testBoolean" : "true",
                "testNumberList":[
                     1, 2, 3, 4, 5
                ],
                "testMap":{
                    "foo":"bar",
                    "bing":123
                },
                "testMapList":[
                    {"test": 123, "foo":"bar"},
                    {"anotherTest": "This is a test"}
                ]
            },
            "name" : "Cube"
        }
    ],

…and so on… any level of nesting. (I did this to properly handle the empty ant_landscape in mitm’s example… but it’s just a good thing to have.)

2 Likes
19

Works for me. Better than should probably with respect to userData.

I had forgotten about adding userdata to the geometry.

This was because of experimentation when I was trying to use userData to set things while loading the object. I found that setting userData through object panel was the way everyone else did it and forgot to remove that.

When I read the output I was surprised to see that there on the geometry itself. Not sure how that could be used but nice to see it can be done.

I will be digging into your code to see how you implemented that and the loader cause it works real nice.

20

Just wanted to say I’ve derived this project in my intellij plugin to import models and it works really well. I will of course provide credit. Thank you for your hard work :slight_smile:

6 Likes