Streamlining Material Editing: A JSON Proposal for j3md/j3m Files

General Discussion
1

Hi everyone,

I’m exploring a potential improvement for material editing.
While developing my material editor, I had to extract types and variable names from the .j3md file by reading the lines one by one and parsing spaces, colons, brackets (the SDK does the same thing). This approach is not very robust and feels limited.

Could a JSON format for j3md/j3m files offer a simpler and more robust approach? This wouldn’t be a core engine change but could be an external project.

Storing FloatArray and TextureArray data in the Material.json file would provide the flexibility to configure these elements directly in the JSON format. This functionality is not currently available in the .j3m format (see AdvancedPBRTerrain.j3md).

This approach could streamline material editing and potentially benefit future development.
What are your thoughts? Does anyone want to take up the challenge?

*** Edit:
For example, here is the PBRLighting.j3md file converted to .json format:

{
  "MaterialDef": {
    "name": "PBR Lighting", // Assuming this is the material name
    "MaterialParameters": [
      {
        "type": "Int",
        "name": "BoundDrawBuffer",
        "value": null // Assuming no default value provided in the original definition
      },
      {
        "type": "Float",
        "name": "AlphaDiscardThreshold",
        "value": 0.0 // Assuming a default value
      },
      {
        "type": "Float",
        "name": "Metallic",
        "value": 1.0
      },
      {
        "type": "Color",
        "name": "Specular",
        "value": [1.0, 1.0, 1.0, 1.0]
      },
      {
        "type": "Float",
        "name": "Glossiness",
        "value": 1.0
      },
      {
        "type": "Texture2D",
        "name": "ParallaxMap",
        "colorSpace": "Linear"
      }
      // ... and so on ...
    ],
    "Techniques": [
      {
        "name": "Default", // Assuming technique name can be extracted
        "LightMode": "SinglePassAndImageBased",
        "VertexShader": "Common/MatDefs/Light/PBRLighting.vert",
        "FragmentShader": "Common/MatDefs/Light/PBRLighting.frag",
        "ShaderLanguages": ["GLSL300", "GLSL150", "GLSL110"],
        "WorldParameters": [
          "WorldViewProjectionMatrix",
          "CameraPosition",
          "WorldMatrix",
          "WorldNormalMatrix",
          "ViewProjectionMatrix",
          "ViewMatrix"
        ],
        "Defines": [
          {
            "name": "BOUND_DRAW_BUFFER",
            "param": "BoundDrawBuffer"
          },
          {
            "name": "BASECOLORMAP",
            "param": "BaseColorMap"
          },
          {
            "name": "VERTEX_COLOR",
            "param": "UseVertexColor"
          }
          // ... other defines mapping parameters to names ...
        ]
      },
      {
        "name": "PreShadow", // Assuming technique name can be extracted
        "VertexShader": "Common/MatDefs/Shadow/PreShadow.vert",
        "FragmentShader": "Common/MatDefs/Shadow/PreShadowPBR.frag",
        "ShaderLanguages": ["GLSL300", "GLSL150", "GLSL110" ],
        "WorldParameters": [
          "WorldViewProjectionMatrix",
          "WorldViewMatrix",
          "ViewProjectionMatrix",
          "ViewMatrix"
        ],
        "Defines": [
          {
            "name": "BOUND_DRAW_BUFFER",
            "param": "AlphaDiscardThreshold"
          },
          {
            "name": "DISCARD_ALPHA",
            "param": "AlphaDiscardThreshold"
          }
          // ... other defines mapping parameters to names ...
        ],
        "ForcedRenderState": {
          "FaceCull": "Off",
          "DepthTest": true,
          "DepthWrite": true,
          "PolyOffset": [5, 3],
          "ColorWrite": false
        }
      }
      // ... other techniques following the same structure ...
    ]
  }
}
3 Likes
2

My library typed materials does something similar to this (see also Statically Typed Materials).

Typed materials generates java classes to help interacting with the materials. E.g. here is one class it generates for unshaded.j3md

/**
 * AUTOGENERATED CLASS, do not modify
 * <p>
 * Generated from Common/MatDefs/Misc/Unshaded.j3md in library jme3-core-3.6.1-stable by the Typed Materials plugin. For more information:
 * </p>
 * @see <a href="https://github.com/oneMillionWorlds/TypedMaterials/wiki">MaterialTyper Plugin Wiki</a>
 */
@SuppressWarnings("all")
public class UnshadedMaterial extends Material {

    /**
    * Do not use this constructor. Serialization purposes only.
    */
    public UnshadedMaterial() {
        super();
    }

    public UnshadedMaterial(AssetManager contentMan) {
        super(contentMan, "Common/MatDefs/Misc/Unshaded.j3md");
    }

    public void setColorMap(Texture colorMap){
        setTexture("ColorMap", colorMap);
    }
    
    public Texture getColorMap(){
         return getParamValue("ColorMap");
    }
    
    public void setLightMap(Texture lightMap){
        setTexture("LightMap", lightMap);
    }
    
    public Texture getLightMap(){
         return getParamValue("LightMap");
    }
    
    public void setColor(ColorRGBA color){
        setColor("Color", color);
    }
    
    public ColorRGBA getColor(){
         return getParamValue("Color");
    }
    
    public void setVertexColor(boolean vertexColor){
        setParam("VertexColor", VarType.Boolean,  vertexColor);
    }
    
    public boolean getVertexColor(){
         return getParamValue("VertexColor");
    }
    
    public void setPointSize(float pointSize){
        setFloat("PointSize", pointSize);
    }
    
    public float getPointSize(){
         return getParamValue("PointSize");
    }
    
    public void setSeparateTexCoord(boolean separateTexCoord){
        setParam("SeparateTexCoord", VarType.Boolean,  separateTexCoord);
    }
    
    public boolean getSeparateTexCoord(){
         return getParamValue("SeparateTexCoord");
    }
    
    /**
     *  Texture of the glowing parts of the material
     */
    public void setGlowMap(Texture glowMap){
        setTexture("GlowMap", glowMap);
    }
    
    
    /**
     *  Texture of the glowing parts of the material
     */
    public Texture getGlowMap(){
         return getParamValue("GlowMap");
    }
    
    /**
     *  The glow color of the object
     */
    public void setGlowColor(ColorRGBA glowColor){
        setColor("GlowColor", glowColor);
    }
    
    
    /**
     *  The glow color of the object
     */
    public ColorRGBA getGlowColor(){
         return getParamValue("GlowColor");
    }
    
    /**
     *  For instancing
     */
    public void setUseInstancing(boolean useInstancing){
        setParam("UseInstancing", VarType.Boolean,  useInstancing);
    }
    
    
    /**
     *  For instancing
     */
    public boolean getUseInstancing(){
         return getParamValue("UseInstancing");
    }
    
    /**
     *  For hardware skinning
     */
    public void setNumberOfBones(int numberOfBones){
        setInt("NumberOfBones", numberOfBones);
    }
    
    
    /**
     *  For hardware skinning
     */
    public int getNumberOfBones(){
         return getParamValue("NumberOfBones");
    }
    
    public void setBoneMatrices(Matrix4f[] boneMatrices){
        setParam("BoneMatrices", VarType.Matrix4Array,  boneMatrices);
    }
    
    public Matrix4f[] getBoneMatrices(){
         return getParamValue("BoneMatrices");
    }
    
    /**
     *  For Morph animation
     */
    public void setMorphWeights(float[] morphWeights){
        setParam("MorphWeights", VarType.FloatArray,  morphWeights);
    }
    
    
    /**
     *  For Morph animation
     */
    public float[] getMorphWeights(){
         return getParamValue("MorphWeights");
    }
    
    public void setNumberOfMorphTargets(int numberOfMorphTargets){
        setInt("NumberOfMorphTargets", numberOfMorphTargets);
    }
    
    public int getNumberOfMorphTargets(){
         return getParamValue("NumberOfMorphTargets");
    }
    
    public void setNumberOfTargetsBuffers(int numberOfTargetsBuffers){
        setInt("NumberOfTargetsBuffers", numberOfTargetsBuffers);
    }
    
    public int getNumberOfTargetsBuffers(){
         return getParamValue("NumberOfTargetsBuffers");
    }
    
    /**
     *  Alpha threshold for fragment discarding
     */
    public void setAlphaDiscardThreshold(float alphaDiscardThreshold){
        setFloat("AlphaDiscardThreshold", alphaDiscardThreshold);
    }
    
    
    /**
     *  Alpha threshold for fragment discarding
     */
    public float getAlphaDiscardThreshold(){
         return getParamValue("AlphaDiscardThreshold");
    }
    
    /**
     * Shadows
     */
    public void setFilterMode(int filterMode){
        setInt("FilterMode", filterMode);
    }
    
    
    /**
     * Shadows
     */
    public int getFilterMode(){
         return getParamValue("FilterMode");
    }
    
    public void setHardwareShadows(boolean hardwareShadows){
        setParam("HardwareShadows", VarType.Boolean,  hardwareShadows);
    }
    
    public boolean getHardwareShadows(){
         return getParamValue("HardwareShadows");
    }
    
    public void setShadowMap0(Texture shadowMap0){
        setTexture("ShadowMap0", shadowMap0);
    }
    
    public Texture getShadowMap0(){
         return getParamValue("ShadowMap0");
    }
    
    public void setShadowMap1(Texture shadowMap1){
        setTexture("ShadowMap1", shadowMap1);
    }
    
    public Texture getShadowMap1(){
         return getParamValue("ShadowMap1");
    }
    
    public void setShadowMap2(Texture shadowMap2){
        setTexture("ShadowMap2", shadowMap2);
    }
    
    public Texture getShadowMap2(){
         return getParamValue("ShadowMap2");
    }
    
    public void setShadowMap3(Texture shadowMap3){
        setTexture("ShadowMap3", shadowMap3);
    }
    
    public Texture getShadowMap3(){
         return getParamValue("ShadowMap3");
    }
    
    /**
     * pointLights
     */
    public void setShadowMap4(Texture shadowMap4){
        setTexture("ShadowMap4", shadowMap4);
    }
    
    
    /**
     * pointLights
     */
    public Texture getShadowMap4(){
         return getParamValue("ShadowMap4");
    }
    
    public void setShadowMap5(Texture shadowMap5){
        setTexture("ShadowMap5", shadowMap5);
    }
    
    public Texture getShadowMap5(){
         return getParamValue("ShadowMap5");
    }
    
    public void setShadowIntensity(float shadowIntensity){
        setFloat("ShadowIntensity", shadowIntensity);
    }
    
    public float getShadowIntensity(){
         return getParamValue("ShadowIntensity");
    }
    
    public void setSplits(Vector4f splits){
        setParam("Splits", VarType.Vector4,  splits);
    }
    
    public Vector4f getSplits(){
         return getParamValue("Splits");
    }
    
    public void setFadeInfo(Vector2f fadeInfo){
        setParam("FadeInfo", VarType.Vector2,  fadeInfo);
    }
    
    public Vector2f getFadeInfo(){
         return getParamValue("FadeInfo");
    }
    
    public void setLightViewProjectionMatrix0(Matrix4f lightViewProjectionMatrix0){
        setParam("LightViewProjectionMatrix0", VarType.Matrix4,  lightViewProjectionMatrix0);
    }
    
    public Matrix4f getLightViewProjectionMatrix0(){
         return getParamValue("LightViewProjectionMatrix0");
    }
    
    public void setLightViewProjectionMatrix1(Matrix4f lightViewProjectionMatrix1){
        setParam("LightViewProjectionMatrix1", VarType.Matrix4,  lightViewProjectionMatrix1);
    }
    
    public Matrix4f getLightViewProjectionMatrix1(){
         return getParamValue("LightViewProjectionMatrix1");
    }
    
    public void setLightViewProjectionMatrix2(Matrix4f lightViewProjectionMatrix2){
        setParam("LightViewProjectionMatrix2", VarType.Matrix4,  lightViewProjectionMatrix2);
    }
    
    public Matrix4f getLightViewProjectionMatrix2(){
         return getParamValue("LightViewProjectionMatrix2");
    }
    
    public void setLightViewProjectionMatrix3(Matrix4f lightViewProjectionMatrix3){
        setParam("LightViewProjectionMatrix3", VarType.Matrix4,  lightViewProjectionMatrix3);
    }
    
    public Matrix4f getLightViewProjectionMatrix3(){
         return getParamValue("LightViewProjectionMatrix3");
    }
    
    /**
     * pointLight
     */
    public void setLightViewProjectionMatrix4(Matrix4f lightViewProjectionMatrix4){
        setParam("LightViewProjectionMatrix4", VarType.Matrix4,  lightViewProjectionMatrix4);
    }
    
    
    /**
     * pointLight
     */
    public Matrix4f getLightViewProjectionMatrix4(){
         return getParamValue("LightViewProjectionMatrix4");
    }
    
    public void setLightViewProjectionMatrix5(Matrix4f lightViewProjectionMatrix5){
        setParam("LightViewProjectionMatrix5", VarType.Matrix4,  lightViewProjectionMatrix5);
    }
    
    public Matrix4f getLightViewProjectionMatrix5(){
         return getParamValue("LightViewProjectionMatrix5");
    }
    
    public void setLightPos(Vector3f lightPos){
        setParam("LightPos", VarType.Vector3,  lightPos);
    }
    
    public Vector3f getLightPos(){
         return getParamValue("LightPos");
    }
    
    public void setLightDir(Vector3f lightDir){
        setParam("LightDir", VarType.Vector3,  lightDir);
    }
    
    public Vector3f getLightDir(){
         return getParamValue("LightDir");
    }
    
    public void setPCFEdge(float pCFEdge){
        setFloat("PCFEdge", pCFEdge);
    }
    
    public float getPCFEdge(){
         return getParamValue("PCFEdge");
    }
    
    public void setShadowMapSize(float shadowMapSize){
        setFloat("ShadowMapSize", shadowMapSize);
    }
    
    public float getShadowMapSize(){
         return getParamValue("ShadowMapSize");
    }
    
    public void setBackfaceShadows(boolean backfaceShadows){
        setParam("BackfaceShadows", VarType.Boolean,  backfaceShadows);
    }
    
    public boolean getBackfaceShadows(){
         return getParamValue("BackfaceShadows");
    }
    
    /**
     *  1.0 indicates 100% desaturation
     */
    public void setDesaturationValue(float desaturationValue){
        setFloat("DesaturationValue", desaturationValue);
    }
    
    
    /**
     *  1.0 indicates 100% desaturation
     */
    public float getDesaturationValue(){
         return getParamValue("DesaturationValue");
    }


}

I’m probably most of the way to generating that json file you describe, I could easily enough add a json output as well/instead of a java file output. Typed materials works during the gradle build, would that work for you?

1 Like
3

One json file for all materials or one per material? It would end up in the generated resources folder after the build.

And do you care about technique or just material parameters? Ive only parsed material parameters so far (because what i wanted was to create getters and setters for each one)

4

I haven’t thought about the details yet, but I would imagine something to serialize/deserialize the Material and MaterialDef classes in json format instead of j3m/j3md.

Based on the existing classes, I propose creating new classes with similar characteristics but focused on json data processing:

*** Edit:
usage example:

assetManager.registerLoader(JsonMaterialLoader.class, "json");
JsonMaterialKey key = new JsonMaterialKey("Materials/PBRMaterial.json"); 
Material mat = assetManager.loadAsset(key);
5

Interesting, so this would need to happen at run time, not at build time? In that case why not just ask the Material itself?

Material mat = new Material(getAssetManager())
mat.getMaterialDef().getMaterialParams();
mat.getMaterialDef().getTechniqueDefsNames();

(This isn’t an option for TypedMaterials because (a) it’s happening before the main build and (b) I want to grab comments out of the file)

Typed Materials does a good job of starting with a j3md file as a string and making some sense out of it. But if JMonkey itself is available it will do a better job. If on the other hand you want that json file produced at built time (or in a context where JMonkey isn’t available), sign me up

1 Like
6

JSON doesn’t support comments, this alone rules it out for anything other than machine to machine communication for me.
As I edit the .j3m(d) files manually and would really hate switching to it.

I see the need to generate and parse files with tools so if you need a standard format that can be used with code and is not (so) hard to work with in a text editor I’d suggest YAML (although it has many quirks too).

2 Likes
7

Why would you do this instead of using JME’s parser? As I recall, it’s not hard to get J3MLoader coerced into being called directly… though spinning up a temporary asset manager is not hard, either.

Pretty sure you can just pass your own AssetInfo in, though.

Then you wouldn’t even have to worry about odd edge cases cropping up.

8

Unfortunately, it’s not feasible. There’s a small issue in the graphical component creation algorithm that prevents distinguishing between a Vector4 parameter and a ColorRGBA. Analyzing the SDK code reveals the reason is logical and I’ll explain it later. (see EditableMaterialFile @author normenhansen).

–

In the meantime, I’ve created a working prototype Material.json file. This format, along with the JsonMaterialLoader and JsonMaterialKey, is clearer, more flexible, and less error-prone than the current one.

I tested the YAML format but encountered interpretation and portability issues. I might revisit it later.

The next step is writing a parser for MaterialDef.json files (see the prototype in my first post). The solution should be compatible with the existing code.

I’ll make the project public once I have promising results.

I’m always open to suggestions for improvement.

Here Material.json prototype:
While the texture definition block is functional, it may require modification to support TextureArray parameters. In such cases, the block might need to be restructured as an array of elements to accommodate multiple textures.

{
	"Material": {
		"name": "MyMaterial",
		"def": "Common/MatDefs/Light/Lighting.j3md",
		"materialParameters": [
			{
				"name": "Diffuse",
				"value": [0.505882,0.505882,0.505882,1]
			},
			{
				"name": "UseMaterialColors",
				"value": true
			},
			{
				"name": "Ambient",
				"value": [0.2,0.2,0.2,1]
			},
			{
				"name": "DiffuseMap",
				"texture" : {
					"path" : "Models/Ferrari/car.jpg",
					"wrapMode" : "Repeat",
					"minFilter" : "BilinearNoMipMaps",
					"magFilter" : "Bilinear",
					"flipY" : false
				}
			},
			{
				"name": "Specular",
				"value": [0.5,0.5,0.5,1]
			},
			{
				"name": "Shininess",
				"value": 12.5
			}
		],
		"additionalRenderState": {
			"faceCull": "Back",
			"blend": "Off",
			"depthWrite": true,
			"colorWrite": true,
			"depthTest": true,
			"wireframe": false,
			"polyOffset": [1.0,1.0]
		}
	}
}
9

As long as we’re clear that this is never ever replacing j3m and j3md files.

Seems like this would come up anyway… even with a json format.

10

This is what I would do if I were in your position. I’d make my editor have its own format with all the options I need and then the user would click a button to export to .j3m when they are ready.
The hard part is if you want to import an existing .j3m, in that case you can support it with limitations, this is not something the user would not do often if ever.

This is how practically all design software works: we edit a .blender file and then export it to .gltf, .xcf → .png (.java to .class even).
The editor shouldn’t work on the target format directly, if you want to make the format open that’s fine too, someone can make an exporter for another engine.

2 Likes