Setting z index manually for transparent objects

richtea · January 7, 2018, 4:29pm

I’m trying to add a number of partially transparent markers on the horizon between the main world and the sky, like this:

[Album] imgur.com

I can’t just make them part of the sky image because they move independently from each other every frame. They work fine as long as they aren’t near each other but when they are they z fight:

[Album] imgur.com

Now I thought I could fix this by explicitly setting the z order:

    viewPort.getQueue().setGeometryComparator(RenderQueue.Bucket.Transparent, new TransparentComparator(){
        @Override
        public int compare(Geometry o1, Geometry o2) {
            if(o1 instanceof ForcedOrderGeometry && o2 instanceof ForcedOrderGeometry){
                ForcedOrderGeometry fog1 = (ForcedOrderGeometry)o1;
                ForcedOrderGeometry fog2 = (ForcedOrderGeometry)o2;
                
                return fog1.getOrder() -  fog2.getOrder();
            }else{
                return super.compare(o1, o2);
            }
        }
        
    });

But that doesn’t seem to be making a difference.

I’ve tried to create a self contained example, it doesn’t quite look the same but it looks equally wrong, without a custom geometry comparator:

[Album] imgur.com

With a custom geometry comparator (arguably worse):

[Album] imgur.com

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.material.RenderState;
import com.jme3.math.ColorRGBA;
import com.jme3.math.FastMath;
import com.jme3.math.Vector3f;
import com.jme3.renderer.queue.RenderQueue;
import com.jme3.renderer.queue.TransparentComparator;
import com.jme3.scene.Geometry;
import com.jme3.scene.Mesh;
import com.jme3.scene.shape.Quad;
import java.util.Random;

public class Main extends SimpleApplication {

    public static void main(String[] args) {
        Main app = new Main();
        app.start();
    }
    @Override
    public void simpleInitApp() {


        viewPort.getQueue().setGeometryComparator(RenderQueue.Bucket.Transparent, new TransparentComparator(){
            @Override
            public int compare(Geometry o1, Geometry o2) {
                if(o1 instanceof ForcedOrderGeometry && o2 instanceof ForcedOrderGeometry){
                    ForcedOrderGeometry fog1 = (ForcedOrderGeometry)o1;
                    ForcedOrderGeometry fog2 = (ForcedOrderGeometry)o2;

                    return fog1.getOrder() -  fog2.getOrder();
                }else{
                    return super.compare(o1, o2);
                }
            }

        });

        for(int i=0;i<10; i++){
            float angle = 0.05f * i;

            Geometry geo = createGeometry(i);

            float x = (500+i)*FastMath.sin(angle);
            float z = (500+i)*FastMath.cos(angle);

            this.rootNode.attachChild(geo);
            geo.setLocalTranslation(x, 0, z);
            geo.lookAt(Vector3f.ZERO, Vector3f.UNIT_Y);


        }
        cam.setLocation(new Vector3f(0,200,0));
        cam.lookAt(new Vector3f(0,0,500), Vector3f.UNIT_Y);

    }
    private Geometry createGeometry(int order){
        Random rnd = new Random(order);

        Mesh b = new Quad(100, 1000);

        Geometry geom = new ForcedOrderGeometry("forced", b, order);  
        Material mat = new Material(assetManager,"Common/MatDefs/Misc/Unshaded.j3md");  
        mat.setColor("Color", new ColorRGBA(rnd.nextFloat(), rnd.nextFloat(), rnd.nextFloat(), 1));  
        mat.setTexture("ColorMap", assetManager.loadTexture("Textures/horizonImage.png"));
        mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
        geom.setQueueBucket(RenderQueue.Bucket.Transparent);
        geom.setMaterial(mat);  

        return geom;
    }

  public static class ForcedOrderGeometry extends Geometry {

    int order; 

    public  ForcedOrderGeometry(String name, Mesh mesh, int order){
        super(name,mesh);
        this.order = order;
    }

    public int getOrder() {
        return order;
    }

    }

}

I don’t really care about what the order is for these objects just as long as the order is consistent between frames and doesn’t z fight. What do I need to do to make some of the transparent items are always completely in front of others independent of the true z distance (I seem to need to make them quite far apart in real space to make them not z fight and that causes other issues.

The_Leo · January 7, 2018, 4:45pm

Does

return fog2.getOrder() -  fog1.getOrder();

fix the issue?

richtea · January 7, 2018, 6:14pm

It actually does, but that if anything makes me more confused about what setGeometryComparator does. I though I could use it to tell it any z order I liked and thats the z order it would use, which should have lead to the image having strange backwards ordering but otherwise looking fine (and as these are all conceptually at the same distance looking right) but I didn’t expect the occluding.

Why does one order seem to work and the other not? Obviously one is physically accurate and the other is not but the whole point was to ignore what was physically accurate

The_Leo · January 7, 2018, 6:30pm

For correct alpha transparency, first draw the furthest object.
This way, the furthest geom is drawn, then the next one overlaps that one and so on.

If you use different order (closest first.)
The closest is drawn. The next one is drawn but no overlapping is done because the depth test fails.

richtea · January 7, 2018, 6:47pm

Ah, so GeometryComparator needs to be at least partially correct? GeometryComparator should just be considered a tie breaker?

(I know it’s not actually a tie breaker as it happens first but for my purposes it is, I can give it a different algoritem to “get it right” when z precision is an issue but I still need to get it right)

The_Leo · January 7, 2018, 7:46pm

Yes, for correct alpha transparency, the objects got to be rendered, furthest first.

The default GeometryComparator in Transparent queue already sorts objects furthest first.

So theoretically, it should produce the correct results for you already. However this is not the case, since looking at the TransparentComparator, it sorts them furthest first based on the center of each object.

pspeed · January 7, 2018, 10:00pm

Someone changed it then. It used to be based on the nearest point on the bounding volume or somesuch.

People waffle and change it thinking that there is a “right answer”. There is no proper Z sorting method… only some that work in some cases and not others. (It’s provably impossible to sort even triangles 100% correct using any method.)

But yes, transparent bucket needs “painter’s sort”, ie: back to front.

By the way, creating your own ForcedOrderGeometry is a super-ugly way to get the behavior that you want. What if you ever needed to extend geometry for some other reason. If you just want to tag a geometry with some behavior then just add a user data element. This is what Lemur’s layer-based geometry comparator does.

Edit: TransparentComparator does use nearest point… not sure what code you were checking:

github.com

jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/java/com/jme3/renderer/queue/TransparentComparator.java#L84


      
          /*
           * Copyright (c) 2009-2021 jMonkeyEngine
           * All rights reserved.
           *
           * Redistribution and use in source and binary forms, with or without
           * modification, are permitted provided that the following conditions are
           * met:
           *
           * * Redistributions of source code must retain the above copyright
           *   notice, this list of conditions and the following disclaimer.
           *
           * * Redistributions in binary form must reproduce the above copyright
           *   notice, this list of conditions and the following disclaimer in the
           *   documentation and/or other materials provided with the distribution.
           *
           * * Neither the name of 'jMonkeyEngine' nor the names of its contributors
           *   may be used to endorse or promote products derived from this software
           *   without specific prior written permission.
           *
           * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
           * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
           * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
           * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
           * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
           * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
           * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
           * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
           * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
           * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
           * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
           */
          package com.jme3.renderer.queue;
          
          import com.jme3.math.Vector3f;
          import com.jme3.renderer.Camera;
          import com.jme3.scene.Geometry;
          
          public class TransparentComparator implements GeometryComparator {
          
              private Camera cam;
              private final Vector3f tempVec = new Vector3f();
          
              @Override
              public void setCamera(Camera cam){
                  this.cam = cam;
              }
          
              private float distanceToCam(Geometry spat){
                  // NOTE: It is best to check the distance
                  // to the bound's closest edge vs. the bound's center here.
                  return spat.getWorldBound().distanceToEdge(cam.getLocation());
              }
          
              @Override
              public int compare(Geometry o1, Geometry o2) {
                  float d1 = distanceToCam(o1);
                  float d2 = distanceToCam(o2);
          
                  if (d1 == d2)
                      return 0;
                  else if (d1 < d2)
                      return 1;
                  else
                      return -1;
              }
          }

The_Leo · January 7, 2018, 10:33pm

Yes, that’s true, eg. two crossed planes can not be properly sorted in either way.

Yes, my mistake, i checked the distanceToCam2 method in TransparentComparator that is not at all used.

Thou, in his test case, neither, (closest edge and closest center) work. Since the objects are too close together and their bounds are overlapping.

pspeed · January 7, 2018, 10:40pm

Also, if you see actual “z-fighting” then it means those triangles actually have the same Z and sorting won’t really help you.

If it’s just a blending issue then sorting can help… but “z fighting” means something very specific related to the depth buffer.

Probably for these you should not be depth testing at all (or not depth writing) in which case you may find that order doesn’t even matter anymore.

richtea · January 8, 2018, 6:41pm

I think I saw z fighting in the real game (tiny changes in camera angle/direction lead to different geometries (or parts of geometries) being the one in front?) but I couldn’t reproduce it in the toy example so possibly a different issue somewhere in my code.

“Probably for these you should not be depth testing at all (or not depth writing) in which case you may find that order doesn’t even matter anymore.”

That sounds fair, I tried mat.getAdditionalRenderState().setDepthTest(false); which made these merge beautifully but made them appear in front of everything else (which I suppose isn’t surprising). Is that what you meant?

pspeed · January 8, 2018, 7:01pm

I thought they were in the sky bucket… can they be put into the sky bucket?