How can I detect defined areas on model? (maybe with color detection)

Troubleshooting General Help
1

Hello guys, its me again :smile:

I would like to divide my model in subregion and detect which area is hit by mouse clicking. First time i thinking about collision boxes. But subregions are not allowed to be no overlap BUT they must cover it all. (therefore be partitioned)

My second thought to make two textures for model, some real (displayed) and one other at Background which is inked for different areas.

Now I have been looking for ways to get the color from the collisions point.
But I haven’t found a easy way.

So I fetched the collision triangle of model.


int triindex = closest.getTriangleIndex(); // triangle  contactpoint match
Geometry Model = closest.getGeometry(); //geometrie contactpoint match point
System.out.println("closest Objeckt" +Model.getName());
MatParamTexture texture = Model.getMaterial().getTextureParam("ColorMap");
VertexBuffer textureBuffer  = Model.getMesh().getBuffer(Type.TexCoord);  
VertexBuffer positionBuffer = Model.getMesh().getBuffer(Type.Position);
VertexBuffer color2Buffer = Model.getMesh().getBuffer(Type.Color);
IndexBuffer ib = Model.getMesh().getIndicesAsList();

But if I do something with color2Buffer like the follow, I get an display error
[Uncaught exception thrown in Thread[LWJGL Renderer Thread,6,main]java.lang.NullPointerException]



System.out.println("colorbuffer id: " +color2Buffer.toString());

Thus i tryed the following:


FloatBuffer textureFloatBuffer = (FloatBuffer) textureBuffer.getData();
FloatBuffer positionFloatBuffer = (FloatBuffer) positionBuffer.getData();
// aquire triangle's vertex indices 
int vertIndex = triindex * 3; // compute verticies of the mesh (triindex number of triangle) *3 number of points
// indices of index list
int vertindex1 = ib.get(vertIndex);       //1. triangle edge
int vertindex2 = ib.get(vertIndex + 1); //2. triangle edge
int vertindex3 = ib.get(vertIndex + 2); //3. triangle edge

// coordinats of edges
Vector3f px1 = new Vector3f();
Vector3f px2 = new Vector3f();
Vector3f px3 = new Vector3f();
// Updates values px of the given vector from positionFloatBuffer at vertext indeces.
BufferUtils.populateFromBuffer(px1, positionFloatBuffer, vertindex1);
BufferUtils.populateFromBuffer(px2, positionFloatBuffer, vertindex2);
BufferUtils.populateFromBuffer(px3, positionFloatBuffer, vertindex3);

// same for texture coordinats
final Vector2f tx1 = new Vector2f();
final Vector2f tx2 = new Vector2f();
final Vector2f tx3 = new Vector2f();               
BufferUtils.populateFromBuffer(tx1, textureFloatBuffer, vertindex1);  
BufferUtils.populateFromBuffer(tx2, textureFloatBuffer, vertindex2);
BufferUtils.populateFromBuffer(tx3, textureFloatBuffer, vertindex3);

Texture tex = texture.getTextureValue();
Image img = tex.getImage(); // returns the image data that makes up this texture.
List imgd = img.getData();
int size = imgd.size();
System.out.println("list buffer: " +size); // returns 1
// so i get only one ByteBuffer to use, from image
ByteBuffer imgbytebuffer = imgd.get(0);

//but why can I get the 4 color values from it (at right position)?
//the following returns for all the same value
float alpha = imgbytebuffer.get() / 255f;
float blue = imgbytebuffer.get() / 255f;
float green = imgbytebuffer.get() / 255f;

The ByteBuffer imgbytebuffer have only one index and i get only 4 equal values from it :confused:
But I have no idea, what can I make else.
In the tutorials I only read how can I set the color, never how can I get the color

If anyone could help, I would be very happy :blush: (maybe the @Pesegato ?)
Other ideas to implement the concept are also welcome :smiley:

Best regards and good night at all of you :}
EsKay.

2

Well, since you are just grabbing the first pixel from your texture and not actually the pixel you want… maybe the lower left corner of the texture does have all the same values.

3

Thx for your answer :} @pspeed

No matter where I click, I get allways the same Values

alpha: -0.10980392
red: -0.10980392
green: -0.10980392
blue: -0.10980392
The same value for all, which is logical, because I acess always to the same postion.
But how can I find the right position in the ByteBuffer o.O

good night (now realy)
I think tomorrow about it :]

4

Well, if the x + y * width math is not working for you…
http://javadoc.jmonkeyengine.org/com/jme3/texture/image/ImageRaster.html

5

I’m using something like that:

    public static void getTriangle(Mesh mesh, Type type, int index, Vector3f v1, Vector3f v2, Vector3f v3){
        VertexBuffer pb = mesh.getBuffer(type);
        IndexBuffer ib = mesh.getIndicesAsList();
        if (pb != null && pb.getFormat() == Format.Float && pb.getNumComponents() == 3){
            FloatBuffer fpb = (FloatBuffer) pb.getData();

            // acquire triangle's vertex indices
            int vertIndex = index * 3;
            int vert1 = ib.get(vertIndex);
            int vert2 = ib.get(vertIndex+1);
            int vert3 = ib.get(vertIndex+2);

            BufferUtils.populateFromBuffer(v1, fpb, vert1);
            BufferUtils.populateFromBuffer(v2, fpb, vert2);
            BufferUtils.populateFromBuffer(v3, fpb, vert3);
        }else{
            throw new UnsupportedOperationException(type + " buffer not set or "
                                                  + " has incompatible format");
        }
    }

as utility method and then

int ti = collision.getTriangleIndex();
                TempVars tmp = TempVars.get();
                try {
                    Vector3f p0 = tmp.vect1;
                    Vector3f p1 = tmp.vect2;
                    Vector3f p2 = tmp.vect3;
                    Vector2f t0 = tmp.vect2d;
                    Vector2f t1 = tmp.vect2d2;
                    Vector2f t2 = new Vector2f();
                    
                    
                    getTriangle(geom.getMesh(),VertexBuffer.Type.Position, ti, p0, p1, p2);
                    getTriangle(geom.getMesh(),VertexBuffer.Type.TexCoord, ti, t0, t1, t2);
                    
                    Vector3f cp = collision.getContactPoint();
                    geom.worldToLocal(cp,cp);

                    Vector3f vn = p2.subtract(p1, tmp.vect4).crossLocal(p1.subtract(p0,tmp.vect5));
                    float A = vn.length();
                    Vector3f n = tmp.vect6.set(vn).divideLocal(A);
                    float u = FastMath.abs((p2.subtract(p1,tmp.vect7).crossLocal(cp.subtract(p1,tmp.vect8))).dot(n) / A);
                    float v = FastMath.abs((p0.subtract(p2,tmp.vect7).crossLocal(cp.subtract(p2,tmp.vect8))).dot(n) / A);
                    float w = 1 - u - v;
                    
                    float s = t0.x * u + t1.x * v + t2.x * w;
                    float t = t0.y * u + t1.y * v + t2.y * w;

You end up with s,t being texture coordinates of clicked point. With that, you can lookup some image/texture which would define areas of your model as you wish.

1 Like
6

Wow Guys you are amazing :relaxed: old stuff from computer graphics come up.
I should know about this stuff, but I haven’t used it since that time and so I have forgotten much of the stuff. :confused:

@abies I will try your method when I have time. Sounds very good. :relaxed:
And then I will report again :}