3d Paint in JME3?

Hey all complete noob here, throw me a link if this has been covered.



Can you do real-time 3D painting on models with JME3? I saw some very old (3-4 years ago) threads on painting on terrain, but they referred to tests that I’m sure pre-dates JME3 (at least I did not see them in the current alpha). Similar question what about the application of decals?



What I’m thinking of is something for virtual miniatures/table top gaming and I would like for the users to be able to customize their minis by painting directly on them and/or applying decals and textures dynamically.



My past game development stuff has been on a completely different environment (Dark Basic Pro) and I’m looking for a new environment so I can get back up to speed on Java programming again.



TAZ

It is possible, but you’ll have to know what you’re doing to accomplish this. There’s no “built-in” solution that you can just take and use.



In short, you cast a ray from the mouse to the scene and find the barycentric coordinates of the collided triangle + the tri index, you fetch the UV coordinates of the mesh based on the triangle index and then you use the barycentric coordinates to interpolate along the UV to find the position on the texture that the user clicked. You can’t just paint in that area unfortunately as you might go onto other triangles that are not visible in the view, so you have to somehow “cull” the paint so it doesn’t appear on those triangles (this part I am unsure about).



I recommend looking into the Blender3D source code to find out how to do this, they have a “texture paint” feature that doesn’t have the issue I talked about.

Interesting, I had not thought about looking at the Blender source for inspiration.



Yeah, I figured it would not be a trvial thing to build from scratch. Unless you went out of your way to make sure that your model’s UV coordinates don’t overlap, you are quite likely to paint in one area and have it pop up in a completely unrelated triangle.



A technically simplier solution would be polygon painting, but that would be more like coloring book technology rather than painting on a surface. But, that may still be viable for making unit colors and the like, perhaps blending polygon color with an underlying texture to keep some details. I generally want to avoid vertex coloring since that is decidedly not the look that I’m after. [Especially for low polygon models.]



I’ll have to look into Blender’s texture paint functionality. While I despise the program’s UI, if there’s an easy way to get in, do the job, and get out. I might have to reconsider it for part of the art pipeline.



Thanks for the info.



TAZ

Oh my, I just watched some Blender tutorials on texture painting and UV Mapping. It’s still incredibly F’ed up. Also watched videos on Body Paint 3d, Deep Paint 3D, Tattoo, Blacksmith 3D, and more. That’s more like what I’m gunning for. Unfortunately, those are all very pricey solutions intended for professionals while I need something more like MS Paint for 3D. :slight_smile:



Sculptris certainly has potential since the alpha is free, but the program is too buggy and crashes regularly if the geometry is not to its liking. The hunt continues…



I chatted with my son about this topic and he agreed that something as simple MS paint would be the way to go. Face picking and coloration/texture application would be the absolute minimum. Decal deployment should be easy as bullet holes. The end result might be spending a lot more time in UV unwrapping models so that they’re easier to paint (no overlapping triangles, groups split up, etc.)



Another idea was to apply paint like a shot gun where each ‘pixel’ of paint gets its own vector and its plotted on where it strikes the model. Not sure what kind of overhead ‘shooting’ dozens or hundreds of bits of paint per frame would cause.



TAZ

@ Momoko_Fan, I would appreciate it if you could give me some guidance for how to get UV coordinates out of a triangle. I have spent several hours searching for methods to get to UV coordinates without avail.



Thanks,

Rishi

For a first step you will need the barycentric coordinates, those are not provided by the default intersection method so you will have to do another intersection test once you get the collided triangle.



For that you will need to use Ray.intersectWherePlanar(). Here’s one of the comments in that method:



[java]// these weights can be used to determine

// interpolated values, such as texture coord.

// eg. texcoord s,t at intersection point:

// s = w0s0 + w1s1 + w2s2;

// t = w0
t0 + w1t1 + w2t2;

float w1 = dirDotDiffxEdge2 * inv;

float w2 = dirDotEdge1xDiff * inv;

//float w0 = 1.0f - w1 - w2;

store.set(t, w1, w2);[/java]

Specifically the w* values give you the weight amount for each triangle vertex so you can interpolate the 3 texture coordinates according to that. Note that “store” is actually the vector you give to intersectWherePlanar().

1 Like

Thanks! I am looking into Barycentric coordinates now. Another question that’s been bugging me is if there is a way to access UV mapping information for vertexes. All of the models I use are UV mapped in modeling programs, so I thought getting UV coords would be a simple matter of choosing a vertex and looking up the UV for it, specificialy. Although this UV coordinate wouldn’t be exactly where the painting occurs, it could be close enough in a high-poly model. Is this approach feasible?

You can access the vertex coordinates by using Mesh.getFloatBuffer(VertexBuffer.Type.TexCoord)

@Momoko_Fan



hello,



I tried my best to accomplish this, but I can’t get the ray.intersectWherePlanar() to work. The ray.intersectWherePlanar() always return false.



Any advice?





pre type="java"
    @Override


    public void simpleUpdate(float tpf){


        Vector3f origin    = cam.getWorldCoordinates(inputManager.getCursorPosition(), 0.0f);


        Vector3f direction = cam.getWorldCoordinates(inputManager.getCursorPosition(), 0.3f);


        direction.subtractLocal(origin).normalizeLocal();





        Ray ray = new Ray(origin, direction);


        CollisionResults results = new CollisionResults();


        shootables.collideWith(ray, results);





        if (results.size() > 0) {


            


            Triangle tr = new Triangle();


            Vector3f res = new Vector3f();


            


            CollisionResult closest = results.getClosestCollision();


            closest.getTriangle(tr);


            


            System.out.println(ray.intersectWherePlanar(tr, res));





        } else {}


    }
/pre

Since you have the triangle index you can just fetch the vertex indices for it and then read the texcoords from there. Also right now you’re generating arrays for the native buffers which also costs a lot of performance.

Hi guys,



this is how I implemented the above, to paint directly on the texture. It is working well now.



I have to questions to improve my work:


  1. to get the UV coords of the vertices of the collided triangle (needed for interpolation), do I really need to scroll the entire buffer? Is there a more efficient way to do that (maybe a quick way to get the correct indices that point to the texcoord buffer)?


  2. if the anwer is that I really need to scroll the entire buffer, therefore is there a better way to detect if the collided triangle matches the tested triangle? I found no better way other than testing the “barycenter” of the two triangles, but this approach doesn’t satisfy my at all, I am sure that there must be a better way to test.



    Any suggestions?



    My code below:





    pre type="java"
        private void detectCollision() {


            


            //Create the ray for rayCasting


            Vector3f origin    = cam.getWorldCoordinates(inputManager.getCursorPosition(), 0.0f);


            Vector3f direction = cam.getWorldCoordinates(inputManager.getCursorPosition(), 0.3f);


            direction.subtractLocal(origin).normalizeLocal();


            Ray ray = new Ray(origin, direction);





            //Collision test


            CollisionResults results = new CollisionResults();


            character.collideWith(ray, results);





            //If the ray collides…


            if (results.size() > 0) {


                


                //retreive the closest collision


                CollisionResult closest = results.getClosestCollision();





                //Get the collided triangle


                Triangle collidedTriangle = new Triangle();


                closest.getTriangle(collidedTriangle);


                


                //We need to transform the triangle according to the spatial’s rotation/scale/position in order to intersect with ray successfully


                collidedTriangle.set1(character.getLocalRotation().mult(collidedTriangle.get1()));


                collidedTriangle.set2(character.getLocalRotation().mult(collidedTriangle.get2()));


                collidedTriangle.set3(character.getLocalRotation().mult(collidedTriangle.get3()));


                collidedTriangle.set1(character.getLocalTranslation().add(collidedTriangle.get1()));


                collidedTriangle.set2(character.getLocalTranslation().add(collidedTriangle.get2()));


                collidedTriangle.set3(character.getLocalTranslation().add(collidedTriangle.get3()));


                collidedTriangle.set1(character.getLocalScale().mult(collidedTriangle.get1()));


                collidedTriangle.set2(character.getLocalScale().mult(collidedTriangle.get2()));


                collidedTriangle.set3(character.getLocalScale().mult(collidedTriangle.get3()));


                


                //Get the barycentric coordinates


                Vector3f collisionResult = new Vector3f();


                ray.intersectWherePlanar(collidedTriangle, collisionResult);


                


                //Calculate the barycentric weights


                float w1 = collisionResult.getY();


                float w2 = collisionResult.getZ();


                float w0 = 1.0f - w1 - w2; 


                


                //We need to retreive the UV (ST) coords of the vertices of the collision.


                //DO WE REALLY NEED TO SCROLL THE ENTIRE BUFFER HERE?!?


                VertexBuffer index = character.getMesh().getBuffer(Type.Index);


                short[] indexArray = getShortArray((ShortBuffer) index.getData());


                VertexBuffer vertex = character.getMesh().getBuffer(Type.Position);


                float[] vertexArray = BufferUtils.getFloatArray((FloatBuffer) vertex.getData());


                VertexBuffer tex = character.getMesh().getBuffer(Type.TexCoord);


                float[] texArray = BufferUtils.getFloatArray((FloatBuffer) tex.getData());


                


                float s0 = 0, s1 = 0, s2 = 0, t0 = 0, t1 = 0, t2 = 0;


                


                for (int i = 0; i < indexArray.length; i += 3) {


                    


                    short index1 = indexArray;


                    short index2 = indexArray[i&nbsp;+&nbsp;1];


                    short index3 = indexArray[i&nbsp;+&nbsp;2];





                    Vector3f p1 = new Vector3f(vertexArray[index1  3], vertexArray[index1  3 + 1], vertexArray[index1  3 + 2]);


                    Vector3f p2 = new Vector3f(vertexArray[index2 
     3], vertexArray[index2  3 + 1], vertexArray[index2  3 + 2]);


                    Vector3f p3 = new Vector3f(vertexArray[index3  3], vertexArray[index3  3 + 1], vertexArray[index3  3 + 2]);





                    Triangle currentTriangle = new Triangle(p1, p2, p3);


                    


                    //Test whether the current triangle is the collided triangle


                    //IS THERE A BETTER WAY TO ACCOMPLISH THIS?!?


                    if (currentTriangle.getCenter().getX() == collidedTriangle.getCenter().getX() && currentTriangle.getCenter().getY() == collidedTriangle.getCenter().getY() && currentTriangle.getCenter().getZ() == collidedTriangle.getCenter().getZ()) {


                        


                        s0 = texArray[index1 
     2]; t0 = texArray[index1  2 + 1];


                        s1 = texArray[index2 
     2]; t1 = texArray[index2  2 + 1];


                        s2 = texArray[index3 
     2]; t2 = texArray[index3  2 + 1];


                        break;


                    }


                }





                //Calculate the interpolated UV


                float u = w0
    s0 + w1s1 + w2s2;


                float v = w0t0 + w1t1 + w2t2;


                


                //Paint the texture according to the new calculated UV


                paintableTexture.setPixel((int)(u 
     TEXTURE_SIZE), (int)(v * TEXTURE_SIZE), Color.yellow);


                //Set the texture for refresh


                mat_tl.setTexture(“DiffuseMap”, paintableTexture.getTexture());


                


            }  else {}


            


        }
    /pre

Sorry for the very noob question, but how triangle index relates to vertex indices?



Can you post me a link where to learn that please?

Answered myself:



pre type="java"
            int index1 = indexBuffer.get(collidedTriangle.getIndex()  3);


            int index2 = indexBuffer.get(collidedTriangle.getIndex() 
 3+1);


            int index3 = indexBuffer.get(collidedTriangle.getIndex() * 3+2);
/pre

1 Like