Line is not pickable

I want pick a Line with my mouse pointer but it doesn’t work.
The following code demonstrate the behavior. The class creates a Line and a Box object and when I move over the Box then it prints 2. But at the Line happens nothing.

Is this a bug or a desired behavior? Is it possible to pick up a Line?

import com.jme3.app.SimpleApplication;
import com.jme3.collision.CollisionResults;
import com.jme3.input.controls.MouseAxisTrigger;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.shape.Line;
import com.jme3.input.MouseInput;
import com.jme3.input.controls.AnalogListener;
import com.jme3.math.Ray;
import com.jme3.math.Vector2f;
import com.jme3.scene.shape.Box;

public class LinePickTest extends SimpleApplication {

    Node shootables = new Node();

    @Override
    public void simpleInitApp() {
        flyCam.setEnabled(false);
        shootables.attachChild(makeLine("line"));
        shootables.attachChild(makeCube("a Dragon", -2f, 0f, 1f));
        rootNode.attachChild(shootables);

        inputManager.addMapping("move", new MouseAxisTrigger(MouseInput.AXIS_X, true),
                new MouseAxisTrigger(MouseInput.AXIS_X, false), new MouseAxisTrigger(MouseInput.AXIS_Y, true),
                new MouseAxisTrigger(MouseInput.AXIS_Y, false)/*, new MouseButtonTrigger(MouseInput.BUTTON_LEFT)*/);
        inputManager.addListener(new AnalogListener() {
            public void onAnalog(String name, float value, float tpf) {
                Vector2f cursorPosition = inputManager.getCursorPosition();
                Vector3f origin = cam.getWorldCoordinates(cursorPosition, 0.0f);
                Vector3f direction = cam.getWorldCoordinates(cursorPosition, 0.3f);
                direction.subtractLocal(origin).normalizeLocal();

                Ray ray = new Ray(origin, direction);
                CollisionResults results = new CollisionResults();

                shootables.collideWith(ray, results);

                System.out.println(results.size());
            }
        }, "move");
    }

    protected Geometry makeLine(String name) {
        Line line = new Line(Vector3f.ZERO, Vector3f.UNIT_XYZ.mult(3));
        Geometry cube = new Geometry(name, line);
        line.setLineWidth(9);
        Material mat1 = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat1.setColor("Color", ColorRGBA.randomColor());
        cube.setMaterial(mat1);
        return cube;
    }

    protected Geometry makeCube(String name, float x, float y, float z) {
        Box box = new Box(1, 1, 1);
        Geometry cube = new Geometry(name, box);
        cube.setLocalTranslation(x, y, z);
        Material mat1 = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat1.setColor("Color", ColorRGBA.randomColor());
        cube.setMaterial(mat1);
        return cube;
    }

    public static void main(String[] args) {
        new LinePickTest().start();
    }
}

Lines have zero width and can’t be picked by other rays without writing your own code.

There are already example code for this?
In addition I need a picking tolerance. It is very difficult to pick a thin line with the mouse.

I’m sure somewhere on the web… but nothing JME specific. Extremely few need this.

If these lines are in 3D, then I think what you are looking for is something to calculate the shortest distance between a line and a line segment. (Or two lines and then clip to the line segment.) Then anything under a certain threshold might be “picked”.

Ok.
I think it must be enought to overwrite the collide method in Line.

 public static class PickableLine extends Line{

        @Override
        public int collideWith(Collidable other, Matrix4f worldMatrix, BoundingVolume worldBound, CollisionResults results) {
    
        }
    
    }

But the big problem what I have is I have no idea about the math. May be is there an easy way.
For example I make a “virtual box” around my line and intersect this with the ray. Or make an Cylinder(no jme object, just for calculation) around it.

The “shortest distance between two lines” is going to be easier to calculate than the other options you mention. (And I’d be very surprised if the cylinder math was not already based on it.)

There is this web site called google where you can look stuff like that up. You may even find Java examples there.

You could also reduce it to a simpler problem and project your lines in 2D and calculate the distance from a 2D line to a point… which is mathematically pretty trivial.

Sorry I didn’t unterstand what you mean. I’ve googled about it.

Vector2f cursorPosition = inputManager.getCursorPosition();
        Vector3f origin = cam.getWorldCoordinates(cursorPosition, 0.0f);
        Vector3f direction = cam.getWorldCoordinates(cursorPosition, 0.3f);
        direction.subtractLocal(origin).normalizeLocal();

        Ray ray = new Ray(origin, direction);
        CollisionResults results = new CollisionResults();

        nodeItemData.collideWith(ray, results);

Thats my code to pick up the lines, which are inside nodeItemData node.

My problem is:
I have an array of points(start and end) which creates a ring. The user can pick this ring and can move it up and down around a human leg.

I may be wrong, but I guess it’s two things - ray casting tutorials and the fact that Ray is in com.jme3.math - that make false impression that now you don’t need any math to collide whatever you want with whatever else you want… and get just the result you expect. While the tutorials just do certain things with certain object types, and Ray is just a pure math object. You mention points and tolerance - but that’s not much of info, how points are represented, is it 2D or 3D space, how do you want to measure tolerance - pixels, percents of screen dimensions, meters, light years? There might be an easy way, but not before anybody else will understand the task…

Thanks for your reply.

An easy example. I have a com.jme3.scene.shape.Line Object which have a start and an end vector.
Vector3f start = new Vector3f(0,0,0);
Vector3f end = new Vector3f(1,1,1);

This line must be able to select by the user by mouse. The user must exact hit the pixel of the Line to select it, this is very difficult. Therefore I want, if the user hit 0,1 pixel next to the line then should be returned the line in the CollosionResults.

if( shortestDistanceFrom(line, ray) < 0.1 ) {
///picked
}

Then you just have to implement shortestDistance() from a google search.

Or consider that the approach you are taking to whatever you are doing is maybe beyond your skillset. There is nothing built into the engine to do this. It’s possible there is a simpler way to make your game that doesn’t require it.

Let me do one more step for you :slight_smile:
What is happening in engine (probably this is what is happening in any 3D engine, not just JME) is shown here:


Orange rectangle is your display’s screen. It is obviously 2D with some discrete resolution. Corner point coordinates reflect this. Red is your area of tolerance (if you want to set it in pixels - it is like 1x1, 2x2, 3x3 etc. Blue lines reflect 3D space that you see on your screen. First of all, it is not discrete (actually it is, but if you understand difference between float and int, you understand what I mean), secondly, it is 3D, and finally it is arbitrarily defined and oriented in 3D World space of JME. Therefore I didn’t depict any blue coordinates - they can have ANY values. Green is your line. Yes, it can have those coordinates you specified and be in a position shown on the picture - depending on the view (camera) orientation. Yellow lines (I know, my picture is not perfect, but it illustrates the very basics) reflect area in 3D space where you have to check for presence of your line to decide was it picked or not. From JME you’ll be probably interested in Camera.getWorldCoordinates(), what’s then… that would depend on those things I mentioned above. Speaking in general, I’d suggest to read about viewing frustums first, and then come to these https://jmonkeyengine.github.io/wiki/tutorials/scenegraph/assets/fallback/index.html
https://jmonkeyengine.github.io/wiki/tutorials/math/assets/fallback/index.html
You won’t be able to go much farther than example tutorials in JME without understanding its coordinate systems and their relations first, anyway.

1 Like

Thank you very much for your help and usefull hints. I have develop a Mesh which almost satified my needs.

package mygame;

import com.jme3.app.SimpleApplication;
import com.jme3.bounding.BoundingVolume;
import com.jme3.collision.Collidable;
import com.jme3.collision.CollisionResult;
import com.jme3.collision.CollisionResults;
import com.jme3.input.ChaseCamera;
import com.jme3.input.MouseInput;
import com.jme3.input.controls.AnalogListener;
import com.jme3.input.controls.MouseAxisTrigger;
import com.jme3.material.Material;
import com.jme3.material.RenderState;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Matrix4f;
import com.jme3.math.Ray;
import com.jme3.math.Vector2f;
import com.jme3.math.Vector3f;
import com.jme3.scene.Geometry;
import com.jme3.scene.Mesh;
import com.jme3.scene.Node;
import com.jme3.scene.VertexBuffer;
import java.util.Arrays;

/**
 * test
 *
 * @author normenhansen
 */
public class Main extends SimpleApplication {

    private ChaseCamera cc;

    public static class IndexLineArray extends Mesh {

        private Vector3f[] points = null;
        private float tolerance;

        public IndexLineArray(Vector3f[] points, float tolerance) {
            this.points = points;
            this.tolerance = tolerance;
            float[] position = new float[3 * points.length];
            int[] indexes = new int[(2 * points.length) - 2];

            int positionIndex = 0;
            for (Vector3f point : points) {
                position[positionIndex] = point.x;
                position[positionIndex + 1] = point.y;
                position[positionIndex + 2] = point.z;
                positionIndex += 3;
            }

            for (int i = 0, j = 0; i < (2 * points.length) - 2; i += 2, j++) {
                indexes[i] = j;
                indexes[i + 1] = j + 1;
            }
            System.out.println(Arrays.toString(indexes));
            setBuffer(VertexBuffer.Type.Position, 3, position);
            setBuffer(VertexBuffer.Type.Index, 2, indexes);
            setMode(Mode.Lines);
            updateBound();
        }

        // https://github.com/gavalian/clasrec-geometry/blob/master/src/org/jlab/geom/prim/Line3D.java
        public static float distanceSegments(Vector3f line1Start, Vector3f line1End, Vector3f line2Start, Vector3f line2End) {
            Vector3f u = line1End.subtract(line1Start);
            Vector3f v = line2End.subtract(line2Start);
            Vector3f w = line1Start.subtract(line2Start);
            float a = u.dot(u);   // always >= 0
            float b = u.dot(v);
            float c = v.dot(v);   // always >= 0
            float d = u.dot(w);
            float e = v.dot(w);
            float D = a * c - b * b;  // always >= 0
            float sc, sN, sD = D; // sc = sN / sD, default sD = D >= 0
            float tc, tN, tD = D; // tc = tN / tD, default tD = D >= 0

            // compute the line parameters of the two closest points
            final float SMALL_NUM = 0.00000001f;
            if (D < SMALL_NUM) {
                // the lines are almost parallel
                sN = 0; // force using point P0 on segment S1
                sD = 1; // to prevent possible division by 0 later
                tN = e;
                tD = c;
            } else {
                // the lines are not parallel
                // get the closest points on the infinite lines
                sN = (b * e - c * d);
                tN = (a * e - b * d);
                if (sN < 0) {
                    // sc < 0 => the s=0 edge is visible
                    sN = 0;
                    tN = e;
                    tD = c;
                } else if (sN > sD) {
                    // sc > 1  => the s=1 edge is visible
                    sN = sD;
                    tN = e + b;
                    tD = c;
                }
            }

            if (tN < 0) {
                // tc < 0 => the t=0 edge is visible
                tN = 0;
                // recompute sc for this edge
                if (-d < 0) {
                    sN = 0;
                } else if (-d > a) {
                    sN = sD;
                } else {
                    sN = -d;
                    sD = a;
                }
            } else if (tN > tD) {
                // tc > 1 => the t=1 edge is visible
                tN = tD;
                // recompute sc for this edge
                if ((-d + b) < 0) {
                    sN = 0;
                } else if ((-d + b) > a) {
                    sN = sD;
                } else {
                    sN = (-d + b);
                    sD = a;
                }
            }

            // finally do the division to get sc and tc
            sc = (Math.abs(sN) < SMALL_NUM ? 0 : sN / sD);
            tc = (Math.abs(tN) < SMALL_NUM ? 0 : tN / tD);

            Vector3f p = lerp(line1Start, line1End, sc);
            Vector3f q = lerp(line2Start, line2End, tc);

            return p.distance(q);
        }

        public static Vector3f lerp(Vector3f point1, Vector3f point2, float t) {
            return new Vector3f(
                    point1.x + (point2.x - point1.x) * t,
                    point1.y + (point2.y - point1.y) * t,
                    point1.z + (point2.z - point1.z) * t);
        }

        private float distance(Ray ray, Vector3f start, Vector3f end) {
            Vector3f rayEnd = ray.direction.mult(100).add(ray.origin);
            return distanceSegments(ray.origin, rayEnd, start, end);
        }
        static int c = 0;

        @Override
        public int collideWith(Collidable other, Matrix4f worldMatrix, BoundingVolume worldBound, CollisionResults results) {
            if (other instanceof Ray) {
                Ray ray = (Ray) other;
                for (int i = 0; i < points.length - 1; i++) {
                    Vector3f start = points[i].clone();
                    Vector3f end = points[i + 1].clone();
                    float dist = distance(ray, start, end);
                    System.out.println(c++ + ": Line start" + points[i] + " Line end" + points[i + 1] + " " + dist);
                    if (dist < tolerance) {
                        results.addCollision(new CollisionResult());
                        return 1;
                    }
                }
            }
            return 0;
        }
    }

    public static void main(String[] args) {
        Main app = new Main();
        app.start();
    }

    @Override
    public void simpleInitApp() {

        IndexLineArray lines = new IndexLineArray(new Vector3f[]{
            Vector3f.ZERO.add(0, 0, 0),
            Vector3f.ZERO.add(0, 1, 0),
            Vector3f.ZERO.add(1, 0, 0),
            Vector3f.ZERO.add(1, 1, 0),
            Vector3f.ZERO.add(2, 0, 0),
            Vector3f.ZERO.add(2, 1, 0),
            Vector3f.ZERO.add(3, 0, 0),
            Vector3f.ZERO.add(3, 1, 0),
            Vector3f.ZERO.add(4, 0, 0),
            Vector3f.ZERO.add(4, 1, 0),
            Vector3f.ZERO.add(5, 0, 0),
            Vector3f.ZERO.add(5, 1, 0),
            Vector3f.ZERO.add(6, 0, 0)
        }, 0.1f);
        lines.setLineWidth(5);
        Geometry geo = new Geometry("IndexLineArray", lines); // using Quad object
        Material mat = new Material(assetManager,
                "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setColor("Color", ColorRGBA.Blue);
        mat.getAdditionalRenderState().setFaceCullMode(RenderState.FaceCullMode.Off);
        geo.setMaterial(mat);
        s.attachChild(geo);
        rootNode.attachChild(s);
        flyCam.setEnabled(false);
        cc = new ChaseCamera(cam, s, inputManager);
        inputManager.addMapping("mouse", new MouseAxisTrigger(MouseInput.AXIS_X, true), 
                new MouseAxisTrigger(MouseInput.AXIS_X, false), 
                new MouseAxisTrigger(MouseInput.AXIS_Y, true), 
                new MouseAxisTrigger(MouseInput.AXIS_Y, false));
        inputManager.addListener(mouseClick, "mouse");
    }
    Node s = new Node();

    private Geometry pickElement(Node shootables) {
        inputManager.setCursorVisible(true);
        CollisionResults results = new CollisionResults();
        Vector2f click2d = inputManager.getCursorPosition();
        Vector3f click3d = cam.getWorldCoordinates(
                new Vector2f(click2d.x, click2d.y), 0f).clone();
        Vector3f dir = cam.getWorldCoordinates(
                new Vector2f(click2d.x, click2d.y), 0.3f).subtractLocal(click3d).normalizeLocal();
        Ray ray = new Ray(click3d, dir);
        System.out.println(ray);
        shootables.collideWith(ray, results);

        if (results.size() > 0) {
            return results.getClosestCollision().getGeometry();
        }

        return null;
    }
    AnalogListener mouseClick = new AnalogListener() {
        public void onAnalog(String name, float value, float tpf) {
            Geometry pickedElem = pickElement(s);
            if (pickedElem != null) {
                System.out.println("Elemet found: " + pickedElem.getName());
            }
        }
    };
}

This works fine. If I move over a line by the mouse pointer then my node is correctly selected.
But the big problem is: If I scale and rotate my model then it doesn’t work anymore. I know the reason but I have no solution for that. The error is I use in my collideWith method the orginal coordinates but after a transformation of my IndexLineArray, the coordinates has other values.
It is possible to transform coordinates by the current Rotation, Scale and Translation? Or has anyboday a better solution.

May be to much code. My problem is:

In my collideWith method of IndexLineArray(see previous post) I check the distance between the ray(which is created by Mouse pointer poistion) and the lines. This woks fine as long as I don’t change position or sacle.
But my IndexLineArray is moved and scaled.

Is there a possiblity to get the transformed coordinates of the IndexLineArray in my collideWith method?

I’ve implemented a method which calculates the distance between the to lines(line and ray) and it works fine. But when I transform my object then my algorithm doesn’t due to I used the original coordinates and the transformed.

So transform the points into Ray space or the Ray into point space.

I mean, if you look at literally any of the existing collision code you can see exactly how that’s done.

Thank you very much!!
That was the key indication. I get the parent and then I transform the start and end point to its world coordinates.

parent.getWorldTransform().transformVector(start, start);
parent.getWorldTransform().transformVector(end, end);