Highlighting and identifying a face on a large mesh via mouseover or click event

This is pretty much my last major technical hurdle (I think) until I can say I’m just fine tuning things for this project. I’m currently researching the optimal (in jME) way of highlighting and interacting with mesh “faces”. First I’d like to highlight individual triangles on a mesh via mouse interaction (both a mouseover effect and a more dramatic “selected” effect).

I’ve done this in another engine before by duplicating the mesh with an invisible copy placed slightly above, making the duplicate mesh’s triangles visible as needed. But I’m hoping there’s a better way, either by using my shaders or some other less performance hogging method than doubling the number of polys. I have another idea or two I’m still considering, but does anyone have any good suggestions?

It would also be nice if the solution helps me determine which face/vertices the mouse is interacting with (by an index #?)-- currently I’m counting on being able to utilize CollisionResult’s getTriangle() method to get a usable index from which I can modify the mesh data. The end goal will be to edit the mesh’s vertex buffers to change the mesh, something like a 3D modelling app, and of course I need to be able to write that data to disk in a specific format.


The cool thing is I have the same situations here which I’m developing plugins for the SDK, and they both are really big headaches, so I want to share it here and call for help too :

  1. I also want a specific mesh/triangle have dramatic “selected effect” (like in 3DSMax) :evil:

1.1 have color blend (blend with green or what ever color that we want)
1.2 and have border , the outline is highlight too
1.3 have the the triangle edges show
2. I want to shoot and check if the specific mesh is "selectable"???
I have thought about this prob a lot but then I end up use the "old trick" which you said in the post:
"use a duplicate mesh/triangle for the highlight"
Color blend is an easy part and you can do it in a simple shader.
For the border <- we can use an inline or outline glow shader for a the specific mesh.
  • Inline Shader:
    The shader make a green light at the border of the mesh. In fact an "inline" shader not really meet what we want cause it hide the detail of the mesh.
  • Outline Shader:
    We have to duplicate the mesh and then apply the outline shader to the "duplicated mesh" which in turn make this mesh little bigger in the normal direction. It looks not really good because the glow is thicker when is come to a mesh conner (it's a artifact of outline glow shader which I don't really know how to fix cause I'm a noob in shader)

Come to the hard one. For the "edge show" effect I don't really think is a good idea to make it in a shader (since I'm a shader noob :[ ). Because the fragment shader have to know the current pixel is in the edge of the triangle or not, I dont' think is possible! <- Please show me how If you can :p
So the current sollution for me :
  • Entire Mesh Edge show : duplicate the whole mesh, make it white or what ever color and make is "wired". Then sync animation for the two mesh in a Control, This solution come with ease but less flexible and heavy , and the hard step is to make the mesh sync. But with it, i can easily make the orginal mesh transparent when ever I want, or can add the vertex highlight (as you can see in the image).

  • Specific Triangle Edge show : duplicate only the triangle and the bones which impact it !!!! Then make it "wired" and sync the animation. The hard steps is to determine which bone is impact (or you have a big waste) and sync the animation! This solution make me crazy, because I need it, but I seem very slow and heavy when I want to simple switch to next triangle ( :x guhhhh !!!)

P/s: it's a same solution which use in the official SceneComposer plugin to highlight the selected object but without the "color blend part"
2. I want to shoot and check if the specific mesh is "selectable"???
This problem seem very easy in the "Engine Level" but since I really want a "very complete Editor" then It come quite hard:
The theory is simple :
- you load a "Shootable Node"
- shoot them
- and then check the CollisionResult
The problem is who can be the "shootable Node". RootNode? But there is some helper of the editor which don't belong to the real world (the gizmo, the grid...). Let say we can move the helper to a different "helperNode"... Then shoot the "realWorldNode" .
How can we tell it when we just click in the helper again???
So I have to design very ugly and extremely complex packages for just the "name and shoot problem", really :p :
Tell you some of my design for the plugin:
-Engine level :JME
-Editor level : Netbean Platform and JME SDK API
-My Editor levell : More detail API of the common problem customize for my own need : easy threading manage and this "selection problem" and other things.... plug seamlessly into The Editor Level.
-My Tool level : The API for the tool such as Property Sync among every tool at the same time, Action , Drag and Drop , Visual Library(Model/Texture chooser), Animation/Timeline Manage ....... utilize the powerful Netbean Platform.
- In Game level :
+ runAtOnce feature
+ FPS _Prototype with AI _PathFinding _NavMesh (for testing)
+ EntitySystem for a very complex world.
+ Database connection for storing things + Editor_NetPlay feature (the main goal of my project)
**************Take a look deeper:
spatial package :
-SpatialInfo : wrapper for Spatial in the "Engine Level". Basic and Smallest Unit.
-SpatialPresentor : manage the display of a Spatial/SpatialInfo in the scene.
shape package : Util for creating JME Shapes in various way(such as Box by 2 Rectangle which have 90 angles or normal way...)
-Shape : extends Node
-BoxShape : have Builder pattern for the customizable parameter.
manager package :
-SelectionManager : manage the selection
-HelperManager : manage the helper.
helper package : The 3d object which control others.
-AbstractHelper : the AbstractHelper extends a Control in EngineLevel, it have a 3d presentor of course.
-GizmoHelper : the Gizmo for the pos/rot/scale job.
-GridHelper : the Grid for the align and snap job.
... and many more
In my editor level, I call the smallest unit object is an "SpatialInfo" which basically have size and position, and they are the wrapper for Spatial in the "Engine Level".
One "SpatialInfo" have a 1-1 relation with a "SpatialPresentor" who can change the display of the spatial (Node as a Box, Geometry as Box, CollsionMesh, Tick or Billboard)
<- The complexes come here, we can change the display of a Geometry, so when we shoot something, how can we know we shoot which Geometry belongs to which Presentor/SpatialInfo.
So my current solution is :
  • + Shoot and find which Geometry is nearest One in the CollisionResult.

  • + Make a Map of which Geometry belong to which Presentor for a quick reference. It's a waste and hard to manage but It's better than searches which happen almost everytime.

  • + The Helper is contain in the ShootList and then choose to select in the Map too.

More complex come when I want to select separated any Invidual, Group, List, Layer of SpatialInfo(s) ????? Let's me explain:
  • Invidual : select one SpatialInfo through its presentor in the scene.

  • Group : They are grouped together , so click one , select all.

  • List : select two or more SpatialInfo at the same time.

  • Layer : All SpatialInfo in the scene is slice into Layer which are not Pos or Parent depend, that is you can select/move or hide them even they're in the different ParentNode.

This is the samething you see in 3DSMax...

So It comes more and more complex ........ but this post is looooong as hell so I will stop now and anyone who find out this design is damn stupid, please tell me, I'm appreciate that!!!!!!!!
Thank for reading !!!
atomix ;)