The brute force way of doing ray casting is to check every triangle in every mesh to find intersections, this can be costly since we need to traverse all mesh buffers...
Therefor the scenegraph is also maintaining a bounding volume hierachy. Every spatial have a bounding box, both Geometry and Nodes. The bounding box for Geometry is the smallest axis aligned box that can fit all vertices inside it, (that's why you have to call updateBounds() on a custom mesh everytime you change the mesh buffers..).
A nodes bounding box is the smallest axis alligned box that can encapsulate all childrens bounding boxes.
This gives you a hiearchry of bounding boxes. Ray casting works on a node by first checking if the given ray intersects the nodes bounding box (this is a cheap test and we can return early if it fails, no need to check all meshes/triangles in it's children). If the ray intersects the bounding box then do the same for the children until you have found out which mesh buffers to traverse.
If one use physics one can also let the bullet physics do ray casting inside the physics engine, how this is done is vodoo and black magic and can be found out by tracing the code... A thing to note is that the meshes/collison shapes used by bullet have nothing to do with the scenegraph, they live in their own space (physicsSpace) For instance we use a RigidBodyControl to connect a collisionShape instance in the bullet physicsSpace to a Spatial in the jME scenegraph.
# The jME scenegraph bounding volume hierarchy is also used in the frustum culling step, since it's cheap to find out if a bounding box is inside the frustom or not.
Phew, I hope my explanation makes some kind of sense. If I'm misstaken by something, please correct me :)
Edit: added some more...
And by dividing up your scene graph and ray casting into a specific node/specific encapsulated nodes, you limit the cost even further. So... this isn't really a very expensive or bad option... especially considering the example I put together isn't remotely close to what I would like the final to actually be like. In the example I am covering the entire area instead of using "clumps" of impostors to accent existing terrain textures. The final should not be placing that many impostors (relative to what it is doing now, that is).
Does this sound correct? Or am I way off base?