@Ali_RS, thanks, I was not aware that Bullet had its own ray casting functionality. Unfortunately I won't be able to rely solely on Bullet's ray casting as objects that need to be ray-visible won't always be in the physics space (even as static objects).
@pspeed, thank you very much for the information. As far as (a) goes, you're right, I don't need the scene graph to do physics.
I'm using jME's Bullet build - I'm just extracting the native libs and then setting up individual PhysicsSpace instances the same way BulletAppState does (just integrated with my server's ecosystem more tidily than cramming in an appstate would be - the server is not a jME application and runs its own heartbeat loop). You're absolutely right about being at the mercy of Bullet's broadphase, and I will freely admit that I failed to consider that (or considered it a long time ago and forgot about it ). However, physics computation is run in parallel to the heartbeat loop and other systems using a mechanism almost identical to the one used by BulletAppState while running in parallel mode. I do have a little bit more leeway than if I had to divide one tick worth of time between everything in the heartbeat loop (most systems are highly independent and can/do carry out their heavy lifting in a thread pool).
My biggest concern about setting aside the scenegraph is handling parenting transforms and ray casts myself. Entities in the world need to be parentable in a manner identical to jME scenegraph nodes. (Note: for ray casts, I can't depend on Bullet's ray casting for everything since some objects may need to be "ghosts" - non-physical but still detectable by rays). Computing transformation matrices to account for parenting isn't that big a deal, but by the time I've implemented parent->child transforms, ray casting through a custom structure, and optional physics syncing it seems like I would have just reinvented the scenegraph wheel. If I have to do that, I'll do it, but if there's a reasonable way to avoid that and get the result I need I'd prefer to avoid duplicating that much functionality on my own.
You're absolutely right - it's messy. Right now I'm trying to alleviate that by putting 100% of octree logic in a control and having that control manage nodes itself. Right now the control actually creates the node that it manages. That's kind of backwards, but I intended to make note that by convention these nodes are strictly not to be touched in any way except to attach the octree root to the rest of the scenegraph (although conventions like that are fragile and easier broken than kept).
My point in explaining all this is not to tell you that I'm right and I'm doing it my way - my point is to (hopefully) make it a little clearer why I'm setting out to do this in the first place since my first post wasn't super detailed. I'm hoping that there's some better way to do exactly what I want and that you'll talk me out of making an octree out of scenegraph nodes because it is so messy and error-prone.
Thank you for linking the SimEthereal sources - I took a good look at the zone functionality. I must say... wow. In a million years I wouldn't have thought of using bit twiddling to place objects into the correct zones. The BoundingBox/BoundingSphere code that I just wrote for determining the right octree level for an object is fairly efficient (no temporary objects, just addition/subtraction and logical operations) but I'm going to venture a guess that your bit manipulation would beat the socks off of it by a factor of 20x or more in a speed test, and that's not counting the cost to move objects to the right node. I love how clean and efficient the sparse grid is. Seems like several levels of sparse grid zones could provide all the features of an octree at a small fraction of the complexity and runtime cost.
I'm especially intrigued by SimEthereal since I already wanted to use it for syncing physics-enabled objects. I have my own entity system (it's kind of an ECS turned inside out - entities have agents that are unique to them and control them rather than the world having systems that processes all entities having certain components). Obviously if I can sync physics simulation and entity system updates directly with SimEthereal it would be a lot less overhead than tossing a scenegraph into the mix too. I'm going to take a good hard look at the SimEth-ES examples and see if I can come up with something better than an octree.
Do you mind elaborating on that? I haven't been able to turn up any relevant information via either Google or my bookshelf, and I was not aware of any major shortfallings of octrees for neighbor checks.