i'll keep it short:
i generate a dungeon. the structure (in memory) takes up a few mb. when converting the dungeon to a node, it takes up a few gb. too much. reason: it's a grid. i have a wall model, a ground model and a ceiling model (right now simple quads). for every single wall i put in to my dungeon, i do this:
get spatial from cache -> wrap in sharednode -> wrap with rotated node (there are 4 directions for walls) -> wrap with translated node.
this is way too much. how can i reduce this? i just want to draw a few models a lot of times at different places, but this nodechain needs a lot of memory.
Why do you even want to have the entire dungeon in memory all the time? Try dynamically generating your dungeon parts from your few mb datastructure.
Or maybe re-use walls, floors and ceilings in visible areas from a pool. That way you even save a lot of object creations.
Why do you even want to have the entire dungeon in memory all the time?
because i insist to. it's about 10 times easiler than writing a dungeon part memory manager to deal with the weakness of the scenegraph principle. also, this wouldn't solve the problem if there a few other players/bots in the dungeon.
what would solve the problem is if i could create one node for the wall and just render it at different positions without creating a new instance for every translation. can i do this, or will this confuse the inner parts of jme? mh...yes it will - collisions won't work anymore ;(
Or maybe re-use walls, floors and ceilings in visible areas from a pool.
what do you mean? i'm already using sharednodes
is it possible to share sharednodes? if yes, i could precreate walls of different sizes and reuse them - this way, i could use a few 20-size-walls (containing 20 1 size walls) instead of 60 1-size-walls. that should help a lot.
I think of it a little like scalextric track. You have a banked bend, a 45 degree bend and a wide bend, out of these three bends and a few different straights you can build up any track.
my dungeon parts are much smaller - i need to many of them. but it might still be the best choice.
HamsterOfDeath, with every edit your reply #2 sounds more and more like you don't have any separation between your game logic and presentation layer. I don't think that is a wise design decision for a dungeon of the size you are planning to do.
If your collision detection really depends on the whole dungeon's geometry being in memory all the time, that means the collision tree alone is going to be absolutely huge!
You seem to already have a very memory efficient datastructure for your generated dungeon, make use of that. From your other posts across the forum I think your pathfinding already uses that datastructure, so I don't really see how partial loading will affect faraway AI at all. In fact, I'd try to design the AI logic in such a way that the AI never even needs to know anything about the display geometry, because it gets all it needs out of the dungeon layout datastructure.
I'd strongly suggest to pull the emergency brake in your design process now, and think about how you can separate your data/logic/presentation layers, just like you would do in any other well designed application. After that, all other problems will become much more solvable.
Partial loading / unloading is just an extension of the LLama Terra
If your collision detection really depends on the whole dungeon's geometry being in memory all the time, that means the collision tree alone is going to be absolutely huge!
i'll be using either bounding boxes (walls, gorund) or low poly models (stuff lying around) for collision handling and high poly models for rendering the dungeon.
From your other posts across the forum I think your pathfinding already uses that datastructure
no, the images just look alike ;)
the a*-representation for the dungeon is a bit smarter and a lot smaller. it's zone based (the dungeon is divided into rectangular zones), not pixel based as the a* graph for the landscape.
so I don't really see how partial loading will affect faraway AI at all
not using the dungeon as a node means i have to simulate the movement of the ai bots instead of using already existing game logic. i'd have to implement all game logic twice. the might be 2 or more factions fighting each other, even when you're far away - i don't want to write a simulation for that.
I'd try to design the AI logic in such a way that the AI never even needs to know anything about the display geometry
the ai itself doesn't, but i need the geometry to handle collisions. if the ai decides to walk from a to b and crosses a bridge (its a* graph knows it's traversable), i need the geometry to be loaded for collision checks so the bot doesn't fall into the river/lava/trap and dies.
everything that happens anywhere in the game world is as real as if you were there watching it.
you don't have any separation between your game logic and presentation layer
i don't see any reason to not to use jme's scenegraph as my logical model. i'd have to reimplement all the math stuff jme provides if i did it myself (how to determine if a railgun shot would hit if not using jmes ray intersection? how to check if bot a can see bot b if not using a ray check? how to handle triangle accurate collisions if not using jmes collisiontree?) and would spend way too much time to find bugs that would cause my and jme's model to get out of sync.
i didn't have these kind of problems at all by not separating layers.
I'd strongly suggest to pull the emergency brake in your design process now,
too late. except for the dungeon, i'm finished with all jme dependent game logic, had no problems, everything works. sometimes, a (theoretically) bad design works better than a (theoretically) good one. i made that experience more than once.
ok you won. i'll dynamically create/remove parts of the dungeon that are visible/invisible based on the camera position and use my low memory logical dungeon for collision handling. (looks funny from upside)
Yay I win 
I hope it works out for you! I'd really like to play hhexen eventually, your game design sounds really interesting.