Well and there is you explanation of why serializeable should be implemented yourself for own object, as it will probably be much more efficiently.
Also this show what use procedual galaxy generation has.
Can you describe a little more about your data structure? How are you breaking it into separate files? A file per leaf?
@EmpirePhoenix
Please, I don’t need the condescension.
Look, guys, I’m not an idiot whatever some people might think. This is my first try and I’ll make things work. The structure of some of the classes were not done with serialization in mind. Let me work on this and I’ll repost progress.
Well if you don’t want to hear what point I try to get trough to you, I will from now refrain from answering to your posts.
But they probably never were designed that way, as you can’t have a good solution hat works for any kind of game.
Take it easy guys. @EmpirePhoenix, it was possible to read your post as being condescending a bit. I read it that way the first time. Sort of an “I told you so.”
@madjack, it is also possible to read it with less condescension… I read it that way the second time. And certainly life is nicer when every post is read using the nicest filter possible.
I only offered to help, not because you are incapable of doing this yourself but because I have a lot of experience with Java serialization and it is possible to customize the writing of an object without abandoning Java serialization completely… which is at least one way to read EmpirePhoenix’s suggestion and it would be a shame to throw it out completely.
You didn’t bring any point. You didn’t even suggest anything. All programmer, me included, knows it’s usually better to customize things. Your post was moot. I don’t want to berate or demean you but your post hit a nerve.
As far as procedurally generating the galaxy, if, by that, you mean generating only a leaf or chunk of it as the game start, this is not something I’m willing to go through. Galaxy generation is very complex and I don’t see how I could start a galaxy from an arbitrary point. This isn’t something as “simple” as a “box” that expands as you move around, a galaxy has a structure, spiral arms, concentric sphere for elliptical, a barred center plus arms for barred galaxies. Finding algorithms to come up with that would take months. Even if I were to do it in weeks, I’m not willing to walk that road.
I will simplify what I’m working with right now and come up with a better, lighter solution. If need be, I’ll customize the saving. Learn to walk before running and jumping and that’s what I’m doing.
pspeed said:
Take it easy guys. @EmpirePhoenix, it was possible to read your post as being condescending a bit. I read it that way the first time. Sort of an "I told you so."
That's exactly the way I read it. *shrug*
I only offered to help, not because you are incapable of doing this yourself but because I have a lot of experience with Java serialization and it is possible to customize the writing of an object without abandoning Java serialization completely... which is at least one way to read EmpirePhoenix's suggestion and it would be a shame to throw it out completely.
I know Paul. You've helped me before and it's extremely appreciated, but I have my own way of learning. :) I will go through the motions myself and if I need to I'll ask (read beg) for some help, hints, pointers. ;)
Yep, I’m the same way. I prefer to know exactly what I don’t know before I ask. 
Much better now! 
It takes about 1.7 seconds to save the entire galaxy’s leafs with files ranging from 2-3 KB to a bit more than 100 KB. For a grand total of 10 MB. This is for 335 leafs.
For those wondering if we’re still on topic, yes we are. But we’ll see if the changes will ease things up and make the game more stable. I gather FPS might also improve as there is less churning in memory, but that remains to be seen.
Ok, now, next on the menu. Since I can save the data to disk I can get rid of the “whole galaxy” data and only put what’s on the screen in memory. ie: loading each leaf. But I do have to keep a reference to the list of leafs composing the galaxy.
Right. Let’s get on this.
Sorry if this sounds like a dev diary. The goal isn’t to annoy anyone.
Well, it’s a success. Not huge, but a success. 
The generated galaxy saves itself after being generated and it also can be loaded from scratch.
Because of the many changes that implies, several modifications had to be implemented. Mainly I had to make a “LeafManager” that: saves the left-from leaf and loads the next leaf as the player flies around.
That’s good although it’s not saving the leaf that has been left yet.
Now on the menu, two things. First. A “modified” field that will be used to compare against for any noteworthy (or save-worthy rather) changes to a leaf. No need to save a leaf if its state hasn’t changed. Second, solar system generation inside the leaf being loaded, if not already done. Technically, the leaf the player finds himself in at start should have the solar systems done.
Now, what I’m thinking about is that while the “Game Thread” has nothing to do (as per the commands enum), I’m thinking I might have it work on generating solar systems on unloaded leafs on disk. I don’t think disk read/write will cause problems like hitching and stuff. I might also bring the thread’s priority down when doing so (if possible) so it’s not interfering with the main action going on.
I think a background thread should be ok… extra GC will be your biggest worry, I think… since that effectively pauses everything. But that can be managed.
Regarding your “modified” field, from experience you might consider a “version” field instead… depending on how granular your changes are and if you have the space to store that over a boolean. The reason I opt to do it is because it allows saving to be done more independently of modifications. ie: as long as there is a snapshot of what you are saving then you only need to keep track of the version saved meanwhile changes can still be happening. Though that may imply a level of threading that you aren’t attempting at this point.
why do you save data to disk instead of node.hide() ?
seems like a blonde moment to me.
It’ll actually be far more simpler than that. If any interaction that might change the state of the game has happened inside that leaf, that field will be set to true and thus trigger a save when player leaves. Nothing complicated.
Ex:
Solar system generation will trigger a save.
NPC “conquered” a star system: save.
Star gone supernova, save. (If you’re lucky it might happen while you’re in there/looking!)
I might use a “private” event listener to do that. I’m not sure at this point. As it is, saving always happen when you leave a leaf until I have fixed all outstanding issues (not many, but they’ve got to be cleaned out before going further).
tralala said:
why do you save data to disk instead of node.hide() ?
seems like a blonde moment to me.
Because I have too much data (More or less 1 GB of RAM) and at one point or another, this has to happen. Consider that: 75 000 stars with planets and each planet having one or several moons. And that's just for starters. Now imagine that with a galaxy that has 500 000 stars. Now with 2 billion stars. Yeah. Not nice.
I know, I know. I could make the world smaller, etc. Refer to my previous answer to those points. :)
aaa i see, your world is super duper omega huge.
madjack said:
It'll actually be far more simpler than that. If any interaction that might change the state of the game has happened inside that leaf, that field will be set to true and thus trigger a save when player leaves. Nothing complicated.
Ex:
Solar system generation will trigger a save.
NPC "conquered" a star system: save.
Star gone supernova, save. (If you're lucky it might happen while you're in there/looking!)
I might use a "private" event listener to do that. I'm not sure at this point. As it is, saving always happen when you leave a leaf until I have fixed all outstanding issues (not many, but they've got to be cleaned out before going further).
The issue I usually run into when setting a flag is that something has to reset that flag. If it's a different thread than the one setting changed to "true" then there is an easy race condition... though synchronization can fix it. Whereas a volatile version number can make things a little more independent.
Seems like you have things under control, though.
pspeed said:
Seems like you have things under control, though.
The problem is that I often have things under control mentally. In practice, it's not always so. Often times it's because I'm making myself jump through hoops. But I'm working on that. ;)
Well, it seems I’ve done it. Yay me!
I can now start a brand new galaxy, do stuff, fly around switching leaf and things get saved when some changes happen. After saving you can load that game and all works. I do have one really weird and rare bug but I this point I think it might be the engine. I’ll wait before digging deeper.
What’s left to do is thread the loading/saving/solar system generating (except the leaf we load at first world generation as we want that one ASAP) and finally the pre-generation of solar systems outside of the current leaf.
I’ve had some solid and long thoughts about save games though. At first I wanted a dynamically-saving world (à la Mythruna) but after much consideration I’ve decided to opt for the “real save game” as we see in other games. Because of the size (or potential size) of the directory structure, instead of recreating the whole thing when saving, I decided to save each leaf as a .temp file. When the user force a save, I simply replace the old with the .temp. Of course, as we fly around and switch leafs, the game looks for .temp file before loading the default one. So, that way, if a user decides to reload the game I simply remove the .temp file. The downside is that only 1 previous saved game is only supported. I might decide to support multiple saved, but for now I’m not inclined to do so.
After some quick and dirty tests I did see some FPS improvements. Not by much, but some.
Since solar system aren’t pre-generated switching leaf takes its toll when the new leaf has many stars. Right now the maximum stars per leaf is set to 1200. I might have to bring that down for slower computers. Ok, I -will- have to. But I’ll need some testers for that. But I’m not there yet.
I haven’t profiled the game yet, but I imagine thing improved there too although I already have seen the nifty’s up/down thing on the FPS.
Congrats on the persistent universe. Everything tastes a little sweeter after that.
Greetings,
just a little question to you pspeed: how do you access each leaf/cel?
You say, world → cell by floor().
So, do you use something like class Node { Leaf[][][] leafs = [32][32][5]; }?
Because if I do it that way (just trying, i need a similar system of infinite maps for my project), I dont know how to access or specify the leaf I am in, if i move into negative coords… for example if I start at 0/0/0 the leaf ID
is 0/0/0, moving to 32/0/0 results in the leaf 1/0/0, but if I’d move into -32/0/0, i would need to access at -1/0/0 … so how did you solve that, or am I just blind to something? : )