What DB for voxel game?

General Discussion
1

Currently I’m looking for a way to store my randomly generated level data in some DB. Limitations are:

  • Big, preferably multi-dimensional data storage (over 10.000.000 blocks, several parameters for each block),
  • Embedded, preferably native Java,
  • Works in Android as well,
  • Fast (so I can load big “chunks” of terrain quite often).

So far I found:

  • OrientDB (awesome in every possible way, except for Android no-no) (I know about Wuman, but this project is as dead as my grand-grandmother),
  • MapDB (seem to be key->single value oriented, so BIG_X times BIG_Y is a no-no),
  • JavaDB (previously known as Derby - again, everything is fine, except for Android part).

I kinda run out of ideas. How do people store game data in Android strategy games anyway?

2

I’m not sure if it’s a good idea to store multiple parameters with each block. That would cause a lot of overhead while storing the data. if you have 10 Million blocks, I assume that the parameters of most of the block which are equal have the same parameter, except they have been modified by the user.

Multidimensional? :slight_smile: You’re always free to define a “multidimensional data model” in any database. That’s up to you and not the job of the database.

What’s the use of “big” chunks? Keep your chunks not to big. I had chunks of a size of 15x256x15 and ran into problems when I wanted to transfer them via Spidermonkey from server to client. The size exceeded 32KB (which is a spidermonkey limit) and that even if I used only 1 single byte for each block (no extra parameters as you plan).

But taking this aside: I do not believe that da database is a good deal. I plan to store my data on the filesystem in a directory structure that represents the dimensions of the world. Why would you use a database? A db is good for complex data models, and the chunks of a world are not complex in any way. The total oposite is the case: you should keep this structure as easy as possible.

3

Firebird?

Databases are in general very fast. But there is a overhead in getting the data to JAVA. So it will never be real time. But I guess the same goes for other solutions as well.

4

How much data do you need to store for each voxel?

10000000 items isn’t that much if you can fit everything into a single dimension integer array and just use some bit manipulation. Then you could just do a standard write to a file to store it.

Databases are slow, depending on what you will use them for.

5

Why specifically use a database engine, is it just for the convenience of a DSL for query?

Why not just store them directly to a file somewhere and then load them, you could split it so each chunk is in a separate file if need be. How much data are you looking at per chunk/block? How do you intend to query the database for these blocks/chunks?

I can suggest a whole load of databases but without knowing how you are going to query on them it’s all the same, pretty much useless advice :smiley:

6

I’m planning to use NoSQL. Here is a good starting point.

7

[quote=“Andreas_Heidt_Altwei, post:2, topic:33520, full:true”]

I’m not sure if it’s a good idea to store multiple parameters with each block. That would cause a lot of overhead while storing the data. if you have 10 Million blocks, I assume that the parameters of most of the block which are equal have the same parameter, except they have been modified by the user.[/QUOTE]

You assume wrong. Strategy game. For example every block that resource (iron, silver) will be mined instead of destroyed, thus will have a random quantity of that resource.

[quote=“Andreas_Heidt_Altwei, post:2, topic:33520, full:true”]
Multidimensional? :slight_smile: You’re always free to define a “multidimensional data model” in any database. That’s up to you and not the job of the database.[/QUOTE]

Normally I would agree with You but DB’s are the part od programming world that I hate with true hatred, not to mention they bore me to death. So there is no way in this lifetime I’m going to do DB custom works. Ready to use out-of-the-box solution is the only option for me.

[quote=“Andreas_Heidt_Altwei, post:2, topic:33520, full:true”]

What’s the use of “big” chunks? Keep your chunks not to big. I had chunks of a size of 15x256x15 and ran into problems when I wanted to transfer them via Spidermonkey from server to client. The size exceeded 32KB (which is a spidermonkey limit) and that even if I used only 1 single byte for each block (no extra parameters as you plan).[/QUOTE]

No server, no client. 100% single player game. As for “bug chunks” what I meant was something around 100*100 grid of blocks (but mention every blocks having several parameters).

Again, strategy game. Events that change my “world” will happen as often out of the screen as they do on screen. So I need a way to query for specific blocks very fast and everywhere in my “world”. Not to mention I believe that having (for example) filesystem of several dozen CSV files, each with very small chunk stored, would be too optimal for Android device.

8

Both MapDB and OrientDB are NoSQL databases. Could You be a little more specific? Again, I’m looking for ready to use tool, that will work out of the box.

[EDIT] Didn’t see the link to Couchbase on my phone. Gonna check it out later and get back to You.

9

Okay. But even in this case you would have the chance to calculate the amount of the ressource in a block. I can see no reason to save this data, until it is really modified by the player. E.g. you set a random amount of silver for block X. If you use the same seed for your random number generator it should always produce the same amount of silver if you always use the same element of the random sequence. In that case you would only have to save the seed for the random generator. Only and really ONLY if the user has changed the amount of the ressource in the block you would have to save a delta value.

:slight_smile: We have something in common. I really hate that db stuff, too, especially while I have to do this boring stuff at ma job. But do not misunderstand me. I don’t mean that you have to customize th database. But a database is a database. The datamodel has to be written by YOU. And I do not believe that there is a solution with a ready defined datamodel that fits your need.

Just therefore I would prefer not using a database. Or if you use on, you really have to convert your object oriented data model in some kind of SQL model (or whatever db you’re using - even if you use an object oriented db, you would have to make modifications and grapple with the database)

Hmm … I could argue here, but that would make so sense, doesn’t it? :wink: To be honest: I do not have enough information about your design that I could argue without presuming things that might b true (or even not). And: for Android I’m really out. I do not have enough knowlodge about this, but again I’d consider that 10 Million blocks with each block having several parameters will need to have at least(!) 10 MB if you only have 1 paramter per block that is of the type byte. How much parameters are you going to hold? Have you already pre-calculated the final size of you’re data structure and if this is suitable for your games target device?

10

Random seed sounds fine (I actually love the idea), thanks. But still even if I just link my db info to random seed instead of storing actual numbers, resources will be all revealed sooner or later (AI will mine too, so most likely sooner).

In the beginning I was aiming at object oriented database in which case all I have to do is prepare my object classes (schema-like-thingy that is the actual datamodel in object DB’s) which would all be pure java. That’s why I love OrientDB so much. And I’ll probably stick with those in the end, but still it kills my aim at Android Devices. That’s why I look for alternatives.

Btw. as far as documentation goes OrientDB (and NoSQL fast databases in general) are able to withstand thousands of queries at the same time, so there shouldn’t be much problem with performance.

11

I didnt use databases for games yet, but when I start to use, I will just write my own database.
It may sounds crazy, but you guys would be surprised on how easy is to build and simple dedicated database engine inside your game, like save the data serialized in the file and load registers by indexes, its very very fast, and reliable.