How to : make a space game?

Hi everybody,

I already made some posts about this subject, but it was in my particular case, and now I want to start things over to do them correctly. Furthermore, it could also help the (numerous, we all know it) people willing to do this kind of game, by giving them the right things to start with.

What I’m wanting to do is an open space opera game. Meaning 6 points, that I ask you to answer providing their number first (so I can edit this post by combining all the answers).

1/ Space dimensions. It should be the biggest possible to contain a huge number of galaxies.
Maybe using chunks to store them?

2/ Showing far galaxies. No engine can render such a huge number of galaxies, especially since we want them to contain a huge number of solar system (with a sun and planets orbiting it).
Would a skybox be the solution? Many galaxies will have to be loaded the lightest possible to draw them relatively to their position and distance.

3/ Showing stars and planets. It’s the same problem : how to handle that many elements (rendering), and apply rotations and translations? Using a skybox when in a galaxy?

4/ User navigation through the Universe. How to efficiently bring galaxies in front of the user? Should he stay at {0, 0} while everything is being moved toward him and resized?

5/ Ships navigation. Making ships moving in the loaded space. We should also manage the ships going out of loaded chunks (if used) by making them move and perform their goal(s).

6/ Somehow optional. Many people want to create their own, auto-generated nebulaes and clouds textures, or 3D model. Programs or ways to achieve it?

I thank you in advance for all the answers you will provide me!

1 yes something like that can work
2. imposters
3 same
4. yes due to precision issues
5. game spcific
6 maybee port space scape to java

1 Like

Thank you for your answer, but I was asking for much longer and detailed solutions.
Could you maybe detail yours?

We don’t know what you already know, so writing more detail risks wasting your time (sifting through known material) and our time (writing stuff you already know).
Just ask what’s still unclear to you.

Try doing searches on the specific questions as well, you will find a lot of information online (and in these forums) already.

1/ Space dimensions. It should be the biggest possible to contain a huge number of galaxies.

I’ve tried using chunks, but then I don’t see how to do for 4/, and for 5/. Furthermore, should I really be using chunks, or rather octrees?

2/ Showing far galaxies. No engine can render such a huge number of galaxies, especially since we want them to contain a huge number of solar system (with a sun and planets orbiting it).

I don’t know about shaders to perform a printing on a skybox (imposters?) and how to efficiently obtain every stars from the chunks that should be displayed. Is there another way?

3/ Showing stars and planets. It’s the same problem : how to handle that many elements (rendering), and apply rotations and translations? Using a skybox when in a galaxy?

It’s the same as 2/ then. Or maybe not?

4/ User navigation through the Universe. How to efficiently bring galaxies in front of the user? Should he stay at {0, 0} while everything is being moved toward him and resized?

Relative to the first three points, what would then be the solution to this one depending on the others?

5/ Ships navigation. Making ships moving in the loaded space. We should also manage the ships going out of loaded chunks (if used) by making them move and perform their goal(s).

The problem here is more of a pure implementation problem solved for this particular case than a graphical problem. I have no idea how to perform this.

6/ Somehow optional. Many people want to create their own, auto-generated nebulaes and clouds textures, or 3D model. Programs or ways to achieve it?

Could it be possible with shaders? I looked at the program mentionned, but no dl link is working.

P.S. : Answers could be for anyone, so you do not need to deal precisely with my knowledge.

The notion of having many galaxies shown is erroneous. When you look up in the night sky, what do you see? Not what you KNOW there is. You see points of light and the occasional blurry and hard to distinguish blob that are the closest and biggest galaxies. That’s even though most of what you see are galaxies. (unless you look towards Sagittarius, the galactic center or the galactic equator, where it’s mostly stars).

You should do the same. Until you get “close enough” you just don’t render those galaxies. What you use is a “generic” night time skybox. All objects that are getting near you render them either on a quad showing a low LOD of a galaxy… Etc.

In short, you should keep a list of the closest and biggest objects to the player and render those only. The rest is too tiny or whatever.

@madjack I don’t think it’s about storing a gazillion galaxies, it’s about being able to store coordinates for those celestial bodies that are actually materialized.

@lightmax

  1. JME and JME’s native Bullet interface deal only with floats. If you want a larger coordinate space, you can:
    1a) Build Bullet with doubles instead of floats, increasing the coordinate ranges from 24 to 52 digits. You’re mostly on your own, because building Bullet requires dealing with cmake which is a bitch unless you already know it; also, it won’t take you beyond a light year even if you’re content with a precision of meters (and you want millimeters).
    1b) Use longs (63 bits). The maximum distance you can express with that is a light year, or 1000 light years at meter precision. If you’re willing to give up on realism (something that you’ll do anyway), you can reduce many of the universe’s scales to it fits… but it’s a bit tight.
    The problem with this is that you can’t leverage Bullet’s raycasting. Or any other library that does geometric calculations for you - not in Java anyway (in C++, some libraries can use any numeric type via templates, but Java doesn’t allow primitive types as generic parameters so Java libraries are typically built for a specific coordinate range). In other words, you’ll be doing several years of library development work before you can even begin to write your game.
    1c) Use two longs. That’s finally enough (127 bits, as opposed to the whole universe in 99 bits at millimeter precision).
    This has the same problem as longs, plus you need to implement two-longs arithmetic yourself (but we’re talking about massive library writing already), but at least this does eliminate the cramped coordinate space.

There’s another approach: Use a multi-tier approach.
In the universe, a galaxy is just a dot.
In a galaxy, the star clusters are just dots.
In a star cluster, the solar systems are just dots.
In a solar system, the planets are just dots.
On a planet, the towns are still just dots (at millimeter precision, no scene can be larger than a two-digit kilometer size with floats).
In a town, you finally have floats with a scene.
You will need all kinds of special-case code. E.g. you can’t sensibly calculate whether a light beam shot at a distance of half a light year will hit a planet-sized region of space, much less a space ship in it. (On the other hand, no player will want to wait half a year after shooting - they’ll warp there and fire a torpedo. Still, the problem persists e.g. for ballistic missiles shot from one town to the next.)

A “chunk” would then approximately the 10 km around the player.
You’d also want to load condensed data.
I.e. at a distance of more than 10 km, just a simplified terrain plus points of interest.
At a planetary scale, a grossly simplified terrain (or none at all, depending on whether the player needs to see maps or not), plus points of interest (e.g. if he controls an ICBM site, the points of interest that he might target, but just their positions, no detail data).
At a solar system scale, only the positions, size, and overall color of the planets and the start.
Etc. etc. - level of detail, essentially.

Given the number of LOD levels and that you’ll need to code every level twice, once for the case with details and once for the case without details, that’s quite some work.
You can reduce that work by making it impossible for people to interact at multiple levels at the same time. I.e. a laser (personal weapon) that goes outside the 10km range always misses, bullets vanish after travelling 10 km, ICBMs will never be allowed to leave the planetary scale. It is a restriction on game design - you won’t be able to have sharpshooters between the asteroids because they’re more than 10 km apart, typically. (But having such a restriction is still less worse than having no game at all.)

  1. You don’t want to even store the individual stars of a galaxy:
    If you store position, color, and brightness as floats, you have five floats = 20 bytes per star.
    At 300 trillion start in the Milky Way, that’s 6 gigabytes of raw data that need to be processed, and just to generate a merry skybox - that’s a bit over the top.
    (Usually, the overhead associated with raw data is somewhere between double and quadruple that. I.e. if it’s in a database, indexes will take at least another 6 GB, if it’s just in RAM, you still need tree nodes and memory management overhead.)

You might generate starfield densities in voxels. As long as you subdivide each voxel until it’s smaller than a screen pixel, the difference won’t be visible.
That’s going to reduce the data volume from gigabytes to mere megabytes - still too much for modern hardware to easily deal with it, so you’ll spend a lot of time optimizing that via precalculations, trying larger voxels while keeping the visual effects to a “reasonable minimum” (for whatever definition of “reasonable”), and generally spend a lot of time pulling your hair.

  1. Is essentially the same as (2), just at a different level of detail.

  2. You’ll want to keep the game state and the scene display in different coordinate spaces anyway. Mostly for precision reasons: if you fly a fleet across the galaxy, a distance of 500 m between ships will be rounded down to 0 at a light year’s distance - ka-boom!
    I suspect you’re thinking a client-server game, in which case the clients can use scene coordinates and the server can use game coordinates. The bottleneck is the network, you’ll always have enough CPU to convert coordinates.

  3. Whenever a ship leaves a chunk, simply load enough neighbouring chunks. Keep enough data around so that no effect that the ship can experience or cause can leave the set of loaded chunks.
    The art is to keep the game design within the limits of what can be kept in RAM - shooting at a light year’s distance is going to require a lot of chunks loaded.

  4. Whatever software you find, it will use a different data model than what’s useful for your game.

Of course, you could simplify your life and simply generate a starfield background (I think there is software for that). Players won’t be able to reach the stars seen in it, but then who needs a playing field of more than a few hundred stars.
That might indeed be the better approach. Eve Online has just a few hundred stars, simulated in a cluster of a LOT of servers - I don’t know how many, but it was enough that they got a special offer from Microsoft so they’d migrate to it (I think they did take the offer, only to revert to Linux a few years later).

1 Like

@madjack : Indeed, I should not render the whole Universe. But what would be a good way to save objects then, so as to access them very fast to print/load them depending on where they are? Is it chunk compatible?

@lightmax here is an example of a spaceship:
https://code.google.com/p/jme-simple-examples/

@lightmax Finding the nearest chunk is a classic “find the nearest neighbour(s)” problem.
If everything is in RAM, you’ll want an octree.
If it is in a database, a “spatial index” is structurally the same as an octree. Just make sure that the extension you’re considering for geometric queries isn’t restricted to surface geometry (such extensions were designed and built for storing Earthly geography).

If things are hierarchically clusteres (galaxy, star cluster, star system, planet), you can restrict the nearest-neighbour search to subelements that are part of the same cluster. I.e. if you want to find the neighbouring planetary chunks, have all chunks be children of the same solar system chunk, and just do a brute-force search there. (In a database, the equivalent would be a primary key that’s galaxy,cluster,solarsystem,planet, town.)
Limit the number of subelements in a cluster to a low four-digit number to avoid performance problems. If the number of chunks gets too large, find a way to keep it down - e.g. by simply deleting the least recently visited wilderness chunk. This assumes that wilderness doesn’t have any features that a player would want to visit again, of course.)

@toolforger : First of all, thank you for your extensive answer.

The user will be some sort of a transcendent entity, guiding one civilization through the whole Universe. The user, and the ships, can navigate through the solar systems, the stars, the galaxy, the galaxies clusters, and so on, as to simulate an entire (well, not as big as of course) Universe.

About the ships fight, I was thinking of getting ride of the current scene to make a smaller one for the fight, same as planet attacking (would they be by ship or by on ground forces).

For the chunk format, I was thinking of using minimal information storage, each chunk being in one file (I don’t know if it can really slow that much the navigation, since we’re not loading many chunks at once - maybe not loading them while the user/ships are in FTL/hyperspace?), the chunks being called depending on their position, so I can know which file to load (but for chunk chunk-4000000000-4000000000-4000000000.chunk the file name can become much too long depending on the place it’s stored…)?
It could be helpful to have the chunks organized following the natural order, Universe-Super Clusters-Clusters-Galaxies-Galaxy-Solar system-Star/Planet but i don’t see how to make this possible. Maybe a simple Universe (folder)-Super Clusters (folders)-Clusters (folders)-Galaxies (folders)-Galaxy (chunk)-Solar system (in chunk)-Star/Planet (in chunk)?

I’d avoid file systems. They are nice in some ways, but they also have too many failure modes that you’d have to keep under control: permission problems, tampering, file system limitations (some file systems get slow with many files in a directory, others impose limits on the nesting depth, etc. etc.).
Databases are built for that. A “chunk” is simply the set of database records that share a common initial section of the primary key. I.e. for the PLANETS table, the primary key would be galaxy/cluster/star/planet, and everything with the same galaxy/cluster/star values is your “folder” if you will.

And the indexing in the database will handle your queries for you.
Just use axis aligned bounding box and its just a:
select data where x > xmin and xymin and yzmin and z<zmax.

Note that querying by AABB will cause the database to use just the first coordinate’s index (say, X).

If that’s already enough to narrow down the search to a really small set of records, that’s well. I.e. it will work on randomly-distributed tiny clusters distributed across huge amounts of empty space - assuming they don’t happen to randomly line up in large numbers on, say, the same X coordinate (i.e. you need to be rather careful to avoid introducing determinism into your source of random coordinates).

If the records have a non-negligible density distribution across the entire space, this will not be enough.
Think a planetary surface with a billion of chunks, each at a fixed position. The database will be able to narrow down the search to a small strip of chunks that have roughly the same X coordinate, but it does not have a good index to use for Y.

Well, okay, I was a bit too pessimistic (but not by much).

Mysql supports OpenGIS, which can handle two dimensions.
Postgresql is similarly restricted to two dimensions (don’t know whether it’s OpenGIS or something unrelated, but it’s limited).
Oracle can do “skip scans”, which might work well enough for spatial data. I’d be unsure and ty to benchmark it (the price tag would put me off though).
You could also try hsqldb or its successor, H2.

Either way, I’d fill a table with a few trillion records with random coordinates of whatever kind of distribution I want, and then benchmark away.
I’d also want to run two benchmarks, one with hierarchical clusters (similar to what you’ll need for the stuff above planetary scale) and one with a uniform distribution (so you see how it will fare with chunks on planetary surfaces).

@toolforger said: Note that querying by AABB will cause the database to use just the first coordinate's index (say, X).

Postgresql is similarly restricted to two dimensions (don’t know whether it’s OpenGIS or something unrelated, but it’s limited).

False, http://www.postgresql.org/docs/8.3/static/indexes-bitmap-scans.html Also multi indices are an alternative option.

Side optimisation possible,
if you rarly write, cache freqently used sectors in views that are materialized.

Yeah, I’d do some quick tests with a few different database and suchlike before making any decisions.

It should be possible using an index on x,y,z which will immediately find the lower level of the range. I’m not sure how it will then cope with the next step and different databases will vary but its worth trying. After all selecting by ranges is a common requirement.

@Empire Phoenix
From the description that the linked page, pgsql will create three really large bitmap indexes, one for X BETWEEN xmin AND xmax, one for Y, one for Z, then find the intersection.
These indexes are going to be rebuilt for every query (it’s unlikely that the same combination of x/y/z/min/max will come along again), so pgsql can’t keep them around for very long.
This looks like reducing the effort from N dimensions to (N-1) dimensions, which is certainly very helpful but still cannot match following octree links (or using a Morton index).

(Somebody please switch off this silly smiley substitution, grr)