Native bullet build process?

Troubleshooting physics
1

Hi all,

does anybody have a description of the build process for native bullet?
I need the library reccompiled with double instead of float for gameworld simulation (on the server, independently of any scenegraph).

I got Bullet downloaded, but I’m not sure about the options I need to set, about what error messages to ignore or fix, and I haven’t even begun to tackle Java wrappers.
Given that JME already has all this done, I thought some tips might save me a lot of time futzing around in dead ends until I find the path to a working native engine.

Pointers? Links? Tips?

2

You don’t need to download bullet, just do whats written it trunk/engine/bullet-native-build.txt
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or theres an increased use of exclamation marks and all-capital words.

3

Ah, thanks.

I found build-bullet-natives.xml, I guess I’ll have to change the CMAKE options to make it build for doubles.
Fun times ahead adapting the Java wrappers to doubles though (sometimes I hate Java for disallowing generics for primitive types).

4
@toolforger said: Ah, thanks.

I found build-bullet-natives.xml, I guess I’ll have to change the CMAKE options to make it build for doubles.
Fun times ahead adapting the Java wrappers to doubles though (sometimes I hate Java for disallowing generics for primitive types).


I’d suggest to not do that, and instead just setting the internal computing of bullet to doubles and then still translating to float values using the existing wrappers, that would really be just a flag change, don’t ask me for the details on where that flag is though.

Currently you have about (rough, depends as always but this is the minimum) 50 square kilometers of completely problem-free physics space that you can explore. This is ofc way below the limits of 32bit float values but because of the internal computations these values can quickly become bigger inside bullet. You will find that funnily most open source AAA games that have “open worlds” actually obey to those limits in one form or the other. So the obvious solution is to make the whole 32bit range available to yourself by letting bullet computer in 64bit internally which native bullet can do.

For “endless” worlds the concept of trying to store endless values in a limited array of bits is flawed per se. What you want to do is simulate the range of perception and resulting granularity for the user / player. This can be done by not moving the player inside the world but moving the world around the player, applying the physics “backwards” basically and only for objects where the player can perceive their physicality :slight_smile: The surroundings and objects could be streamed in from any kind of data source which doesn’t have to be limited to the 32bit range but can be any kind of descriptor you want, you only need to know the player position in the coordinate system to stream in that part of the world.
The factual limitations of the perception range and granularity (you cannot see or point at the mars rover from here) will allow you to limit to the 32bit range easily. If you are looking at said mars rover through a telescope no problem, with the telescope position in those absolute world locations and the direction its pointing at you can determine what location you actually see and pull it from the database (or whatever).

The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or theres an increased use of exclamation marks and all-capital words.

5

Oh, finite bit arrays are enough to have an endless world. If anybody wanted to go to expressing coordinates to sub-quantum accuracy across the entire visible universe, 120 bits would be more than enough. I doubt anybody would want that though. :slight_smile:

About whether the Java side needs doubles: We’ve had that discussion already, and I don’t think we’ll ever agree on that, so I’m not going to argue the point.

I agree that for some projects, it may be enough to run Bullet with doubles but use only floats on the interface between Bullet and the scene graph.

Hm.
Getting coordinates into Bullet might work by applying a local transformation. Only the transformation needs to have double precision, the rest can be float. I think I saw something along these lines.

Getting coordinates out of Bullet would require an inverse transformation. I’ll probably want to tell bullet “here, this capsule shape is defining the local coordinate system, please transform all coordinates so that they are relative to this capsule”.
Does such a feature exist in Bullet?

6

Bullet computes absolute physics values but its absolutely no problem to adapt them to another matrix… Its something that is happening constantly in 3d graphics… I would again suggest not trying to make the system work differently but adapt your own systems to accommodate for the system you intend to use to solve some problem.
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or theres an increased use of exclamation marks and all-capital words.

7

Your suggestion is to avoid rewrapping the native lib with doubles everywhere; that would indeed save considerable work. However, to avoid relevant precision loss, the transformation needs to be done in doubles; so if the Java side is to avoid doubles, the transforms need to happen on the bullet side.

Is there a way to submit a coordinate transform matrix to Bullet, so that it will transform all global coordinates via that matrix whenever I’m getting or setting coordinates? The matrix itself would have to use doubles, so I’m not 100% relieved of modifying the Bullet wrapper as taken from jme, but I’d have to touch just that one routine that submits the matrix instead of touching every single interface function.
[EDIT] I’m seeing there’s a setWorldPosition and a setWorldOrientation function somewhere; is that it?

8

There is not really a need to go from double to float if you stay within the nominal range of floats, then it will only boil down to accuracy which should suffice if you stay within reasonable ranges for the values. So if you had 50.000 as max locations before you can easily go to 5.000.000 on the java side if you just update bullet to doubles and transfer the variables to float format still without experiencing issues. (I pull these numbers out of my rear obviously but they should about fit). Given all of OpenGL works in 32bit you will be limited to such ranges for “visible” content anyway, the algorithms used to create effects experience similar issues when going too far in the float range (before the “end of accuracy”) due to the same reasons (computations creating values going beyond the input values).
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

9

Floats remain precise up to a value of roughly 17e6. So at 17 km, you are precise down to the millimeter, 170 km allows for centimeters. (I happen to have these numbers lying around, thought you might appreciate having better numbers than those pulled out of wherever.)

I want to try out a single physics space for the entire simulation. I hear Bullet can apply broadphase optimizations for raycasting, which sounds as if it might have begun to be a viable; if that works out, people can stop worrying about segmentation issues, freeing up a lot of people from spending tons of ingenuity, time and code on segmentation issues.
That means I’ll have global coordinates using doubles. No problem inside Bullet, but if I want to use floats for most operations, telling bullet that “ship is at x = 170,000,000.001f m, laser beam at 170,000,000.002f m” won’t work since floats can’t express the difference; now if I can tell bullet “in my local coordinate system with x = 170,000,000d, if ship is at 0.001f and laser beam is at 0.002f”, then I don’t get noticeable inaccuracies. (Simplifying from a matrix down to a single coordinate offset here.)
The question I’m having is whether I can even tell Bullet about my local coordinate system. If I can, I may get away with using doubles just for the Bullet functions that deal with the local coordinates matrix; if not, I’ll have to write a new all-doubles wrapper for Bullet.

10

As said the level of “enough accuracy” is dependent on the calculations, there is no absolute value to define when you get funny results for physics or 3d effects.

I don’t get why you have to forcefully try to move the coordinate system in bullet before instead of moving its output data after the fact when you apply it to your objects. This is solely to please your own brain so it can think in one coordinate system and the computer has to cope with that nonsense… @EmpirePhoenix also went this questionable way, I think he even did a double version of the wrapper, ask him. Else I guess you will have to write it all yourself if you want to go with your head through the wall (or rather prove yourself it doesn’t make sense - which this will amount to in the end).
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

11
@normen said:I don't get why you have to forcefully try to move the coordinate system in bullet before instead of moving its output data after the fact when you apply it to your objects. This is solely to please your own brain so it can think in one coordinate system and the computer has to cope with that nonsense..

You’re again on that short-circuit logic of “if I don’t understand it, it must be a bad idea”. Before jumping to conclusions, you should at least attempt to post questions. You might, well, just understand what I’m doing.
Your assumptions about my motives are dead wrong, I don’t care about the coordinate system.

But nevermind. I looked at the bullet wrapper, and rewriting it to all doubles should be less work than convincing you of anything that you cannot or would not understand, so I’ll simply stop here. Feel free to stick with whatever beliefs you have, but please take them to your private space and don’t pollute my question threads with that, it’s just distracting if you don’t understand what or why I am doing there (and clearly you didn’t).
The one risk remaining that I have on my radar is that Bullet doesn’t scale to large object counts as well as they claim.

For anybody else who’s interested, here’s my current plan:

  1. Test how well Bullet scales.
  2. If the test works out, redo the Bullet wrapper using doubles (should relatively straightforward unless there are hidden traps - experience reports, anyone?)
  3. Set up the game’s pipeline as storage (float or double) single global Bullet space (double) jme engines on clients, each with its own local coordinate space (float).

I don’t need a double-based Bullet for step 1, so I’m off writing some performance tests and seeing where the limits of Bullet truly are.

12
@toolforger said: You're again on that short-circuit logic of "if I don't understand it, it must be a bad idea". Before jumping to conclusions, you should at least attempt to post questions. You might, well, just understand what I'm doing. Your assumptions about my motives are dead wrong, I don't care about the coordinate system.

But nevermind. I looked at the bullet wrapper, and rewriting it to all doubles should be less work than convincing you of anything that you cannot or would not understand, so I’ll simply stop here. Feel free to stick with whatever beliefs you have, but please take them to your private space and don’t pollute my question threads with that, it’s just distracting if you don’t understand what or why I am doing there (and clearly you didn’t).
The one risk remaining that I have on my radar is that Bullet doesn’t scale to large object counts as well as they claim.

For anybody else who’s interested, here’s my current plan:

  1. Test how well Bullet scales.
  2. If the test works out, redo the Bullet wrapper using doubles (should relatively straightforward unless there are hidden traps - experience reports, anyone?)
  3. Set up the game’s pipeline as storage (float or double) single global Bullet space (double) jme engines on clients, each with its own local coordinate space (float).

I don’t need a double-based Bullet for step 1, so I’m off writing some performance tests and seeing where the limits of Bullet truly are.

Please don’t try that shit on me, its just silly. You did not explain what you want to do and I am trying to understand what you do and ask questions to help you, please note the fineprint in the post you interpreted as “I don’t understand it but it must be a bad idea”, it doesn’t allow that interpretation. But as you seem unable to explain what you want (or finally how you want to run it on a normal computer system I guess) I will stop those attempts now unless you can explain what you want. This includes why and how you want to cope with a million-square-miles area at millimeter resolution inside one physics system. You seem perfectly able to convert the wrapper to doubles, so go ahead.

The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

13

I’ll let that go uncommented.

14
@toolforger said: I'll let that go uncommented.
Apart from those downvotes I guess ^^ Oh no, I guess it must have been somebody who understood your hidden explanation and also thinks I am such an asshole here :roll:

So I will comment it for you:

Obviously you know that you don’t have a proper explanation. Maybe you’ll say “I don’t want to adapt your solution space to my problem space” again, which is the sentence I will always remember you for. Even back then you could not give a proper explanation for what the actual use case is, how your actual code or application would look or what it would do. I was the one who said he doesn’t understand it in the first place, completely keeping your “problem space” in the mist is the best way to keep the option to tell other people they don’t know what they’re talking about.

Ergo: You do not seek information, you are too intelligent not to know that this way you wont get it.

The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

15

I downvote comments that detract from the topic or otherwise move me farther away from a solution than I was before. Since you said you don’t care about your score, I reckoned that’s the easiest way to give proper feedback.
Another piece of feedback: not all downvotes were mine.

You say I didn’t properly explain, I say you didn’t properly listen.
No amount of arguing will change your or my position on this issues, so it’s time to agree to disagree and move on.

I have just a single request to make:
Please stop pursuing unasked answers. Mentioning an unasked aspect to check whether it might be on topic is okay; hijacking an entire thread just to make a point that’s already been identified as off-topic is not.

16

@toolforger: Okay I will try it a last time. That “problem” seems to be users not being able to input a float into your system to do that laser beam calculation, at least thats what I get from this thread. So. If your system is supposed to be able to do that, let it input doubles, or better even some “real” galactical coordinates, maybe encoded as a string, you could even use star trek coordinates, it doesn’t matter. Now if you figured out where that spot in space is you can then go ahead and pick in that area, which doesn’t have to be bigger than (float) and also better isn’t because theres a thing you don’t seem think about: Even if each object would just be 2byte in memory (which a full bullet object with collision shape by far isn’t) then all the objects in a mult-square-million meter area would amount to gigabytes of data, if not terrabytes. Not to speak of actual collision and physics computations. This amount of data is normally streamed from disk into memory and if you have to do that anyway you can on the way just translate the coordinates to the local coordinate system. Plus you only have to load those objects in that space into memory. Your “user” would still just go laser.Fire(“1245:38276:45”); That is much easier than rewriting two libraries on the way which I consider “going with the head through the wall”. So, did I understand your problem and did I present a solution? Btw, yes I downvoted the post for the same reason you pronounced you do ;p
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

1 Like
17

No laser beams actually (that would be silly at beyond-float distances), but probes.
In fact detection will be part of the game mechanics. Scan a large sky area and get less range and/or details; scan just where you know the enemy fleet is approaching and get more details from a larger distance.

Floats are simply too cramped. I really tried, but I couldn’t make them work without segmenting the universe. Segmentation has been ruled out for this project because of coding overhead and because it’s constraining game mechanics in ways that don’t fit well with the game logic.
Doubles allow mm granularity at distances of up to 4.6 billion km from the origin. This is still far smaller than a real-life star system, but good enough for a game.
Reality-sized galactic coordinates would require 80 bits, which is beyond the 52 bits of a double or the 64 bits of a long. This would require two longs to deal with that, and I’m not crazy enough to try reimplementing shape intersection with broadphase optimzation with two longs per coordinate.

Streaming from disk isn’t present, the simulation is supposed to run entirely in RAM. The simulation state will be streamed out to disk on a regular basis (once per minute or so) to allow a restart in case of crashes.
This eliminates disk I/O and the overhead of shoehorning a object data model to a relational database.

I’d still do coordinate transforms when building the world.
Using strings for coordinates doesn’t help BTW. Those strings will ultimately have to be converted to floats to go into the jme3-bullet routines, and you’re back at the same precision loss.
Maintaining millimeter accuracy beyond 17 km from the origin means you need to use relative coordinates. I.e. the local transform I was talking about.

You’re right that I wouldn’t be able to put quadrillions of space objects into RAM. I won’t be doing that anyway.
However, with a server with 10 gigs of free RAM and 1 KB per object, I could still deal with 10 million objects. Updating all of them in every 60Hz cycle would put some serious pressure on the L1 cache, and such a scheme will break down at the first O(N²) algorithm that’s anywhere in bullet, jme, or my own code.
If everything is at O(N log N), this should be fine though.
Besides, I don’t plan to really manage 10 million objects; it’s more like 10,000 to 100,000, maybe up to half a million if things scale better than I expect today.

The real issue that I have is:
I need a large universe, and I need some efficient geometric operations: vicinity search, intersecting a really narrow and really deep frustum with whatever is in the geometric space.
I could code that up myself, and spend several man-months on that.
Or I could leverage existing code.
jme3’s octree might have worked but is using floats -> too coarse.
bullet with doubles is just enough; I need to scale down the universe somewhat, but it’s manageable.
longs have even more significant bits, and they’s somewhat easier to deal with, but well, I haven’t found any library that does geometry using longs.

P.S.: Just made sure what the relevant options for cmake are (it’s just one), so modifying the build process should be simple.
Performance tests first though.

18

sorry but i dont think bullet is built for large simulations, large amount of objects maybe, but not in a very large space, or small amount of objects in a very large space, due to the number of calculations required to compute things causes the growth rate of the error to sky rocket ( may want to look at a whole study dedicated to that called numerical analysis). Either way due to those two restrictions, you dont need doubles, floats are fine for any practical purpose, and if they are not then you are doing something wrong, because there is a limit to how far apart 2 objects may be.
Also the thing your talking about fixing all this with local coordinate transforms, i hope you do realize it exactly accomplishes world partitioning, just more complicated, and completely unnecessary.
Also bullet does space partitioning for its queries, that means that potentially you can use bullet or do it yourself maybe and shard the simulation

1 Like
19

I have learned enough about numeric analysis at the TU München to know I don’t want to do that :slight_smile:

I have small objects scattered in a large space. I can’t express both the position and the size of an object at the same time using a float, as in most cases, the shape’s corner points will get rounded to the same coordinate and the end will be a zero-size shape.
I don’t think that floats will fit.

I’m not sure what you mean by “Bullet does space partitioning”. If you mean the broadphase optimization with its AABB trees, then I agree, and I’m trying to leave the partitioning to bullet so the rest of the program does not have to deal with partitioning.
If you mean something else, please elaborate; I’m genuinely interested.

20

I guess he meant something like this: http://hub.jmonkeyengine.org/forum/topic/infinite-zoom-galaxy-with-physics-example-code/ “Coding overhead” one evening btw. Since bullet already partitions the space it would indeed be silly to compute objects in one far away corner of the physics space that don’t collide with any others.
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless theres emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.