I’m designing a MMO-like server/client system, and I have a question about using the jME PhysicsSpace and native Bullet on the server. Relevant background information about the server:
The server is not a jME application - it has its own main loop and lifecycle. The main loop (heartbeat) is run on a ScheduledThreadPoolExecutor. Each iteration of the main loop is guaranteed to happen strictly in sequence with the previous one, but of course it may run on a different underlying thread each time.
The physics system as it stands will calculate each tick as part of the heartbeat (not sure if this is a great design or not - this is flexible and can change), meaning that the thread that runs one tick may not be the same thread that runs the next tick, as per item 1.
Here’s where I’m not sure about interfacing this design with the Bullet native libraries. The Javadoc for PhysicsSpace.create() says: “Has to be called from the (designated) physics thread.” Does this mean that each PhysicsSpace instance must be used by one and only one thread for the duration of its lifetime? The thread pool will properly synchronize the Java portions so that there are no memory consistency problems, but what about the native portions? Do the native binding and/or Bullet ensure memory visibility across threads, or will running different physics ticks on different threads cause memory visibility problems with the native code?
I would stick to object creation in separate threads and leave the main loop single-threaded for so many reasons unless processes are completely unrelated and rarely need synchronization between one another.
In your first point - you say each loop is guaranteed to occur in sequence - do you mean expected? Or do you terminate the thread if it takes longer than it should?
Thanks - I expected as much, but wanted to verify that. I’ll have to spawn dedicated threads for the PhysicsSpaces.
The internals of the server are largely decoupled in terms of synchronization. I’ve designed an entity system (not an ECS) where each entity is self contained and interacts with the world primarily by an event dispatch system, with the intent that events will be processed by different entities in parallel. I do not believe that this poses a problem in terms of keeping events in “proper” order, as any event that an entity receives “happens-before” any subsequent event. If one entity triggers events during its event processing, any other entities will finish processing previously triggered events before they process the new ones. This gives a reasonable ordering guarantee but gives a lot of flexibility for multithreading. The idea is that most of the work in running the world simulation will happen in the entity system. Other systems where synchronization is important, like the physics system, are currently just tied to the heartbeat and can interact with the entity system via the event system or by clearly specified thread-safe methods.
In short, running the heartbeat loop in the thread pool is just a convenience for sharing native threads with the other systems.
Thanks for pointing this out - I went back and reviewed my code and discovered a significant bug. In an earlier version I had this correct - each iteration of the heartbeat loop would schedule the next one at the end of its processing. If one heartbeat took “too long” it didn’t matter, since the next iteration wouldn’t be scheduled until the prior iteration finished all of its work. However… I then decided to get clever and eliminate the extra scheduling overhead caused by scheduling a single run per iteration by using scheduleAtFixedRate(), which could easily cause one heartbeat to run before another finishes. Oops. I may just move the heartbeat into a dedicated thread for simplicity… modern OS thread managers tend to do really well with scheduling so there probably isn’t any practical benefit to trying to share threads (and, as I just demonstrated, it’s very, very easy to mess up ).
Needless to say, server development is still in a very early stage.
Have you ever used threadlocal? I’d always given it a miss until I saw it in action - it can be surprisingly useful - especially when you’re spawning or dealing with large quantities of the same type of object in a multi-threaded manner.
And here is a demonstration of what it can achieve.
What’s happening here is that instead of creating a mesh generator object every time you need a mesh - it already has them set up - thread locally - so you don’t need to keep creating the objects - and each one can be accessed in a multi-threaded manner - exactly as it was designed to do.
No, I’ve never used it. I’ve always just used plain ol’ synchronization and maybe thread safe collections. Thanks for the example - I’ll look into it more.
The absolute slowest way to protect data structures, by the way. So much work has been done to create lock-free code for you in the JDK to avoid synchronization overhead.
Your code sounds really ‘busy’ anyway so it may not matter.
Yes, but in practice I generally only use synchronization in places where the synchronized code will be called infrequently and without contention and where using JDK classes is likely to just over complicate things and/or obscure intent. When I expect a piece of code to incur frequent concurrent use, I almost always delegate to a nonblocking JDK collection like a queue or a ConcurrentHashMap.
If by ‘busy’ you mean that most of the code runs for extended periods of time without accessing data across threads, then yes, it is busy by design. Lots of cross-thread data access just means much more potential for lock contention and/or subtle threading bugs.
Your description made it sound like there were events flying around all over the place. That’s what I meant by busy. It may have just been the way it was worded, though.
Yes, keeping individual systems loosely coupled by events is part of the design, and it should greatly simplify multithreaded code. Events get tossed into a queue and then dispatched from there, so events can easily flow across threads. No thread safety concerns, no synchronization except for whatever is common to all cross-thread event listeners. Adding another system is as easy as designing it, adding it into the heartbeat loop and/or registering event listeners, and letting it run.
The lock free collections in java are actuality very good. Also they are mostly not built on top of the old java synchronize stuff. They use proper compare set atomic instructions. In fact some lock free structures need that sort of thing. Performance is great from my tests, and balance is good when you set them to be fair, at the cost of some throughput of course.
Also these are not simple data structures. So it good some other sucker wrote them for us.
I basically replaced all my old synchronized code with the classes under java.util.concurrent.* . These are old classes now. And if you want a little more control there is ReadWriteLock, count down locks and well. I would 100% recommend looking at these classes and recommend refactoring where possible. It will improve everything about your threaded code.
Yes, I use these extensively. Most of the thread safety in my code (probably over 90%) comes from one of these, usually ConcurrentHashMap or ConcurrentLinkedQueue.
That’s almost the best thing about them.
Good to know. I probably gave a bit of a misimpression of my use of plain ol’ synchronized locking in my earlier posts. Easily 90% of my multithreaded code is using java.util.concurrent classes. Recently I’ve only used synchronized in places where a lock on a value change is needed, i.e., accessing a non-thread safe collection in a multithreaded environment where I expect that lock contention will be extremely rare and a full-blown lock free structure is overkill. Volatile gets used for the occasional boolean flag value (i.e., shutdown signals). Basically I just use synchronized/volatile in the (fairly rare) cases where I deem that using the java.util.concurrent classes would be either overkill or just add complexity/more typing that I didn’t really need to do (i.e., why add a lock class instance when all I want or need is the very simple synchronization/memory visibility guarantees of synchronized methods/blocks? At best I get to type more than I needed to, at worst I do something dumb like forget to release a lock.).
All that being said, at your recommendation I think I’ll go back and have another look over the Javadocs for some of the classes I’ve never used and see if I’m missing out on some goodies that would improve my life/code.
Architecturally, it sounds a lot like what happens in an ES/ECS… except a lot of the ‘listener’ code is eliminated and ‘events’ are really ‘component updates’ and thus interest is tightly managed.
Yes, although the event system is intended to be used for just about everything that happens inside the world, and possibly even things that are convenient for integrating server systems together that don’t have much direct correlation to the world simulation. I am developing my own ES for this, and the event system will be a very central component of it. In my system, entities have “attributes” (components, and yes, there is an “AttributeChangedEvent” class in my sources ) and “agents.” Agents are where my system sharply diverges from a classic ECS. Each entity gets at most one agent, which is the sole controller of that entity. The intention is that there will be an agent for AI controlled entities, an agent running an embedded scripting engine that I wrote for scripted entities, an agent for controlling characters of users logged in over the network, etc. Agents, however, are not nearly as powerful as ECS systems are since each entity can be controlled by at most one agent. The ES is still very up in the air, and in the end I may wind up stripping it out and replacing it with Zay-Es. On the bright side, with the exception of the 2000+ line scripting engine it all fits into 3 or 4 small classes.
).
