An introduction to Entity Systems

Hey all,

I posted up a little article about Entity Systems on the Attack of the Gelatinous Blob indieDB page.

If you have been wondering what they are all about, or have never heard of them before and are now curious, then it would be worth a read. If you are a seasoned ES veteran then you might be a little bored :slight_smile:

enjoy

10 Likes

Awesome, will have to check it out later

As always, i always like reading about how others solve certain situations. I can’t wait to read something about AI :wink:



On more detailed questions:



I have seen you use

[java]

List<entityIDs> = es.getAllEntitiesWithComponents(AttackComponent.class, HealthComponent.class);

[/java]

in your EntitySystems…



i am curious about this function, since my first implementations was similar. I did a full iteration over all entitys for each each entity system. The result was that this caused a big bottleneck and the result for these queries are likely the same on every frame…



What i did to solve this was to introduce a EntitySystemToEntityMapper (ESTEM) .



Basically when a Entity gets a new Component then the ESTEM needs only the recheck every EntitySystem (which are more likely less then entity’s) and refresh the mapping…



The refresh happens as first thing on every frame if a component was added or removed.



I additionally have ‘swiched’ also the execution: Instead of doing

[java]

for each EntitySystem{

Execute System for all Entitys with this components

}

[/java]



i do:



[java]

for each Entity{

Execute all EntitySystems available for the components

}

[/java]



The performance win was really big… The ESTEM boosted performance incredibly, while the second part was mainly for multithreading optimisation…

1 Like

Theres no need to extend the simple system to add perfromance improvements. Rather apply… performance improvements ^^ Caching comes to mind, afaik @pspeed caches even changes to the entries, so he basically never needs to scan the whole dataset.

I have two Maps of Maps, each used for a different kind of lookup, sort of like an index on a database table.

The main map for looking up all entities with a certain component looks like this:

Map<Class<? extends Component>, Map<Integer, ? extends Component>>



So there I look up the component type of HealthComponent.class and that gives me a map of entityId:actualComponentObject (every entity with a component of that type). For multiple component types I perform the lookup for each type, and then get all entities that are in each one. It turns out to be incredibly fast.



I also have another map for getting all components for a specific entity:

Map<Integer,Map<Class<? extends Component>, Component>>

There I can look up the entity using its ID, then I can pick what component I want using the Component.class

Using Artemis (The Artemis Framework: Entity and Component Game Design | Gamadu) those things get solved for you @zzuegg

@zzuegg said:
As always, i always like reading about how others solve certain situations. I can't wait to read something about AI ;)

On more detailed questions:

I have seen you use
[java]
List&lt;entityIDs&gt; = es.getAllEntitiesWithComponents(AttackComponent.class, HealthComponent.class);
[/java]
in your EntitySystems..

i am curious about this function, since my first implementations was similar. I did a full iteration over all entitys for each each entity system. The result was that this caused a big bottleneck and the result for these queries are likely the same on every frame..

What i did to solve this was to introduce a EntitySystemToEntityMapper (ESTEM) .

Basically when a Entity gets a new Component then the ESTEM needs only the recheck every EntitySystem (which are more likely less then entity's) and refresh the mapping..

The refresh happens as first thing on every frame if a component was added or removed.

I additionally have 'swiched' also the execution: Instead of doing
[java]
for each EntitySystem{
Execute System for all Entitys with this components
}
[/java]

i do:

[java]
for each Entity{
Execute all EntitySystems available for the components
}
[/java]

The performance win was really big.. The ESTEM boosted performance incredibly, while the second part was mainly for multithreading optimisation..


Formalizing the "systems" in an entity system is generally the wrong approach in my opinion. It forces you to do everything single threaded and the greatest feature of a properly implemented entity system is that you are free to use the entity system from any thread at any time.

"Systems" are just the "code that isn't an entity system". They can be JME controls, they can be separate executor services, etc..

Like any data base system (and an entity system is essentially a fancy database, in memory or not), the query is main point of optimization. There are numerous ways to optimize this without throwing away the scalability benefits of an entity system's thread-flexibility architecture.

For example, in Mythruna I can grab EntitySets. These are essentially Set except they have a few extra methods on them.

An EntitySet is similar a database ResultSet except that it acts just like a Set and the query can be reissued. But in a sense, it is a handle on a changing set of results... when they actually change is totally under the caller's control.

For example, a visual "system" (a JME control in this case), might then just grab one for Position and ModelInfo components:
[java]
EntitySet entities = es.findEntities( Position.class, ModelInfo.class );

....

public void update(float tpf) {
// Apply any changes to the results since the last frame
if( entities.applyChanges() ) {
// Do things for the entities that are no longer members of the result set
remove( entities.getRemovedEntities() );

// Do things for entities that are new members of the result set
add( entities.getAddedEntities() );

// Do things for entities that have updated components
changed( entities.getChangedEntities() );
}
[/java]

Mythruna takes advantage of the fact that its Entity System is all used by one JVM process and then just propagates changes under the covers. EntitySet.applyChanges() is essentially a free call if none of the entities has changed. And it's a minimal call if they have since the changes were already accumulated and ready to go.

The concept could be extended to multi-process systems, too. The idea of cached queries like this is not a particularly new one, I guess and there are various ways to approach the problem depending on requirements.

To me, though, the query is the single biggest point to optimize and worth a little time to do right.

Hehe its somewhat similar to the entity system i use, main difference is, I do stuff fully eventbased, so I rarly need querys at all. Also in my entity system, each component has all clientdside proxy data everytime, so there is more than just the id. (Eg position rotation), since this simiplifies networking due to me never having to know anything on that layer about the entity at all.

@EmpirePhoenix said:
Hehe its somewhat similar to the entity system i use, main difference is, I do stuff fully eventbased, so I rarly need querys at all. Also in my entity system, each component has all clientdside proxy data everytime, so there is more than just the id. (Eg position rotation), since this simiplifies networking due to me never having to know anything on that layer about the entity at all.


In my entity system an "entity" (little E) is just an ID. You can do all interactions with the entity system, an EntityID, and the components you want to mess with.

EntitySets provide Entity objects (big E). These are locally cached "entities" (little E) representing a snapshot of when the query was last run.

The problem that I had with typical event systems in a general architecture better suited to update loops was the potential for out of band messages. Now, all of the events are collected inside the current EntitySets and only applied to the local copies with applyChanges() is called... then those Entity objects in that set are frozen in time until the next update is called.

Multithreading in this architecture is essentially free. I pretty much never have to think about it. The network layer iterates over the entities that it is interested in and blasts them out.

I will say that my current network architecture that I'm developing for physics state is _much_ more complicated than that. It's essentially bypassing the entity system on the server. The physics thread uses the entity system to track the mobile objects and blasts state updates to a ZoneManager. State is collected per zone and a separate state collector thread packages the relevant state out for each player. These objects are very specific to the networking layer and there are no entities at that point.

The client then unpacks these messages and modifies local state in the client-side entity system. To the client, it looks just like it's running a local physics instance directly updating the entities... all through the entity system.

I should write an article sometime.
2 Likes

Please, do not confuse some system that calls some objects “entities” with an entity system as described here. This is a very specific implementation that doesn’t allow much “idea adding” or hampering with the general structure. Its very specific and should be done exactly as described to actually be useful.

Nice reading!

Concise and easy to understand.

Thanks a lot

@normen said:
Please, do not confuse some system that calls some objects "entities" with an entity system as described here. This is a very specific implementation that doesn't allow much "idea adding" or hampering with the general structure. Its very specific and should be done exactly as described to actually be useful.


You mean what was describred by @pspeed ?
@shirkit said:
You mean what was describred by @pspeed ?


I don't think so. Normen and I generally agree on Entity System design. Pretty sure it was directed at the other posts talking about taking it in odd directions.
@pspeed said:
I don't think so. Normen and I generally agree on Entity System design. Pretty sure it was directed at the other posts talking about taking it in odd directions.


That's why I asked, suddenly I thought that my whole implementation of Entity Systems wasn't an ES!

Hey guys, I have a question(Ok maybe 3) for you.

The overall concepts of the ES aren’t a problem for me, I’m used to using Hibernate at work and breaking things down to data sets is fairly commonplace in my head. Its rather easy for me to view ES as a relational database in memory.

The confusion starts to come in:
How the loops generally look in an example system?
How do systems trade data?

You also mention this function:

[java]es.getAllEntitiesWithComponents(ComponentA.class, ComponentB.class);[/java]

could you pseudocode if for me? Im pretty sure i could run from there after seeing what this does behind the hood.

And Sploreg, the article you wrote is probably the best crash course on ES that I’ve seen so far. Too many “tutorials” on ES assume a rather large amount of previous knowledge from the reader. Good job. Wish it came front page on google three weeks ago when i started reading on the subject lol

I recommend using Artemis library: The Artemis Framework: Entity and Component Game Design | Gamadu

Also, this is some good material to read about Entity Systems

http://slick.cokeandcode.com/wiki/doku.php?id=entity_tutorial

http://t-machine.org/index.php/2007/09/03/entity-systems-are-the-future-of-mmog-development-part-1/

http://www.richardlord.net/blog/what-is-an-entity-framework

http://www.richardlord.net/blog/why-use-an-entity-framework

http://www.altdevblogaday.com/2011/08/24/the-game-entity-–-part-iv-game-systems/

1 Like

Ive read most of those already, and i dont want to just use someone else’s, i want to write my own :stuck_out_tongue:
Computer scientists: the only people lazy enough to automate everything and yet stubborn enough to WANT to reinvent the wheel, am i right? lol

Im really just hung up on how to the systems communicate to each other, and what is typical operations of a system on its components.
i.e. does it persist its components to the db at the end of every loop? how does it signal for the destruction or creation of an entity? what asynchronous, or synchronous communication is there between systems?

its the algorithms that keep things running that are the concern, not how to break from the OOP mindset of the placement of data.
i.e. whats behind the es.getAllWhatever() black box of a function i quoted?

i may not be stating my problem very clearly. let me know and ill try to clarify further.

I think you are overthinking it. And with the t-systems article, it was important to read all of the parts and a lot of the comments. Many clarifications only existed in the comments.

“Systems” (there is no such formal thing) “communicate” through components… in that they care about the components that they care about and some other code might be changing those components.

In my ES, I have transient components and persistent components… so not all of them go to the DB. Really the using code shouldn’t care about that, though.

As a rough example, the physics system watches all components with Position, MassProperties, and a PhysicsState ← indicates awake or asleep. It moves the rigid bodies around and then sets the position component. Other systems might be interested in entities with Position and “ModelInfo” components and so move some visual representation of the objects because now the position has changed.

My ES is thread safe. The entity sets that it gets from a query are local views and must be updated to sync with the “real” values. This allows the system to control when it gets a refresh of data (generally once per frame, for example). I think some of this was already covered above, though.

I’m having trouble to make my implementation thread safe (multi-threaded). I have some systems that updates the components, and other systems that uses those values that were/will be updated by the other systems (health component for example). I still don’t know how to make it work properly with everything. One system handles how units attack eachother, while the other moves the units. If the unit dies while it’s moving, sir, we have a problem (on my end).

@shirkit said: I'm having trouble to make my implementation thread safe (multi-threaded). I have some systems that updates the components, and other systems that uses those values that were/will be updated by the other systems (health component for example). I still don't know how to make it work properly with everything. One system handles how units attack eachother, while the other moves the units. If the unit dies while it's moving, sir, we have a problem (on my end).

It’s a tricky thing to get right but really worth it in the long run. The key is making sure that your “systems” are getting local views of the entities… that way threads don’t stomp on each other because they aren’t even sharing the same data.

Then the only trick is updating the local views safely and there are a variety of ways to do that. The brute force approach would be to sync on the whole ES while updating. I use a more subtle approach where my query result sets keep thread-safe change lists that the system holding the results can apply when it desires. It’s like a change event model but entirely hidden away from the using code. The results (or EntitySets) keep a concurrent linked queue of changes. The “entity system” remembers all EntitySets that it gave out (until they are explicitly released) and sends any change events to them.

The loop that applies the changes can be simple or complex with varying levels of overall performance as needed. Mine does a lot of tricks to try to ignore changes that would never be relevant for the EntitySet. The brute force approach here is just to keep all changes and wait to ignore them until it’s time to apply them to the local entities. For many ES usage patterns, that would be just fine, I think.