I made various changes:
-Added SQL (tested with hsqldb only)
-Added Persistent Components
-Added a HybridEntityData class which stores normal Components into the HashMap and PersistentComponents into the database.
-Changed path to com.jme3.entitysystem
-Added a getEntitySet method to the EntitySystem that differs between necessary components and optional components.
-Updated the test game
(For exampe I added an AnimationComponent and the EntityControl now checks weather the Entity has an AnimationComponent and than display this animation or do nothing.)
How to use the Database?
Download and add the hsqldb library http://hsqldb.org/
[java]
entitySystem = null;
try {
Class.forName(âorg.hsqldb.jdbcDriverâ);
dbConnection = DriverManager.getConnection( âjdbc:hsqldb:mem:test.db; shutdown=true;hsqldb.log_data=false;â, âsaâ, ââ);
entitySystem = new EntitySystem(new HybridEntityData(dbConnection));
[java]
public class VisualRepComponent extends PersistentComponent {
private String assetName;
public VisualRepComponent()
{
}
public VisualRepComponent(String assetName)
{
this.assetName=assetName;
}
public String getAssetName()
{
return assetName;
}
}[/java]
Important: Do not forget the empty constructor, otherwise the reflection will not work and the Component cannot be load from the database.
New Download:
I want to try to release the new version as a plugin. But I have still no gc access.
(Or I am to dumb using SVN)
@pspeed
1.Do you use a connection pool to get a fast access to the database from multiple threads?
2.How did you improve the performance of the check weather an EntitySet is interested in the Event or not? I mean this seems to be the only point where the performance really matters.
@ogerlord said:
@pspeed
1.Do you use a connection pool to get a fast access to the database from multiple threads?
2.How did you improve the performance of the check weather an EntitySet is interested in the Event or not? I mean this seems to be the only point where the performance really matters.
If it will not offend you, I will just check mine into the contrib repo. The response to my offer before was pretty underwhelming and I didnât want to step on anyoneâs toes.
I made a stress test with 1000 Entitys that are changed every update loop and 10 EntitySets which are listening and applyChanges.
The framerate went down to 550 fps. Sure its optimized for multiple threads but I think there must be still things to improve.
Visualvm shows that the applyChange method of the EntitySet is 5 times slower than the processEntityEvent method.
Here are some of my thoughts:
1)As you may noticed I removed the singleton pattern in the first version I released.
This means every Entity has a reference to the EntitySet and the EntitySet has a reference of the EntitySystem.
I could use this to make directly some changes to the EntitySet when a component ist set/removed.
Therefore this EntitySet would not need the Events anymore.
On the other hand I am not sure weather the EntitySet is actually intrested in this changes.
I mean if I set a component, do I really need to check the getChangedNext() method to know this in the next update circle?
2)There could exists EntitySets who are not updated frequently.
In this case there would be tons of EntityEvents in the List of this EntitySet.
One Idea was that only the last changes of a component are relevant. Maybe it is possible to abstract something here.
All in all I dont know where I could make the big performance improvement.
Do someone knows a Profiler who displays the hierarchy how the performance of a method class is put together by the methods called by this method?
JProfiler is able to do this but it is not free and my trial experienced some weeks ago.
Ah, I found the Netbeans Profiler and it has exactly the features I needed.
Therefore it is clarified that the performance during the test is divided into:
45% processEntityEvents
45% applyChanges
10% rest
@pspeed said:
If it will not offend you, I will just check mine into the contrib repo. The response to my offer before was pretty underwhelming and I didn't want to step on anyone's toes.
Ah, now I understand why pspeed uses an interface for the ComponentHandler.
My implementation of the MapData and the SQLData have serveral similaritys.
Besides in his implementation it is not necessary to check every time if this component is persistent.
I hope I can see the Filter implementation today and how to check weather a Component contains a special value or not.
I think I would need 2 Filters for this, one for SQL and one for the map.
When I now go even further in his direction the systems will be actually very similar in the end.
Edit: Is it possible that the performance of the SQLComponentData is completly irrelevant and the only function of SQL is to give an overview of the components during and after the game?
That depends entirely on how often you save.
If you save on every world simulation tick, SQL would probably be very noticeable.
This would decrease if youâre content to save every second, or every minute, or just at end of game session.
Saving every second or every minute would introduce lags unless you put the save into a separate thread, which means the world simulation needs to be thread-safe.
Iâd be interested to hear what the standard advice for dealing with that is.
@toolforger, this ES is supposed to be threadsafe and persistence should be done async in order to minimize the effects of the simulation. Different components could have different persistence needs, for instance trading transactions might be persisted as soon as they happen but it might be good enough to save entity positions every 2 seconds, hammering a SQL database with hundreds/thousands of writes 60 times a second is not a good ideaâŚ
@toolforger said:
That depends entirely on how often you save.
If you save on every world simulation tick, SQL would probably be very noticeable.
This would decrease if you're content to save every second, or every minute, or just at end of game session.
Saving every second or every minute would introduce lags unless you put the save into a separate thread, which means the world simulation needs to be thread-safe.
Iâd be interested to hear what the standard advice for dealing with that is.
âŚor you us a proper tool for the job.
I giggle a little when people make these absolute statements that are just patently false.
Even if you managed this âsaving in a separate threadâ yourself, the simulation thread wouldnât care at all. It just hands the stuff off to be saved and moves on.
@pspeed
I see, my background is primarily with fully durable databases where writes is synced to disk (mysql/postgresql)⌠in memory databases are obviously much faster.
@kwando said:
@pspeed
I see, my background is primarily with fully durable databases where writes is synced to disk (mysql/postgresql)... in memory databases are obviously much faster.
Yeah, it would definitely be slower in those cases but I think it would still be workable. A component write is not a complicated one. All of the SQL is done with simple paramaterized prepared statements and I think even a local postgres over local sockets would not be a bottleneck.
But for me, hsqldb is durable enough. I have the write delay set to 1 second or something⌠and beyond that it journals. Very fast stuff. I was impressed right from the start.
Frankly, I would probably do a postgres implementation very similarly and perhaps just see how it goes. At worst, Iâd just shunt all of the writes off to another thread⌠synchronized durability generally isnât a requirement and itâs kind of a myth in this case anyway. Without far-reaching transaction support one component might get written and another might fail⌠and in general the ES and the App will just deal with it. The nature of the ES-based app is that itâs already sort of written this way anyway.
This hsqldb stuff sounds interesting. Iâve used Derby in the past and found it works well - and embeds directly into your Java application which can be really nice sometimes. I gues HSQLDB does the same since its pure java?
@zarch said:
This hsqldb stuff sounds interesting. I've used Derby in the past and found it works well - and embeds directly into your Java application which can be really nice sometimes. I gues HSQLDB does the same since its pure java?
Yep, you can run it embedded or as a separate server. I run it embedded. I believe it is the fastest Java SQL database.
Roughly 8x faster than Derby when used embedded, I guess:
from: http://hsqldb.org/
Today I tried to implement Normens Inventory and noticed that I was missing some important features:
public Entity getEntity( EntityId entityId, Class ⌠componentClasses )
and
public Set findEntities( ComponentFilter filter, Class ⌠componentClasses )
This means things become more complicated.
-I can no longer only link to the EntitySet for all Entitys to get the EntitySystem
->Because I do not use the benefit of a direct access to the entity system atm I think about switching back to singleton
To implement the Set getEntities(ComponentFilter filter) method for a Map is easy, but this does not work for SQL.
This means I have 3 options:
-Special SQLFilter who injects additional stuff into the queries (dangerous but fast)
-Store everything always in HashMaps and only create a backup in SQL
-Create all Components and then check them with the Filter (very slow)
Hm, when I read my own article again it always sounds that I only have problems.
Thats not true but I like to write down my thoughts because I can deal with the subject while I write things down.
@ogerlord said:
Today I tried to implement Normens Inventory and noticed that I was missing some important features:
public Entity getEntity( EntityId entityId, Class ... componentClasses )
and
public Set findEntities( ComponentFilter filter, Class ... componentClasses )
This means things become more complicated.
-I can no longer only link to the EntitySet for all Entitys to get the EntitySystem
->Because I do not use the benefit of a direct access to the entity system atm I think about switching back to singleton
To implement the Set getEntities(ComponentFilter filter) method for a Map is easy, but this does not work for SQL.
This means I have 3 options:
-Special SQLFilter who injects additional stuff into the queries (dangerous but fast)
-Store everything always in HashMaps and only create a backup in SQL
-Create all Components and then check them with the Filter (very slow)
Hm, when I read my own article again it always sounds that I only have problems.
Thats not true but I like to write down my thoughts because I can deal with the subject while I write things down.
You do not need a special SQLFilter. I already explained how I do it but you will see soon when I post the code later.
