JME3(INOFFICIAL) network system

THIS IS NOT A GENERIC NETWORKCODE, THIS IS KINDA SPECIFIC, AND SIMILAR TO THE SOURCE ENGINE NETWORK SYSTEM!!



Well, I'm around 2/3 done with a JME3 network synchronisation system.



There are still some empty functions that needed to be filled with code,

The Serverside player calss needs pseudophysic added,

The Physicsystem from JME3 needs to be properly integgrated for the serverside collision detection and physic calculation.





I think it's however time to show some simple examples and aska bout opinions:

Basically ith as the SEntity and the CEntity class, wich represents the Serverside and the Clientside part of each given object.



A Simple static world entity:


public class SSacredPlace extends SEntity {
   public SSacredPlace(){
      this.SetAlwaysTransmitted(false);
      this.SetName("Test");
      this.SetArmor(-1);
                 this.SetTransmitRange(500f);
      this.SetCollisionType(CollisionTypeEnum.Physics);
      this.SetHealth(-1);
      this.SetMoveType(MoveTypeEnum.None);
      this.SetModel("place");
      this.SetConstantUpdate(false);
                            this.Activate();
   }
}


This creates a Serverside entity:
with the model place.meshxml
infinite armor,health (no damage)
physical collisione (loaded from file place_phys.meshxml)
static, no movement
Tells it that it is not always needed to be transfered,
setting the distance it exists on the clients to 500 jme units (if nearer server sends object data, if further it's removed on client)
Setting it to a reliable update only on change events mode (other option is to have updates every tick but unreliable aka player movement,ect)

Clientside must exist a Class with the exact same name but first letter replaced witha  C , so CSacredPlace then, this can be overriden in the networkconstructor creation, the player class does this to distigueish between  the own player and other:


   @Override
   protected String getClientName(String string,boolean isOwner) {
      if(isOwner){
         return "de.empirephoenix.entitys.player.LPlayer";
      }
      return "de.empirephoenix.entitys.player.CPlayer";
   }





It is very simpel due to the class based concept to create classes doing various stuff.
All data is serilaized in a own system directly to a bytearray wich then is send via UDP, all Message objects implementing the Reliable interface, are automatically resent, untill the recival is confirmed, however most won't need to work so low in the system.


package de.empirephoenix.messages;

import java.io.DataInput;
import java.io.DataOutputStream;
import java.io.IOException;

import com.jme3.math.Vector3f;

import de.empirephoenix.network.shared.BinarySerializeable;

public class Vector3fWrapper implements BinarySerializeable {
   private Vector3f vector;

   public Vector3fWrapper() {
      vector = new Vector3f();
   }
   
   public Vector3fWrapper(Vector3f vector){
      this.vector = vector;
   }
   
   @Override
   public BinarySerializeable newObject() {
      return new Vector3fWrapper();
   }

   @Override
   public void readData(DataInput readfrom) throws IOException {
      vector.x = readfrom.readFloat();
      vector.y = readfrom.readFloat();
      vector.z = readfrom.readFloat();
   }

   @Override
   public void writeData(DataOutputStream writeto) throws IOException {
      writeto.writeFloat(vector.x);
      writeto.writeFloat(vector.y);
      writeto.writeFloat(vector.z);
   }

   public Vector3f getVector(){
      return vector;
   }
}



The serverside player, handles everything in terms ov transmit range, updating,removing,adding entitys.
The CEntity, handles clientside model loading ina  background thread, making sure that it will not lag the game (like tons of shitte free mmorpgs do when models are loaded, or the source engine ,ect the only engine I know that is capable of this is the UT3 engine so far)


Then SEntity has methots for the most default settings, wich are all autoamtically synchronized:
SetModel
DropToFloor
SetName
SetHealth
FreeLineOfSightTo(Vector3f target)
EmitSound
SetColor(ColorRGBA color)
GetForward
GetRight
GetUp
SetLocalPosition
SetOwner
SetParent
GibClient
GibServer
SetArmor
IsOnGround
IsVehicle
IsUseable
IsPlayer
IsValid
IsWorld
LocalToWorld
Remove
SetLocalRotation
SetAnimation
SetMoveType
SetCollisionType
SetVelocity
WorldToLocal
SetAlwaysTransmitted
SetScale
SetTransmitRange
SetConstantUpdate
SetDatabaseId
SetDatabaseConfigValue //onyl a placeholder you need to implement this yoursel fif you need this
Collided


I hope to present a demo in the next two weeks, but wanted to show the whole concept already, so if you have any questions, fell free to ask them :) I hope after this library is done and working that the whole network system can be integrated in the jme3 source kinda like the jbullet system is.

Hm i will put the code into a seperate addon this evening , so everyone can take a look at it.



Edit: Ok I'm not going to make it today, but i will the next few

Awesome job mate! I heard a little about this some time ago in the team's mailing list and I've been eagerly awaiting more news ever since :slight_smile:



Q: I suppose it's not #1 on your things-i-wanna-do list, but I really think a comparison with JGN would be beneficial. Afaik it's the networking system most widely used together with jME, and it also stands a chance at being integrated with jME3 (not that it's strongly considered, or even properly evaluated as of yet). Fact is, the question will arise, same way newcomers get confused about ogrexml vs collada/md5 etc…



Q: Is this project anything else to you besides a prospective jME3 network system? I mean, are you also planning to make a game on top of it for instance?



I don't know what answer I want here tbh :slight_smile: I'm just trying to figure out where you stand right now, and where you and the project might be in a year from now.

Wella ctually tihs is a side project that is part of my game , but I only find it fair to release it for others as well.



well it is simpler, offering only the options most games need, and brings a integrated authentification system( works over md5), also ther are already base classes with handles mos stuff autoamtically, so for a simple shooter, you only need to use a SEntity, set the values correct, give each player a weapon and bind it to the input, and you are done.



The serialsation works more direct, by converting the classes directly into bytearray, at the cost of a little more coding, but at the other side, making sure that everything is sended efficiently. , eg a Vector3f is: 4byte int identifiyer + 3*4byte floats. I don't think that there is a more efficient way to send data. Antoher thing (not sure how jgn does this) is, that it uses internally a second thread, wich i used for reciving messages, and deserializing them back into a object, then adding them to a recivedmessage cache, however only when update is called, messages are processed by the Mesagelisteners, making it easy to keep in the opengl thread (just call Update() only there)



Basically I stared writing it, because I don#t like all the stuff you need to do to use JGN, with ym system it is basically:



server = new Server(new InetSocketAddress("localhost",1000),new MysqlLoginAuthentificator()); //uses the myseql login system perfect for mmorpgs for example, can of course instead use one allways allowing(shooter) or whatever you can think of.

client = new Client(); (uses random free port by default, can be modified to use a fixed one, also Update needs to be called)

then all you need for a basic connection is:

client.connect(new InetSocketAddress("localhost",1000));



If you want it to do then a bit mroe you need to add Listeners:

For spawing a player :


public class SPlayerConnectManager implements ClientMessageListener {
   @Override
   public void Connected(ClientRepresentation client) {
      SPlayer player = new SPlayer(client);
      player.Activate();
   }

   @Override
   public void DisConnected(ClientRepresentation client) {
      SPlayer ply = SSolarSystem.GetPlayerByConnection(client);
      if(ply != null){
         ply.Remove();
      }
   }

   @Override
   public void messageRecived(ClientMessage recived) {
   }
}



Connect is only called, when the Client succesfully passes the Authentificator.
Disconnect, when the Client manually Disconnects, or times out.
messageRecived is self explaining I guess.

The classtable is jsut a simple Translator actually:
Giving every object, that can be Sended a short as an identifier, and is able to genreate a empty new object from a given id:


public BinarySerializeable getObjectFromId(short id) {}
public short getObjectId(BinarySerializeable toid) {}


For the basic JMe and primites ther are already preconfigured Classtables.

So the main difference is: JGN is a complex,multifunctional network library, capable of sending Messageobjects
My system is a simple Server-Client network framework, brinign all the basic classes to build a neworked game without much work, and capable of sending Messages, and synchronizing objects.

Uh, why do you have such specific things like armor in the base library? Imho there should be some kind of datatype or the SEntity is serializable so the users subclasses can have methods like that. It might also be better to serialize on that level than on the level of Vector3f etc. Like JGN, where you simply set (any) value in your entity when your data packet arrives. Or why do you do that? Are some values only transferred at certain times? I think putting the logic into what needs to be transferred when should be done by the user… But maybe that approach of wrapping single objects is not too bad for that…

Mabye it would be possible to have certain datatypes that have a “level” and the user implements one of those interfaces to define what message is how important (like position updates every 50ms, objective log update every 1000ms or something…)



What I dont understand is why do the classes have string names and the naming convention for the classes “S…” and “C…”, what is that for? Is it used via reflection somehow? Also I find those capital letter methods a bit scary, what is that all about?



So unless this becomes a more generalized implementation I think it will be hard to add it as “standard” networking to jme3…

A very basic network stack like kryonet that can be easily extended to something like this would right now be the best for jme I think…



Cheers,

Normen

Well about the higher level i might agree, it is kinda specific (so intigrating it directly into the core might not be the best idea, we can have it however as a seperate svn folder, to keep it near the engine), this network system is not thought to be the best oo oriented approach for everything, instead it is thought to give around the same simplicity that the source engine gives you, by having a kinda large base entity

Yes the different values are transmitted depending on type, and flags set, however if you only need the function in the base entity, you never need to tough the network layer, else only a very small part of it.

So position might be only on change event, always, or forced only (even if the value/event is declared as alwaystransmit) on change(like for static objects), with the idea that usually there are many objects only changin a few values always, (like a player will probably nearly always move,and rotate, but far less his health will change or his model) It mgiht be possible over annotion to make this a bit nicer, but I havent thought about that so far.



When on always transmit it will transmit unreliable every Update call on the Server world (Ticktime)

else it will transmit reliable only on change.



The reson for the manual serializing is issues that for example JGn has with compley objects, while in my sytem there can be used a subprotocl for every object



   @Override
   public void readData(DataInput readfrom) throws IOException {
      networkid = readfrom.readInt();
      short count = readfrom.readShort();
      while(count > 0){
         short key = readfrom.readShort();
         BinarySerializeable object = Client.getObjectFromId(readfrom.readShort());
         object.readData(readfrom);
         storage.put(key,object);
         count--;
      }
   }

   @Override
   public void writeData(DataOutputStream writeto) throws IOException {
      writeto.writeInt(networkid);
      writeto.writeShort(storage.size());
      for (short key:storage.keySet()){
         writeto.writeShort(key);
         BinarySerializeable object = storage.get(key);
         assert (object != null):"Failture in stored value";
         int objectid = Server.getClassTableId(object);
         
         writeto.writeShort(objectid);
         object.writeData(writeto);
      }
   }


Yay we can serialize HashMaps, something JGN jsut dies silently away without an error message (cause last time i checked only primitives were guranteed to work had issus to send a ArrayList back then)

Does your system use NIO? Both TCP and UDP are supported?

To be honest, I like JGN more than this. The requirement to have different classes for server and client is kind of strange as well. The capital letter for the method names also isn't Java coding guidelines compliant.


My system is a simple Server-Client network framework, brinign all the basic classes to build a neworked game without much work, and capable of sending Messages, and synchronizing objects.

JGN is capable of the same thing. To me JGN seems more straightforward and simpler than this. You have a client and server class, you can have objects and you can synchronize them. You can also send messages.
JGN has a lot of packages, but each package has only little classes.
The only issue I have with JGN is that it's essentially unmaintained, however I assume if we include it with jME3 then other people as well as us would be able to maintain it.

The requirement to have different classes for server and client is kind of strange as well. ??

Not really, as the client side nromally does completly other stuff than the serverside, seems for me absolutly logicall.

(Other engies work the same way http://developer.valvesoftware.com/wiki/Networking_Entities)

Also i should mention there is a shared baseenetity, supplying all the getters, as they are the same on client and server.



However if you prefer JGN i won't care, after all this is just something I need for my own game ^^, about the capital letters, yes i agree, need to fix that when i have time, but that is not really a problem to change.

My system strictly uses DatagramSockets and thats it.



Internally there are two layers,

onw only for the sending and reciving Messages, wich could be used for nearly everything, and then the one above that that actually is the bridge to jme and offers the synchronizing.

Empire Phoenix said:


The serialsation works more direct, by converting the classes directly into bytearray, at the cost of a little more coding, but at the other side, making sure that everything is sended efficiently. , eg a Vector3f is: 4byte int identifiyer + 3*4byte floats. I don't think that there is a more efficient way to send data. Antoher thing (not sure how jgn does this) is, that it uses internally a second thread, wich i used for reciving messages, and deserializing them back into a object, then adding them to a recivedmessage cache, however only when update is called, messages are processed by the Mesagelisteners, making it easy to keep in the opengl thread (just call Update() only there)


I really like this solution. Funny thing is, we started to make our own networking framework in C++ and we came up with similar solution with byte[] cause we thought that way the least amount of information will be sent. I had only used JGN a bit before, didn't really understand it's serializing. But what I understood was that it serializes entire class and this way each object sent over the network takes more memory than with our approach. What we are doing is like so: first 2 bytes for the indentifier (cause I thought we wont be using that many objects), based on 2 bytes with

switch(identifier){
      case 1: do something like read player id and 12 bytes for that players location
                  break;
etc.

Empire Phoenix said:

server = new Server(new InetSocketAddress("localhost",1000),new MysqlLoginAuthentificator()); //uses the myseql login system perfect for mmorpgs for example, can of course instead use one allways allowing(shooter) or whatever you can think of.
client = new Client(); (uses random free port by default, can be modified to use a fixed one, also Update needs to be called)
then all you need for a basic connection is:
client.connect(new InetSocketAddress("localhost",1000));


I like this built in mysql login system also. I'm considering starting to use your networking framework instead of our custom C++ networking framework that is not eaven 1/3 complete. Since our client is actually written in Java for Android. But we just thought that C++ server would be faster. And we also using only UDP.

I don't agree with your way of naming methods with first capital letter, but that doesent bother me much.

And You offered some examples if we ask. I'll pm you.

How are you handling disconnection if you're using datagrams? O,o…



I like most of your ideas though :wink: And yeah as stated - your naming conventions are horrible :stuck_out_tongue:


  • Mikkel

Ever ytwo seconds a keepalive packet ping is send, wich must be responed by a answer. If it is not answerd for X seconds the client is dropped, and a disconnect event is invoced. If a normal disconnect happens, the client sends  a disconnect notify to the server, to speed this up.



The naming conventions, are currently a bit better already, as I rename the methods accordingly to their real names.

Also on a simple test, my netbook was able to handle 600 continually moving objects without a problem. (After that ram colapsed ^^, eclipse needs to much)

Empire Phoenix,



while you seem to developing quite a nice networking system, wouldnt you agree that it has a certain level of tailored specificness towards your own project which makes it bulky to someone else who would be attempting to integrate into their own project?



I know I am not on the development team, so my opinion here may be quite unwarranted, however, wouldn't it be simpler for JGN to be upgraded to coincide with JME3, seeing as it has been built from the ground up not only as a networking system, but also a networking system which is JME ported?

I just feel that JGN's simple and straightforward approach for combining with JME2 would make it an excellent candidate to work with JME3.



No offence to you Empire Phoenix whatsoever, I am simply speaking from my bias of being a JGN fan, having worked with it some in hobby development.

Empire Phoenix said:

Ever ytwo seconds a keepalive packet ping is send, wich must be responed by a answer. If it is not answerd for X seconds the client is dropped, and a disconnect event is invoced. If a normal disconnect happens, the client sends  a disconnect notify to the server, to speed this up.

The naming conventions, are currently a bit better already, as I rename the methods accordingly to their real names.
Also on a simple test, my netbook was able to handle 600 continually moving objects without a problem. (After that ram colapsed ^^, eclipse needs to much)


Ah yeah, that's what I thought you would do :p

Thank you - I had no clue on UDP before I came to this thread, and its quite interesting tbh. :)

Can't wait to see the final results - in the mean time I'll fun around with my own server for the sake of learning. :D

- Mikkel.
Eggsworth said:
I know I am not on the development team, so my opinion here may be quite unwarranted, however, wouldn't it be simpler for JGN to be upgraded to coincide with JME3, seeing as it has been built from the ground up not only as a networking system, but also a networking system which is JME ported?
Your opinion is definitely not unwarranted, but it might be(come) misplaced. I feel, by now, this thread should focus on nothing but furthering the development of Empire Phoenix' work. However, until the topic name is changed to something less indicative of JME3-specific code, the discussion is bound to take different directions.

JGN is definitely still being considered, and is probably our strongest bet to date. Bottom line is I don't think networking is going to be a major priority for the current developers any time soon. If someone takes the time to actually do all the work and provide a network system completely integrated with JME3, that'd be a major contribution. Even if it didn't make it into the JME3 core it would be the default JME3 networking system so long as there's no other integration readily available.
wouldn't it be simpler for JGN to be upgraded to coincide with JME3, seeing as it has been built from the ground up not only as a networking system, but also a networking system which is JME ported?

The JGN "integration" to jME consists of a single class with a single page of code. There is a small test as well but that is it.
JGN is a general purpose networking library, not designed to work with jME but with any game (myself I was able to use it with the "JGame" game engine quite successfully).

@Empire Phoenix: If you're willing to put the work needed to make this into a general-purpose networking engine for jME3 then we can integrate it into core, but until then we can't do much. Like was mentioned already, the requirements with the class names, the serialization, and the game-specific code can't stay if this is to be used in jME3.

Well cahnged title, and does not need to be in the code, actually the naming issus are already mostly corrected, however it will not be more generic, as i don't want to waste bandwidth and efficiency for such stuff, as it is mostly a side product for my own game.

Empire Phoenix said:

Well cahnged title, and does not need to be in the code, actually the naming issus are already mostly corrected, however it will not be more generic, as i don't want to waste bandwidth and efficiency for such stuff, as it is mostly a side product for my own game.
That is exactly why it is more of a "Java FPS network system" and not so much related to JME3 at all until there's an effortless way for new JME3 adopters to make use of your system. I still think the title is somewhat misleading tbh.

Edit: Well, after all you are working with JME3, so I suppose it will actually be tied up to JME3 pretty well.

Well i finally had some time to make some progress, (do never underestiate the amount of time a university needs)



Here is a video showing a network synchronized short vehicle drive,

-currently there is no prediction/interpolation on the client

-there are by total 10 networked entitys in different transmit modes

-all physic calculation is done on the server wich si around 200km away from me

-The networked entitys:

4 wheels, unreliablemode, constantupdate

1 vehicle, unreliablemode, constantupdate

3 blue lights, reliable, updateonchange

1 white light, reliable updateonchange

1 terrain01 entity, reliable, updatenever

1 player(the red box you see short) controlling the vehicle over a playerinterface wich was attached to it when he used the vehicle

total networkload with updats every 50ms 6k/s, networkcode is able to slowdown updates with factors of two if bandwidth exceeds the limit the client gave upon connecting



Su navegador no soporta objetos flash





@erlend

This here is now the second level, wich controlls logic, this one is really hard attached to Jme3 as it uses it’s features directly.

Empire Phoenix said:
@erlend
This here is now the second level, wich controlls logic, this one is really hard attached to Jme3 as it uses it's features directly.
That's all I asked for, thank you and congrats :)

Where's your code at? It'd be nice being able to keep an eye on your progress from time to time, like when we feel it's time to re-assess your work.