Jadex experiences

This thread is supposed to collect practical experience reports with Jadex.
Advocacy is off-topic, discussing workarounds for reported problems is on-topic, discussion of alternatives to Jadex should be kept to the minimum; the idea is to make this thread into a useful resource to turn to when deciding whether Jadex is suitable for a given task.

The LGPL licensing is a bit of an issue.

tl;dr: The LGPL restricts packaging and deployment options, imposes several obligations that mean more coding, and doesn’t help the end user (but may hurt the game experience).
This applies only to use of Jadex in a closed-source game; open-source games are unaffected.

Full details:

The LGPL section 6 mandates that whenever I use it in a closed-source game, I must

  • “prominently” carry a notice that the game uses this library,
  • deliver the library sources with the code, and
  • allow people to plop in a modified version of the library into the program.

The first point isn’t much of an issue, an About dialog would suffice. I’ll have to make sure that that dialog is available in the first version of the game that I release for beta testing, which is earlier than I’d have liked, but it’s not really a problem.

The second point (deliver sources) can be dealt with by offering a Jadex download from the same place where the game can be downloaded from, so it’s dealt with the same way as the first point.

The third point is a real burden.
I can deliver a binary blob that has the library in it, but I must make sure that users can rip out the library and replace it with their own code. (Section 6a) This applies to deployment methods that merge all code into a single Über-jar (e.g. one-jar, Maven shade, etc.); to the very least, I’d have to document the alternate java -jar command line that would make the replacement jar override the Jadex inside the Über-jar. I still need to make sure that any obfuscation in use does not obfuscate the interface between Jadex and the rest of the program, else the replacement won’t work.
Alternatively, I can keep the library as a separate jar. (Section 6b) User can simply replace the jar.
Options 6c, 6d and 6e essentially have the same consequence as section 6a; they have a slightly ambiguous wording and could be construed to drop the relinkability requirement, but that would be legally risky, and I currently think it would violate the spirit of the license anyway.

Technically, it’s all a solvable problem, though it does require some care.
However, it also makes it easier to cheat. Cheater simply replace the Jadex jar with one that “just happens” to select NPC actions that are favorable to the player. And that’s not good.

Cheaters gonna cheat. Nothing you can do client side to stop them. (In spirit of staying on topic, if anyone wants to debate that assertion then feel free to start another thread. I assure you it is correct though. It is impossible to stop a determined cheat from modifying their own client - all you can do is make it a bit harder).

If this is a single player game it really doesn’t matter.

If it’s multiplayer then if one client can modify AI behaviour you have an architectural security design flaw anyway.

For distribution just include it as a separate jar - I certain wouldn’t do anything more complicated than that and things like runtime switches etc are way overkill.

I realized that the cheating problem is less important than I initially thougth while writing the post :slight_smile: (I probably should have left out the final “this is bad” sentence, it’s overstating the issue.)
Still, it’s a restriction on design space, and while it’s certainly not important for everyone, I suppose one should at consider the issue before subscribing to Jadex.

Please remember: We have lots of Simulations they look a bit like a Game, but the main Focus from Jadex is not Game-Developing. I just want to show with my Project that it can handle games very well too. Thats one of the reason why I implemented the 3d Support and now even developing a playable Game.
But its funny to follow the process what kind of new Aspects that creates. I think none of the Core-Developer ever thought about Cheating-Possible-Problems :slight_smile:

Some code reading showed me TCP configuration but I couldn’t find references to UDP.
Does Jadex support UDP?
If no: Is its TCP usage factored out into a transport layer that one could hack an UDP transport up for?
Does the Jadex project have interest in such a transport? If no, would it be willing to accept contributions for that?

Background:
UDP is important because for any real-time application, lag avoidance is far more important than reliability for most network transfers. Most data will be state updates that are going to be overwritten by the next update anyway, and within a few milliseconds. Transfers that still need to be reliable (player commands) usually have reliability established by explicit coding inside the application.
Without UDP, Jadex cannot use distributed agents on the client side, and even load balancing between servers would require special considerations. Running a bunch of agents within the same machine would not be any problem, of course.

Its true what you say but I wouldn’t make it a premise. Many game devs found that in reality TCP performs just fine for a lot of game types.

All this holds true for single player games, but for multiplayer games(MMO ones to be specific) where you provide SaaS, you dont have to do any of that because the code going is going to be “used” by you and the library will not be available to the end user. More specifically the behaviour is going to be simulated on a closed system and only positional updates going to be propagated to the end user.

Also to point something, out, UDP solutions are much more complex, and only used when you need real time RESPONSE. In the case of MMO’s and similar games where you dont need to say shoot your buddy in the foot and have him instantly screaming back, you could potentially use TCP to support transactional type games, where a response lag of up to 300ms is acceptible in combination with dead reconing. Using both however is out of the questions because sending large amount of UDP traffic will affect all TCP traffic in a very negative way. In fact most games dont use UDP contrary to the common belief.
However to really use TCP you will need to make quite a few optimizations that are very low level and not very well known to most, and are very platform specific(eg. Windows vs Linux vs Mac, dont know about android).

PS:
Its much easier to develop for TCP since the support for it is universal, and requires very little configuration as far as firewall and NAT rules are concerned, where UDP is very low level and usually difficult for people to use(games such as splinter cell chaos theory make that very evident, they are absolutely horrible user experience), so people that are on a college campus or coffee shops wont be able to play, most of the time (UDP filtering is a pain in the ass).

PSPS:
More recent networks, now days, with more recent TCP specs, have a few aditional customizations(i forgot what they are called) which let you effectively not use partiuclar parts of the protocol that you dont want, which means you no longer need to build up UDP but can instead strip TCP down.

Okay, let me rephrase this: For those games that need/want UDP, is that an option in Jadex?

Ok, so i am not sure, i googled that a bit and it seems that they use UDP only for host discovery,
on the other note this is a jmonkey forum so wouldnt it be better to ask it on the jadex forum?

Also another note, why would it matter what jadex uses since that networking bit seems to imply that its used internally to distribute the code across several nodes, rather than interact with whatever client you built for your game, because the site even says that the networking stuff is meant for local networks(or at least host discovery???).

Anyways here is the description for their networking implementation which implies the networking is meant to be internal to your application, and maybe over the internet in a vpn type style.

overlay.png Overlay Network

In order to support distributed applications it is a necessary prerequisite that platforms on different nodes get to know of each other. Using Jadex the nodes will automatically discover each other using the platform awareness mechanism. Awareness will work in local intranets as well as in networks with internet access. Jadex provides an extensible set of discovery techniques that are used to find other hosts. Of course, it can be customized if awareness should be used and if so which discovery techniques are active. The awareness mechanim leads to an automatic overlay network formation of platforms that can reach each other. From a component perspective, the current overlay network structure is largely hidden as components just use required services of service providers located in a logical scope.

I’m thinking about the design options that you have to deal with unreliable packets. The ones that I can think of right now are these:

  1. (Re)design the game so that an occasional delay of 500-1000 ms doesn’t matter. Turn-based games, for example, couldn’t care less, and TCP is fine.
  2. (Re)design the game so that you don’t need to split the agent network across unreliable network connections. I.e. do not have any agents on the client side, or if you do, don’t connect them across the Internet, instead, synchronize agent state via UDP (a nifty technique for that was implemented in Quake 3).
  3. Build a UDP transport layer for Jadex (i.e. the question I raised above - I now see it’s part of a larger set of design choices). Piggyback acks on normal messages, have the transport resend messages at regular intervals until they get acked.
    3a) Go the extra mile and distinguish overridable and one-shot messages. Overridables would be messages like position updates, which would get overridden by the next position update even if not acked; one-shots are those that must be processed exactly once, e.g. buying something. The link above says that this can make the code quite fragile because it’s easy to get the dependencies between overridables and and one-shots wrong; it would be interesting to see how this issue is affected within the limits of Jadex’ messaging model (unfortunately, I won’t find the time to explore that).

TCP is much faster than 500-1000 ms, when did you last try anything with tcp? Nowdays if a whole webpage takes longer than 300 ms to load you have to optimize it. Also games like WOW and pretty much any other mmo is built on tcp, heck every facebook game is tcp. The game does not have to use lockstep(like RTS ones) but certain lag compensation algorithms have to be applied either way, also you will need to look into something called TCP tuning and here is an article that relates that to java a bit, ofcourse most tuning occurs on the OS level anyways.

As long as the game is not twitch based boom headshot type of game this sort of thing provides awesome RTT, as signified by this paper

4.1.6 Round Trip Time (RTT) Round Trip Time (RTT) is analyzed to characterize the server- client connections. Due to the large volume of RTT data, we sample three periods of the total trace, each of which belongs to Thursday 13:00~16:00, Saturday 20:00~22:00, and Sunday 15:00~17:00. As there is no significant difference in those three data sets, we calculated the average of all RTT data in those data sets. The average value of RTTs of these three periods is about 126 milliseconds and Figures 13 shows the distribution of RTTs. Peak points stand at 0 millisecond and around 200 millisecond, and 90 percent of RTTs are distributed less than 200 milliseconds. Those peak points are related to TCP’s delayed ACK mechanism implemented in the Windows TCP/IP stack. ‘Lineage II’ is running on Windows operating system that adopts the delayed ACK feature [8]. In Figure 13, about 11 percent of analyzed RTTs are less than 1 millisecond and about 15 percent fall within 200±1 milliseconds. Note that the default delayed ACK value in Windows TCP protocol stack is 200 milliseconds. That is, in normal cases, about 11 percent of sent packets from the server are acknowledged almost instantly but if the client has no data packet for piggybacking the ACK, it waits until delayed ACK timer expires (200 milliseconds). If shorter RTT is needed for game’s real time interactivity, then the ‘Lineage II’ client software may have to turn off the delayed ACK feature.
taken from this http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&ved=0CE0QFjAD&url=http%3A%2F%2Fwww.research.ibm.com%2Fnetgames2005%2Fpapers%2Fkim.pdf&ei=n_YLUeraCfPlyAH4zoEQ&usg=AFQjCNEPR2bjSK2jtpdAyPbLzfEjaiMzsg&sig2=oDXniRxTRZu1i_KOupNMpQ&bvm=bv.41867550,d.aWc

and this was back in 2005, there are much more advanced techniques in the latest TCP RFC.
Bottom line what you said may have been true in early 2000’s but the landscape has change a lot lately, and twitch response games if implemented carefully might actually be viable for this, since 126 ms RTT delay on average is what you would have seen in earlier counter strike source as far as latency goes.

Many games use TCP effectively but many also use more than one channel (which is why SpiderMonkey supports this as even Mythruna uses more than one TCP channel). The main problems TCP is that message ordering is forced and reliable. So one bad packet can stall your whole pipe… and if that message wasn’t important then you’ve stalled the pipe (for as much as 1 second or more) unnecessarily. Using more than one channel can help make sure that the really important packets have a way of going around other blockages… though they can still get blocked.

In FPS games where positional state is king, TCP is just intractable. Any stalled data is redundant as soon as you start receiving it and you generally don’t care about packet ordering as only the latest packet is interesting.

Mythruna uses UDP for object positional state and player input state… and several channels of TCP for everything else.

Implementing your own reliability algorithm on top of UDP is a significant undertaking. You have to examine whether or not it’s worth it. In the new networking code for Mythruna, I implement a half-reliable protocol on top of UDP for object state as a way of reducing message size (I can delta known good state). It still allows packets to be dropped and ignored but sends some bits over and over until it knows they were received or completely stale. The code that handles that is more than half of all of my networking code and took a significant amount of time to test… and probably still has 100 bugs. I don’t recommend it unless you really need it.

This is getting too far off-topic.
All I say is that without UDP support, Jadex won’t be useful for games that use UDP. Anything else really belongs to another topic.