Collection inside a network message

Troubleshooting networking
1

Greetings,
I got a wierd problem that I seem to fail to solve myself or find any info about this particular one on the net (jmonkey forum included but please feel free to ‘let me google it for you’).

I have got a HashMap in a message. Tested it and confurmed that it is being populated correctly with the data I need to send over. PlayerDef and PlanetDef are simple structures that are marked with “@Serializable” and registered accordingly.

[java]
@Serializable
public class SceneGraphUpdate extends AbstractMessage {

public final HashMap<PlayerDef, ArrayList> content;

public SceneGraphUpdate() {
    this.content = new HashMap<PlayerDef, ArrayList<PlanetDef>>();
}

public SceneGraphUpdate(PlayersNode players) {
    this.content = new HashMap<PlayerDef, ArrayList<PlanetDef>>();
    this.construct(players);   // <--- this.content is populated with data found in players node here
}

[/java]
[java]
@Serializable
public static class PlayerDef {
public String name;
public ColorRGBA color;

    public PlayerDef() {
        name = "";
        color = null;
    }

    public PlayerDef(String name, ColorRGBA color) {
        this.name = name;
        this.color = color;
    }
}

[/java]

Everything works fine so far, server sends the message and it is received by client.
[java]
@Override
public void processMessageQueue(HostedConnection app, Message msg) {
if (msg instanceof SceneGraphUpdate) {
SceneGraphUpdate m =
new SceneGraphUpdate(this.getApp().getWorld().getPlayersNode()); // <— tested PlayersNode does have data for the SceneGraphUpdate to work with
this.getApp().getNetwork().broadcast(m);
}
}[/java]

BUT here is the problem… HashMap that was full of data before server broadcasted it now holds and empty HashMap.
[java]
@Override
public void processMessageQueue(Client app, Message msg) {
if (msg instanceof SceneGraphUpdate) {
SceneGraphUpdate m = (SceneGraphUpdate) msg;
WorldSynchronizer.sync(this.getApp(), m.content); // <— m.content is now empty
this.getApp().getWorld().startSimulation();
}
}
[/java]

Same thing happens when I tried sending a mock ArrayList, simple array though arrives with the data it stored before the transmission. At the moment it seems that I have looked everywhere I can imagine and absolutely lost. Any info would be much appreaciated at this stage (just a little over a week left for the assignment due date and still plenty of work to complete).

Thank you for all your help. :slight_smile:

2

I can confirm that if you serialize a collection using standard java serialization and send it over the wire at the other end you get the collection and its contents.

All I can suggest is drilling down into it and really instrumenting exactly what you are sending and what you are receiving. Also make sure you catch and log exceptions at every point.

3

Is the hashmap completely empty? Or just keys with empty lists? Were there any errors logged?

4

Thank you for the prompt reply :slight_smile:

<cite>@zarch said:</cite> Also make sure you catch and log exceptions at every point.

Not quite sure what points you refere to. SceneGraphUpdate message life span is limited by the period of time, creating and broadcasting it by server and receiving and reading by client. There is nothing in between those points in the app that throw an exception, or I am missing something.

<cite>@pspeed said:</cite> Is the hashmap completely empty? Or just keys with empty lists? Were there any errors logged?

The map arrives completely emty (no keys, obviously no lists or even empty lists). Getting only INFO level messages in the console.

P.S.
First code example coppied with a slight mistake there.
HashMap<PlayerDef, ArrayList>; <- array list is missing a type parameter that is actually present in the project code
HashMap<PlayerDef, ArrayList<PlanetDef>>; <- this is how I actually declare and assign it.

5

Presumably you do something to send it over the network, read it at the far end, etc.

Check for errors at all the steps along that process.

6

The only modification that I have implemented into the messaging is storing all received messages in a Queue and processing those in the update loop. The idea is to include a timestamp into the sent messages so that I can order the queue by its time of creation.

Here is the code to make things clearer. Using a concurrent queue store an object of type Client or HostedConnection together with a Message.
[java]
public abstract class NetworkAppState<APP extends NetworkApp, MC extends MessageConnection>
extends AbstractAppState
implements MessageListener<MC> {

private String name;
private APP app;
private ConcurrentLinkedQueue&lt;MessageQueueItem&gt; msgQueue;

private class MessageQueueItem {
    public MC messageConnection;
    public Message msg;

    public MessageQueueItem(MC service, Message msg) {
        this.messageConnection = service;
        this.msg = msg;
    }
}

.
.
.

[/java]
[java]
.
.
.

@Override
public void messageReceived(MC messageConnection, Message msg) {
    this.msgQueue.add(new MessageQueueItem(messageConnection, msg));
}

[/java]

All I am doing during the update is iterating over the queue and calling the method to handle the message that you saw in my first post.
[java]
@Override
public void update(float tpf) {
super.update(tpf);
Iterator<MessageQueueItem> iterator = this.msgQueue.iterator();
while(iterator.hasNext()) {
MessageQueueItem i = iterator.next();
this.processMessageQueue(i.messageConnection, i.msg);
iterator.remove();
}
}
[/java]

Together with the code from my first post this is the entire path that message takes before getting read by the client implementation. I have gone through it line by line in the debugger numerous times by now and is completely lost (hense I have asked for help from you guys). There is no errors logged, server is constructing the hashmap correctly and I can confirm that the message gets through and arrives into client’s processMessageQueue routine, but empty of all content (except mock attributes of primitive types that I have put in there just to see what happens).

7

The hint about exceptions was that some people write code like
try {
// do something that may throw a checked exception
} catch (Exception e) { // catch it to get rid of the compiler error
// do nothing because, duh, we don’t know what to do with it
}

The proper way to deal with that is to do something with the exception.
Either rethrow it via throw new RuntimeException(e);, or, if you’re already in a strategy layer of the software that can decide whether to retry, abort, or vary a failed computation, do that (and log the problem as well, if only at TRACE level).

A collection being deserialized empty is not something that normally happens, so my first guess would indeed be that something during collection element (de)serialization fails with an exception which gets ignored.
You could also try setting an exception breakpoint, both for caught and uncaught exceptions. Enable the breakpoint immediately before entering the serialization code. Do the same on the deserialization side. That should quickly test the “swallowed exception theory”.

8
<cite>@toolforger said:</cite> The hint about exceptions was that some people write code like try { // do something that may throw a checked exception } catch (Exception e) { // catch it to get rid of the compiler error // do nothing because, duh, we don't know what to do with it }

The proper way to deal with that is to do something with the exception.
Either rethrow it via throw new RuntimeException(e);, or, if you’re already in a strategy layer of the software that can decide whether to retry, abort, or vary a failed computation, do that (and log the problem as well, if only at TRACE level).

A collection being deserialized empty is not something that normally happens, so my first guess would indeed be that something during collection element (de)serialization fails with an exception which gets ignored.
You could also try setting an exception breakpoint, both for caught and uncaught exceptions. Enable the breakpoint immediately before entering the serialization code. Do the same on the deserialization side. That should quickly test the “swallowed exception theory”.

As I recall, messages are serialized and deserialized in the background. You can’t catch these errors. They should be logged, though… and you can register an error listener.

…at any rate, serialization errors will also kill the connection.

To be honest, I’ve never tried to send a hash map over a network connection. Maybe there is a bug in that serializer. (shrug)

9
<cite>@pspeed said:</cite> To be honest, I've never tried to send a hash map over a network connection. Maybe there is a bug in that serializer. (shrug)

I would not be able to tell, but I have just found what was causing this behaviour. <span style=“text-decoration:underline;”>Collection declared as final</span>. Once I removed final from the map declaration, message arrives with all its content.

I should relly do it properly and pull complete source sode from the jMe repo to dig dipper but for now I just made a copy of MapSerializer, gave it a different name and registered it for the HashMap class. After that ran my app with and without final declaration. The result was that when using final declaration methods [java]public <T> T readObject(ByteBuffer data, Class<T> c) throws IOException[/java] and [java]public void writeObject(ByteBuffer buffer, Object object) throws IOException[/java] are never called. -_0

If you have any ideas why that happens please share, it would be interesting to know what exactly causing this behavior. The issue is solved for me with the notes going in about final collections inside network message…

Thank you all for your help, hope this conversation was not completely without its worth for me as well as for you guys.

10

Final fields are not meant to be set after the object is initialized. SpiderMonkey sets fields after the object has been initialized and so ignores final-declared fields.

Just don’t use final if you want data sent.