Hi everyone,
I’m trying to make some data visualisation tool using jME. The tool should be able to receive data through networking and display a real-time (scene)graph on screen, which you can travel through and where you can point-and-click an element to get more information about it.
Data
What data
Here’s a description of the data:
- 800 elements. Let’s call them “nodes” (about five floats, three strings, and a long - could be truncated to several longs if necessary)
- each with 255 references to another element. Let’s call them “connections” (about two longs, a float, and a string)
- also, all nodes have a separate culled hit box to aid point-and-click.
- that’s a total of 204000 connections + 2 * 800 elements = 205600 things to draw.
It should preferably display everything.
I already approached this problem twice. In the last iteration I used Zay-Es because it has a simple to use way to detect changes and I did not yet know how much data I was displaying.
Input
I already have pretty solid networking, so that isn’t a problem.
However, this is a ranked list of what data updates frequently:
really fast
- color
- size
- position
- hitbox size (might as well be constant)
really slow
My main problems are:
How should I store so much data in Java?
I think there are three options:
- Use Zay-Es
Positives: easy change detection; I already have most of the code.
Negatives: Uuhhhh that’d be at least 206k Java objects and at most 822k Java objects. I don’t know if Java is happy with that. - Use big arrays:
Positives: no overhead - might as well do it in C.
Negatives: speaks for itself really - Use Lists:
Positives: It’s a compromise between arrays and ES, which means it doesn’t use 822k objects, but it’s also friendlier with memory allocation than pure arrays.
Negatives: I need to write some sort of change detection myself to optimally update buffers and what not.
I just don’t know what would be a smart way to do this, as I never had to use this much data.
How do I visualize 206k data efficiently with jME?
I tried to represent connections with debug arrows and elements with spheres, but that gives a total of 206k draw calls (which doesn’t sound good). So I tried batching it, but the batching thing would freeze completely after more than ~150k objects.
That basically means I need to create a custom mesh, which isn’t that bad.
The last two options are:
- Use the buffers in the custom mesh to pass ‘real’ data to a custom shader.
Positives: I don’t need to pre-process data on cpu, and I can truncate the original data slightly, because it has to be less specific.
Negatives: I’d have to use buffers like TexCoords or Size or something, which seems like a hacky way to send data to a shader. - Use the buffers in the custom mesh to pass ‘correct’ data to unshaded mat.
Positives: I don’t have to write glsl.
Negatives: Whenever data is received, it needs to get pre-processed into the correct attributes (i.e. color, size), and that’d happen on cpu.
What is the best way to do this? I honestly don’t care about development time - only about performance.
p.s.: Also, I’d like to have, like, 30-60 fps, because I should be able to move the camera.
