Have a look here: test-spotbugs/mesa.yml at 3d0f534bcffca0bc47f4be17c15796f0a1a24fe5 · MeFisto94/test-spotbugs · GitHub
How good does it support 4.5 in terms of the feature set?
You actually need to look into buildenvs/javagl at master · riccardobl/buildenvs · GitHub as well
The latest mesa release (20.2) is complete up to 4.3, but earlier it was 3.something. Next release will be up to 4.5 + most of 4.6. You should have 20.2 because of the updated mesa from oibaf/graphics-drivers in the docker file. Have you tried recently and seen if newer versions are working?
Apparently we stopped at 3.3 (not including 4.0):
I tried to see the latest version with glxinfo, but the container does not support to run as root, so I’ll keep it at that at the moment.
Is there any ETA on the new release? Because since we use this to validate/find our errors, we need a stable release so we don’t track down MESA bugs thinking they’d be ours.
@Samwise Is OpenGL 3.3 at least a bit helpful for your stuff, I guess only really partially, right?
Also we’ve been thinking/seeing a need for PBOs, is this already incooperated on your fork and/or would the work collide with your work there?
Yes it is partially: VertexArrayObjects are core since 3.0, UniformBufferObjects were added to core in 3.1, then 3.2 added SyncObjects and 3.3 added the GpuQuery types any_samples_passed, timestamp and time_elapsed.
Ofc those versions added more features, but those are the ones relevant in addition to stuff like instanced_arrays (ie that is used when using VertexBuffer.setInstanceSpan()) and more that the engine currently uses already, so regardless of my work, an upgrade to 3.3 could ofc course cover more features in the tests
I am wondering about extensions availability though, because it might not be necessary to upgrade to 4.3 to get multi draw indirect for example, as only the extension ARB_multi_draw_indirect is required, which was written against 4.1, 4.3 is just when it made it into core basically
but aside from that, this test library can ofc be used to verify that the engine still works the same it did before (like it correctly detects VAOs not to be available and thus bind buffers the usual way for example)
What exactly do you want to use it for? (sure, transfering pixels) but does it refer to :
?
EDIT: nvm I just read the WIP post, i totally see now
But unfortunately no, i did’t implement them. However in my implementation it would be a
public class PixelBuffer extends TypedBuffer
with a static factory method used like
PixelBuffer pbo = PixelBuffer.createWithSize(size);
and a method added to the UntypedBuffer if you want to create a PixelBuffer-View on it, because you also want to have the same data viewed as a buffer of another type maybe, like
PixelBuffer pbo = someUntypedBuffer.asPixelBuffer();
and would offer methods like
public void upload(Texture tex);
public void download(Texture tex);
and could make use of the SyncObjects to check when the data is actually available
EDIT: you could also map the UntypedBuffer and write data to it that way ofc, as long as you have the PixelBuffer-view on that memory you can still use it to transfer the data into a texture then
so although it is not part of my fork, adding PixelBufferObjects just as another “standalone” class like BufferObject-class would collide with my implementation in some way
Yes, especially also maybe trying every version to ensure the code has a proper fallback and especially in cases where only partial extensions might be available etc.
So it would make sense to integrate your PR first, provided we want that Buffer Abstraction, hmm.
On the other hand we might also tightly integrate this like Future<Void> setTextureAsync(Texture tex);.
My main problem is that we wanted to do the Engine Showcase and maybe have both legacy and new way, so people have some kind of migration, but it appears that we cannot really stream textures without PBOs. At least not good
Thats right we cannot stream them, however we could setup a framebuffer with a texture as render target, cut the to-be-uploaded texture into small pieces and draw a few of them each frame with corresponding offsets until they are finally all drawn into the framebuffer resulting in the complete texture on the GPU 
super ugly, waste of memory, probably a pain with the resolutions and totally doesn’t sound like what one would do, but i guess the engine would support it already
2 Likes
Speaking of that I like the render technique only rendering at a lower scale (and skipping pixels) and then using temporal information to reconstruct it, that way a higher resolution can be achieved at the cost of laggyness for fast moving scenes
I browsed trough the code a bit, is there a reason why the IndexBuffer and the VertexBuffer in com.jme.scene are not typed buffers?
@Samwise, sorry it has taken me a bit to get back to you, I got a little busy with Monkey-Netty these last couple days.
I have my test hardware setup for running these tests. These tests will be run on only NVIDIA GPUs as that is mostly what I have available (it is too much of a pain to get the AMD drivers setup and running on these test rigs as I have conflicting CUDA software on them). Are there any CPU requirements for running these tests? I plan to use dual socket Intel Xeon servers as that is what I have setup for my GPU testing right now.
What would you like me to record for the tests? Would you like screenshots of every test?
I will try to get them done over the next couple days.
EDIT:
I just ran and checked, here is the list of current GPU to servers that I have setup right now:
- GTX 1080 TI (SLI)
- GTX 1080 TI
- RTX Titan (NVLink)
- RTX Titan
- Quadro K2000D (x3)
- GTX 960
- GT 630
If you want anything other GPUs, let me know and I will see if I have them here and I can swap them out in the server.
EDIT 2: We have a bunch of older cards that we are going through to check which have opengl 3+ support. Most are old quadros, I’ll post a list of cards we find.
1 Like
Thanks for browsing
Yes there is, although it’s different reasons for IndexBuffer vs VertexBuffer:
To me it appears the IndexBuffer is a CPU only construct to unify the way of reading vertex data from the VertexBuffers. The GLRenderer for example doesn’t import it (thus doesn’t use it and it would be responsible for uploading it), its not extending NativeObject and there also is no method like mesh.setIndexBuffer(IndexBuffer), instead the index buffers you set on meshes and that are sent to the GPU are VertexBuffers of Type.Index
For the VertexBuffer however I am open for input (I am open for any input in general of couse). The original intention was simply not to break any existing code because I assume almost every project uses them, additionally I assumed a lot of projects on the jmonkeystore use it and i just wanted it to work the exact same way without having to test all existing implementations.
Since the TypedBuffer class itsself is nothing the user code should use (there is no point in writing a custom implementation as the TypedBuffer requires a Type in the constructor which comes from an enum and there already is an implementation for each of the enum values. The other way round, unifying them in the user code like List<TypedBuffer> and adding a bunch of buffers to it requires checking the Type and casting it anyway before usage due to the fact that just because QueryBuffers and ShaderStorageBuffers for example are both TypedBuffers ofc, you cannot just attach a TypedBuffer to a ComputeShader because the only buffers that can be used with ComputeShaders are UniformBuffers and ShaderStorageBuffers). So in the end it’s just a construct to simplify the implementation in the Renderer to handle all those different bufferviews, while the Renderer was already capable of handling VertexBuffers.
However this doesnt mean you cannot create VertexBuffers as views on UntypedBuffers (see https://github.com/AKasigkeit/jmonkeyengine/blob/master/jme3-examples/src/main/java/jme3test/buffers/TestMultiDrawIndirect.java#L57 for example, it creates the index buffer and vertex buffers that way)
Again I am of course open for input, because it is sure possible to make VertexBuffer extend TypedBuffer, personally I just don’t see an advantage for the user or for the engine code
Along those line I should also mention the BufferObject class that is currently in core: Unfortunately, it is a little flawed (it was added with an already deprecated std430 layout, additionally the std140 layout doesn’t work properly with all variable types, eg floats always get padded to take up the space of a vec4, or arrays of floats or ints dont take into account the base alingment required by the specification etc)
So there was the option to fix that and thus break all projects that unconsciously rely on it or, as i did, offer an implementation that provides more lowlevel access (what the whole buffer rework was about) and additionally support querying GL for the actual layout of a Unform Block and Shader Storage Block (which means you can use shared, packed, std140 and std430 layouts, though packed should be used with caution), allow the user to register custom structs and map the data set on the buffer on the CPU side via reflection to the layout previously queried before sending it to the GPU (see https://github.com/AKasigkeit/jmonkeyengine/blob/master/jme3-examples/src/main/java/jme3test/compute/TestCombined.java#L90 lines 90-104 and lines 142-152 as example)
Additionally you can create ShaderStorageBuffers and UniformBuffers with a custom FieldBufferWriter which leaves it up to you to serialize the data but doesn’t require reflection
@tlf30 no need to be sorry at all I am incredibly thankful for that offer, myself I am sorry i didnt’t yet come to write tests for jme3-testing.
I am wondering though, does the test library support running an application for a single frame to make sure the rendered image is the same, as well as running for a fixed amount of time or frames to measure the performance? Because it would be interesting to see the performance of the GT630 for example running the TestCombined.java (again, it draws 1 million quads and also updates each of those quads position and scale via compute shaders each frame)
I don’t want you to have to run each test on each setup manually doing screenshots, so if you tell me what would be the best tests for you to make sure it works and also measure performance so it can be compared, I will try to set them up this weekend
Regarding CPU requirements, i don’t think there are any.
Again, thanks a lot for the offer and sorry it takes me so long to provide appropriate tests
1 Like
I have volunteers from my company who will help do testing, so we don’t mind taking screenshots, but I do believe that jme-testing can take single frame snapshots. As for capturing performance, you can just have the tests print to console the per-frame render stats, and we can also take a screenshot from the output of the DetailedProfilerState if it is attached to the tests. I have not checked to see if you have updated it, but you may want to update it for some more insight into the compute shader stats.
Let me know when you are ready with a test procedure and we can get started. We also found some older quadros and a GTX 950. We have some other cards too but they do not have OpenGL 3 support.
1 Like
Yes, it can take single frame snapshots and actually compare them, because that’s what it has been used for, confirming the render looks the same (with a margin of error of course).
This could also be done for a series of frames to ensure consistency, but performance was never an issue, because measuring on Github Actions would be pointless and automated testing of it is also (in theory you need a statistical analysis of the frame times and maybe even comparing the individual stages).
Now what can be done is rendering a few frames, tracking the frametime of those frames and averaging them or checking if individual frames fall far from the average, etc, but that won’t be a solid value if it’s not just being used to “see how the frametime for this scene is in general”.
I am in the Anchorage airport flying up to Deadhorse Alaska. I will be in the Artic Circle for the next two weeks. I will have limited internet while I am living at camp, but I try to stay up-to-date on the jme hub every night. If you don’t mind lets hold off on testing on the GPUs until I get back home on Dec 9th. If you want to run tests before then, my VP is willing to help, but he does not have much jme experience.
Let me know if there is anything I can help with while I’m away, I may only have a 128 kbs connection, but I will have my laptop.
I think the future trend must be Vulkan 
2 Likes
I am sorry to get back to you that late
Indeed I prepared some tests, however I am currently working on another feature (PBOs and appropriate texture streaming systems) that I would like to finish so it can be included in the tests.
Unfortunately I couldn’t figure out how to properly use the jme-testing framework so i created a tests branch and updated the test cases to automatically take screenshots, one in the first frame and another one after 10 seconds, they also use the DetailedProfilerState where it makes sence and include the number of total frames on the screenshot.
Thus it still requires running each test manually but it does not require any further input.
So if this if fine for you I would like to finish the PBOs, add probably 2 more test cases to work the same way as mentioned above and then come back to you.
1 Like
Yes, that would be fine. I look forward to seeing everything once you are done.
can you break this down into small feature pr’s so it’s easier to digest?
3 Likes
Can you elaborate more on this?
In theory it’s interfaces and implementations of regular unit tests once per each rendering framework (lwjgl2 vs 3)
Ah well I tried again and I guess I just had the engine put into the wrong directory last time, however the pbr test fails on my machine with some similarity value around 0.962… but I’ll check again when I’m home with the current master of jme to see if this comes from my changes
It takes me a little longer than I expected because I was quite inspired by the streaming system you posted in the monthly wip topic and after I finished adding the PixelPackBuffer and PixelUnpackBuffer I started working on a system to handle streaming the textures as well as a system to replace the Videorecorder and screenshotstate implementations which both give a huge performance improvement (meaning no nasty lag upon taking screenshot for example) since it’s both cases where you don’t care if the image comes in like 3 frames later. It currently works already but it needs some refactoring and better test cases.
So speaking about the test cases I would prefer implementing 2 or 3 cases that make use of several features at once for jme-testing to be run completely automatically to ensure the rendered results are the same, while still providing more test cases in the way I explained in my last post because I like that they are basically still interactive which might invite people a little more to play around with the code and get visual feedback instantly
5 Likes