JUnion 1.2.x - Struct Types, Feedback Welcome

User Code & Projects
1

Hi :slightly_smiling_face:, it was already over a year ago since I posted an update about Project JUnion - Struct Types for Java

I would definitely like to hear your feedback :slightly_smiling_face:

  • After almost a complete redesign of the system, one can now allocate structs on heap, off-heap and stack.
  • Then one can retrieve the underlying primitive array/Buffer
  • As a link between user code and ram is the Bridge interface. DefaultBridge uses Unsafe to read/write memory. It would be interesting to not depend on Unsafe. From Java 9+ eg VarHandleBridge could be implemented which uses VarHandles instead and then let the user choose.
  • StructList [WIP] data structure - resizable struct array
  • Currently all structs use native byte order (primitive Java arrays use native byte order too afaik), support for other byte order can be added
  • Currently all structs use aligned memory access, adding an annotation to mark a struct as unaligned can be added as well, thou unaligned access will be slower

JUnion 1.2.2 Early Access: Releases · TehLeo/junion · GitHub

Since this is early access, expect bugs (eg. StructList is not finished yet).
To make this library actually usable, I think the next stage will be to write detailed test cases. Afterwards, a test project/library to really test it out would be nice. Eg. could write basic vector, quaternion, matrix lib with structs and then have fun with micro benchmarking it against existing Java object math libraries :smiley: Or another use case would be to write Java ECS with structs :slight_smile: .

8 Likes
2

Sry for asking after such a long time, I have two questions:

  1. Will JUnion work with java 12+
  2. I’m currently using other low-level memory packages such as OpenHFT Chronicle Bytes, should there be any conflicts between them and JUnion?
3

Thank you for your question!

Yes, JUnion works with Java 12+ atm, you can give it a try by downloading the newest realease JUnion 1.2.2 EA

There should be no conflicts, and with the newest version, a created struct array can be backed by a Java nio direct/heap buffer or java array.

1 Like
4

Thanks, much appreciated!

1 Like
5

Also another question. Is it is possible that I can create say a primitive array and also have it mapped to a DirectBuffer?

6

A Java array is stored on Java heap, and DirectBuffer stores data off-heap, thus they are mutually exclusive by definition.

But you can create a struct array which stores data as tightly packed as a primitive array and is mapped to a DirectBuffer.

7

Hi @The_Leo. This project is much appreciated so thanks for that. I haven’t gotten around to looking through the code, so may I ask, how do you simulate stack allocation ?

1 Like
8

Thank you for the question! :slightly_smiling_face:

The stack memory is stored off-heap. When a thread requests a variable to be allocated on stack for the first time, at that time a new stack for that thread is created. The initial and maximum size of stack can be set in MemInit class.

When a variable/array is allocated on stack, the position of the stack is stored to local variable, once the method that allocated this returns, the memory is freed with the use of try { } finally {} (to ensure the memory is freed in case of exception) by setting the stack position back to the value prior to the call.

3 Likes
9

Another question. I seem to be running into an issue regarding compiling a project using JUnion. I added the required config into the gradle file as per the outline here:

Though I seem to be running into an issue with the following error:

Exception in thread "main" theleo.jstruct.hidden.CompileException: Not compiled with jstruct.
    at theleo.jstruct.Mem.sizeOf(Mem.java:379)
    at com.ritualsoftheold.terra.core.JUnionTestDriver.main(JUnionTestDriver.java:7)

Is there a sample gradle project that I could look at to figure out where I’m going wrong?

10

The error is received because the code was not compiled with junion. Btw for the gradle in the linked document uses union from maven central, the newest release is not yet on maven as it is not finished.

I will put adding a simple gradle project on my todo list.

11

Hi again @The_Leo . Just curios have you ever tested your structs with cache aware algorithms ? Would be interesting to know if the c2 compiler adds these flat structures in the cpu caches for data with spatial locality. I guess the best test would be an ecs implementation that uses junion, :wink: :wink: @pspeed . Wish I had the time to test stuff like this. Anyway, just a curiosity. Have a nice day !

12

I haven’t created a test yet with focus on a cache-aware algorithm.
What I did test, was vector multiplication etc, and the result almost the same to that of primitive arrays.
Yes, ecs with structs is on my todo list as well. If things go well I my end up using it in my game.

1 Like