After some debugging it turns out that it's not my code (even though its a bit weird :D)
What you'll see below is the discrepancy between blender importer (to the right, it doesn't support the wireframe modifier) and xbuf (to the left, it does fail for me):
Now some explaining: The hitboxes are separate geometries which are moved to the non-animated position by the artists (NOT the T-Pose!). Upon loading, I try to re-parent the hitboxes to the attachment nodes, so that it stays at the same world transform. (I do this by setting the local transform of the hitbox to: inverted attachment node transform combined with old hitbox world transform).
As you can see it works fine for blender. Cyan shows the attachment nodes of the hitboxes, so I am uncertain why they are wrong in blender, leading to the hitboxes to be right? Could it be that the attachment nodes get mode after the first update tick, so that I need to adjust my hitboxes after that? But then I wonder why they are correct in blender?
Edit: Here the code, now that xbuf should be correct it seems like I could be doing things wrong (since the boxes are rotated 90° around the origin):
Transform tHb = hb.getWorldTransform().clone(); // Before being added to the attachment
hb.setLocalTransform(tHb.combineWithParent(attachNode.getWorldTransform().invert())); // hb.setWorldTransform(tHb)