Given a triangle consisting of 3 vertices p1, p2 and p3, construct a fourth vertex p4 = p1+(p2-p1)×(p3-p1). Convert from Euclidean (3D) to homogeneous (4D) coordinates, setting w=1. Construct a 4x4 matrix whose columns are the 4 vertices.
Do this for both triangles to obtain matrices A and B. M=A-1·B is the transformation from A to B, M-1=B-1·A is the inverse (from B to A). The rotation and translation components can be extracted as a quaternion and vector via e.g. Matrix4d.get().
Note that you need to specify the vertices in the same order for both triangles. Otherwise you’ll get a matrix which includes scale/shear transformations. If the order is correct and the two triangles are the same except for translation and rotation, the bottom row of M (and M-1) will be [0,0,0,1] (give or take rounding errors), and the upper 3x3 submatrix will be orthonormal (multiplying it with its transpose will yield the identity matrix, again give or take rounding errors).
Can someone help me to get this done with pure Java3d classes? I would pay for.
There is no context so while I’m sure that paragraph makes sense, it also makes no sense because we have no idea what you are attempting to achieve. Explain to us what it is you want to do, and we may be able to get you to understand. We will not do it for you, however.