There are two coordinate systems. We know the 3D coordinates of the origin and the 3D vectors of the axes of the second coordinate system with respect to the first coordinates system. Then how can we find the rotation matrix that transforms the first coordinate system into the second coordinate system?
The problem described can be solved as follows. Let
M = m_11 m_12 m_13
m_21 m_22 m_23
m_31 m_32 m_33
denote the desired rotation matrix. We require
1 0 0 * M + t = x_x x_y x_z
0 1 0 y_x y_y y_z
0 0 1 z_x z_y z_y
where t denotes the translation; we see that this matrix equality can be solved by multiplying from the left with the identity matrix, which is the inverse of itself; hence we obtain the following equality.
M + t = x_x x_y x_z
y_x y_y y_z
z_x z_y z_y
This can be rearranged by subtracting t from both sides to obtain the desired matrix M as follows.
M = x_x x_y x_z - t = x_x-t_x x_y-t_y x_z-t_z
y_x y_y y_z y_x-t_x y_y-t_y y_z-t_z
z_x z_y z_y z_x-t_x z_y-t_y z_z-t_z
Note that this was relatively easy as the initial matrix consists out of the basic vectors of the standard base. In general it is more difficult and involves a basis transformation, which basically can be done by Gaussian elimination, but can be numerically difficult.
I've written an article about it that demonstrates how to do it, with source code. The short answer is that you build a 3x3 matrix with the dot products of the different axis
http://www.meshola.com/Articles/converting-between-coordinate-systems
