Understanding rotation.
Start with 2D.
A 2D coordinate defines a position via 2 values, x and y. The x is the distance along the x axis and the y is the distance along the y axis. By convention computer graphics generally have the x axis defined from left to right and y from top to bottom.
Note code in this answer is pseudo code and does not represent any particular language.
Axis
We can represent an axis as a vector, for example the x axis is 1 unit in the x direction and 0 units down, and the y axis is 0 units along the x direction and 1 unit down.
For code we define a vector as (eg x axis xAxis.x = 1, yAxis.y = 0)
Axis have an important quality that they are always 1 unit long. See below Unit vector
So with the axis defined and a point we can find its position by moving first along the x axis then along the y axis.
xAxis = { x : 1, y : 0 }; // define the x axis
yAxis = { x : 0, y : 1 }; // define the y axis
point = { x : 10, y : 10 };
// position the point first move along the x axis by distance x
pos.x = point.x * xAxis.x;
pos.y = point.x * xAxis.y;
// then move along the y axis by distance y
pos.x = pos.x + point.y * yAxis.x;
pos.y = pos.y + point.y * yAxis.y;
Rotation
This seems like the long way to position a point. But when you rotate a coordinate systems you are in effect rotating the x and y axis. So to get the coordinates in a rotated coordinate system you need the 2 rotated axis. See below Unit vector from angle
The x axis is rotated by the amount of rotation, and the y axis is at 90 deg to the x axis.
rotation = PI / 4; // using radians rotate clockwise 45 deg
// get the x axis at 45 deg
xAxis.x = cos(rotation);
xAxis.y = sin(rotation);
// get the y axis at 90 deg (PI / 2) from the x axis
yAxis.x = cos(rotation + (PI / 2));
yAxis.y = sin(rotation + (PI / 2));
We can now move the point in the rotated coordinate system
point = { x : 10, y : 10 };
// position the point first move along the x axis by distance x
pos.x = point.x * xAxis.x;
pos.y = point.x * xAxis.y;
// then move along the y axis by distance y
pos.x = pos.x + point.y * yAxis.x;
pos.y = pos.y + point.y * yAxis.y;
There are some short cuts. The y axis is at 90 deg to the x axis (unless skewed) to rotate a vector 90deg we swap the components negating the y
// get the x axis at 45 deg
xAxis.x = cos(rotation);
xAxis.y = sin(rotation);
// get y at 90 deg from x
yAxis.x = -xAxis.y;
yAxis.y = xAxis.x;
Also moving along the axis, each component is independent of the other so the calculation can be done one line.
pos.x = point.x * xAxis.x + point.y * yAxis.x;
pos.y = point.x * xAxis.y + point.y * yAxis.y;
The origin.
Coordinate systems are defined by an origin as well as the unit vectors describing the axis. To rotate a point around a specific point we need to make the point of rotation the origin. We then move the coordinate to rotate relative to the new origin
point = { x : 10, y : 10 };
origin = { x : 5 , y : 4 };
// move point relative to the origin
pos.x = point.x - origin.x;
pos.y = point.y - origin.y;
Now we can apply the rotation
rotatedPos.x = pos.x * xAxis.x + pos.y * yAxis.x;
rotatedPos.y = pos.x * xAxis.y + pos.y * yAxis.y;
But the rotated point is still relative to the origin, we need to move it back relative to the origin.
rotatedPos.x = rotatedPos.x + origin.x;
rotatedPos.y = rotatedPos.y + origin.y;
In computer graphics we generally keep related coordinates relative to its own local origin. It has a coordinate system that we call the local coordinates, with [0,0] or in 3d [0,0,0] as the point of rotation. This means we can skip the part of the calculation that moves the point relative to the rotation point.
We can then do the rotation and positioning in one line for each axis
pos.x = point.x * xAxis.x + point.y * yAxis.x + origin.x;
pos.y = point.x * xAxis.y + point.y * yAxis.y + origin.y;
Scale
It is also very common that we want to scale a coordinate, this can be done by changing the length of the unit vector representing each axis. For example if we want to scale the coordinate 2 times along the x axis we make the x axis 2 times as long
point = {x : 5, y : 6}; // point in local coordinates.
xAxis = {x : 1, y : 0}; // normalised x axis
xAxis.x = xAxis.x * 2; // scale x axis
xAxis.y = xAxis.y * 2;
// apply transformation.
pos.x = point.x * xAxis.x + point.y * yAxis.x + origin.x;
pos.y = point.x * xAxis.y + point.y * yAxis.y + origin.y;
3D
For 3D everything is much the same but with one additional axis and component
xAxis = {x : 1, y : 0, z : 0}; // direction and scale of x axis
yAxis = {x : 0, y : 1, z : 0}; // direction and scale of y axis
zAxis = {x : 0, y : 0, z : 1}; // direction and scale of z axis
origin = {x : 0, y : 0, z : 0}; // position of origin.
So to move point into the 3D coordinates system described above
point = {x : 5, y : 6, z : 4}; // point in local coordinates.
// move point.x distances along x axis
pos.x = point.x * xAxis.x
pos.y = point.x * xAxis.y
pos.z = point.x * xAxis.z
// move point.y distances along y axis
pos.x += point.y * yAxis.x
pos.y += point.y * yAxis.y
pos.z += point.y * yAxis.z
// move point.y distances along y axis
pos.x += point.z * zAxis.x
pos.y += point.z * zAxis.y
pos.z += point.z * zAxis.z
// then relative to the origin
pos.x += origin.x
pos.y += origin.y
pos.z += origin.z
Or more compact
pos.x = point.x * xAxis.x + point.y * yAxis.x + point.z * zAxis.x + origin.x
pos.y = point.x * xAxis.y + point.y * yAxis.y + point.z * zAxis.y + origin.y
pos.z = point.x * xAxis.z + point.y * yAxis.z + point.z * zAxis.z + origin.z
The matrix
The above starts to become a little unwieldy so to simplify it we can convert the above objects point, xAxis, yAxis, zAxis, origin into a set of arrays (called matrices)
point = [5,6,4]; // vector as array [x,y,z]
origin = [0,0,0]; // origin as array [x,y,z]
rotation = [1,0,0,0,1,0,0,0,1]; // 3 axis [x.x,x.y,x.z, y.x,y.y,y.z, z.x,z.y,z.z]
// rotation /*
[x.x,x.y,x.z, // x axis
y.x,y.y,y.z, // y axis
z.x,z.y,z.z] // z axis
*/
The notation can be simplified and in many languages overloading will allow you to do the math directly in shorthand form.
pos = point * rotation + origin;
3D rotation
In 2D we generally only rotate around a single imaginary axis ( the z axis in and out of the screen) in 3D we rotate around one of 3 axis, the x, y or z axis.
The order we rotate in also affect the final rotated location. Rotate 5 degree around z then 10 deg around y is different than 10 deg around y then 5 around z.
As each axis is itself a vector that can be rotated, we can rotate each axis by a rotation matrix. The result is a matrix that has many rotations all combined.
So say we want to rotate about the z axis 10 deg we create the 3 rotated axis
ang = 10; // in deg
xAxisA = [cos(ang) ,sin(ang),0]; // 1 unit long
yAxisA = [-sin(ang),cos(ang),0]; // 1 unit long
zAxisA = [0 ,0 ,1]; // 1 unit long
Or as a rotation matrix
A = rotationZ = [cos(ang), sin(ang), 0, -sin(ang), cos(ang), 0, 0, 0, 1];
And we want to then rotate about the y
xAxisB = [cos(ang) ,0 , sin(ang)]; // 1 unit long
yAxisB = [0, 1 , 0 ]; // 1 unit long
zAxisB = [-sin(ang),0, cos(ang)]; // 1 unit long
Or as a rotation matrix
B = rotationY = [cos(ang), 0, sin(ang), 0, 1, 0, -sin(ang), 0, cos(ang)];
We can then rotate each axis in the z rotation by the y rotation.
// rotate each rotation axis by the second rotation axis.
xAxisAB.x = xAxisA.x * xAxisB.x + xAxisA.y * yAxisB.x + xAxisA.z * zAxisB.x;
xAxisAB.y = xAxisA.x * xAxisB.y + xAxisA.y * yAxisB.y + xAxisA.z * zAxisB.y;
xAxisAB.z = xAxisA.x * xAxisB.z + xAxisA.y * yAxisB.z + xAxisA.z * zAxisB.z;
yAxisAB.x = yAxisA.x * xAxisB.x + yAxisA.y * yAxisB.x + yAxisA.z * zAxisB.x;
yAxisAB.y = yAxisA.x * xAxisB.y + yAxisA.y * yAxisB.y + yAxisA.z * zAxisB.y;
yAxisAB.z = yAxisA.x * xAxisB.z + yAxisA.y * yAxisB.z + yAxisA.z * zAxisB.z;
zAxisAB.x = zAxisA.x * xAxisB.x + zAxisA.y * yAxisB.x + zAxisA.z * zAxisB.x;
zAxisAB.y = zAxisA.x * xAxisB.y + zAxisA.y * yAxisB.y + zAxisA.z * zAxisB.y;
zAxisAB.z = zAxisA.x * xAxisB.z + zAxisA.y * yAxisB.z + zAxisA.z * zAxisB.z;
Or short hand
rotationAB = rotationZ * rotationY;
or
AB = A * B;
The resulting matrix AB is the combined rotation of the z rotation and then the y rotation. You can keep going, rotate around the x
// rotate about the x Axis
xAxisC = [1, 0 , 0 ]; // 1 unit long
yAxisC = [0, cos(ang) , sin(ang)]; // 1 unit long
zAxisC = [0, -sin(ang), cos(ang)]; // 1 unit long
C = rotationX =[1, 0, 0, 0, cos(ang), sin(ang), 0, -sin(ang), cos(ang)];
The final rotation is
ABC = A * B * C
But remember that the order is important.
A * B * C != C * B * A; // order of multiplication is important.
Using matrix, vector libraries.
The above is a what would be covered in a first year uni computer science course over a few weeks, but it would assume you have a good understanding of vector maths. Writing the code can become very repetitive and due to the nature of the problem, very hard to read and spot mistakes in.
I always believe that the best way to learn is to write your own library, but in this case a library is a good starting points as there is a lot more depth to this subject than just rotation, scale and translation.
There are hundreds of matrix/vector maths libraries out there, a good starting point would be github
Maths notation
In mathematics the axis are called î (pronounced i hat) for the x axis and ĵ (j hat) for the y axis. Each axis is defined by a unit vector, î = [1,0] and ĵ = [0,1] (for 3D we use three 3D vectors î = [1,0,0], ĵ = [0,1,0] and k hat = [0,0,1] sorry I could not find k hat in the character set)
Vector basics.
A vector
A vector is a set of numbers that represent a direction and distance. In math a vector is a matrix. Eg v = [1,0] in code as a structure/object/class v = { x : 1, y : 0} A vector can also be described as a distance and direction, eg 10 kilometers South East. It is easy to convert from one type to the other (see below)
Unit vector.
A unit vector is a vector that is 1 unit long. (1,0) is a unit vector its length is 1 unit long. You can multiply the unit vector to find a point n units for the origin.
n = 5;
axis = { x : 1, y : 0 };
point.x = axis.x * n; // 5 * 1 = 5
point.y = axis.y * n; // 5 * 0 = 0
Vector length
You can use pythagoras to get the length of a vector. For example the vector {x : 3, y : 4} has a length equal to sqrt( 3 * 3 + 4 * 4 ) = sqrt( 9 + 16 ) = sqrt( 25 ) = 5 five units long and clearly not a unit vector
Normalised vector (Normalized US spelling)
The process of normalising a vector converts a vector to a unit vector. It is done by dividing the vector's components by the vector length.
vec = { x : 3, y : 4 };
length = sqrt( vec.x * vec.x + vec.y * vec.y ); // length = 5
// nVec is the normalised vector vec
nVec.x = vec.x / length; // 3/5 = 0.6;
nVec.y = vec.y / length; // 4/5 = 0.8;
Unit vector from angle
To create a unit vector at a specific direction we use a little trig
angle = 90; // in deg (normally this is radians)
v90.x = cos(angle);
v90.y = sin(angle);
TRT^-1in mathematical notation. If so, I agree math.SE is a better choice here. – user354134