I am having a bit of an issue with any rotation around the x-axis inside of OpenGL. What I have is a basic cube that is being rendered inside of a shader, with normals for lighting calculations. The transformation of the cube is the projection matrix multiplied by the model matrix. This concatenation is done inside the shader, while the rotation and translation calculations are done inside a Matrix class I wrote. When any rotations are done around the y-axis, everything rotates as expected as shown by these pictures:


The problems start when any rotations occur around the x-axis as shown here:


Apparently the x-axis rotations skew the scene and cause everything to be shown out of proportion. Here is the code the calculates the matrices and passes is to the shader every time this is rendered:
glClear(GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);
viewMatrix.setIdentity();
viewMatrix.translate(new Vector4(translation));
viewMatrix.rotate(1, 0, 0, -rotateX);
viewMatrix.rotate(0, 1, 0, rotateY);
Matrix4 mvMatrix = new Matrix4(viewMatrix);
Matrix4 pMatrix = new Matrix4(projectionMatrix);
lightShader.useShader();
lightShader.setUniformVector("vColor", new Vector4(.4f,0,0,1f));
lightShader.setUniformMatrix("mvMatrix", mvMatrix);
lightShader.setUniformMatrix("pMatrix", pMatrix);
triangleBatch.draw(lightShader.getAttributes());
Display.update();
The shader code is as follows:
uniform mat4 mvMatrix;
uniform mat4 pMatrix;
uniform vec4 vColor;
varying vec4 outFragColor;
attribute vec4 inVertex;
attribute vec4 inNormal;
void main(void) {
vec3 newNorm = vec3(inNormal);
mat3 mNormalMatrix;
mNormalMatrix[0] = mvMatrix[0].xyz;
mNormalMatrix[1] = mvMatrix[1].xyz;
mNormalMatrix[2] = mvMatrix[2].xyz;
vec3 vNorm = normalize(mNormalMatrix * newNorm);
vec3 vLightDir = vec3(0.0, 0.0, 1.0);
float fDot = max(0.0, dot(vNorm, vLightDir));
outFragColor.rgb = vColor.rgb * fDot;
outFragColor.a = vColor.a;
mat4 mvpMatrix;
mvpMatrix = pMatrix * mvMatrix;
gl_Position = mvpMatrix * inVertex;
}
Finally the matrix math code for rotations is as follows:
public Matrix4 rotate(Vector4 a, float radians){
Matrix4 temp = new Matrix4().setToRotate(a, radians);
this.multiply(temp);
return this;
}
public Matrix4 setToRotate(Vector4 a, float radians){
a = new Vector4(a).normalise();
float c = GameMath.cos(radians);
float s = GameMath.sin(radians);
float t = 1-c;
float x = a.x;
float y = a.y;
float z = a.z;
float x2 = a.x*a.x;
float y2 = a.y*a.y;
float z2 = a.z*a.z;
this.setIdentity();
this.matrix[0] = c + t*x2;
this.matrix[1] = t*x*y + s*z;
this.matrix[2] = t*x*z - s*y;
this.matrix[4] = t*x*y - s*z;
this.matrix[5] = c + t*y2;
this.matrix[6] = t*y*z + s*x;
this.matrix[8] = t*x*z + s*y;
this.matrix[9] = t*y*z + s*x;
this.matrix[10] = c + t*z2;
return this;
}
The matrices are stored in column major.