Why are the GLSL texture-coordinates in my shader not linear?

Question

I have a problem with my glsl shaders on my Android 2.3.4 Tegra 2 device.

I want to do some calculations to create texture coordinates in the fragment shader in a glsl shader and noticed a problem and tried to narrow it down and therefore made a testprogram. I rendered a texture with a width of 256 pixels to exactly 256 pixels onscreen, save the framebuffer and check it pixel by pixel (compare with source) and it renders correctly. According to the opengl specification when I use the texture coordinates from 0.0 to 1.0 it should sample each pixel in the center (0.5/256, 1.5/256, 2.5/256, etc.), which seems right.

Then I wrote the x texture-coordinates themselves into the texture and checked those as well. The results are somewhat off and I don't know why. Instead of getting a gradient of 0..255 which I expected I got 1..128,128..255. That means the value 0 is skipped and the value 128 appears twice which means they are not perfectly linear. If the value was rounded in any way then not in a way that is obvious to me. I also checked the subpixel values for the texture-coordinates and they are what I expected. Their value is always 0.5/256, which should be half a pixel and ensure that the texture is sampled within the pixel correctly.

Again, the texture rendered with these coordinates is exactly what I want, but the coordinates themselves are not what I expected. This is a problem for me because I want to render with calculated texture coordinates and although I calculate them in a way that should be correct, the resulting texture is wrong. I need these coordinates to be correct to the pixel level and I don't know why they are off.

Does anyone know what is causing this behaviour and how I can correct it?

This is the fragment shader that renders the texture correctly:

precision mediump   float;
varying vec2        vtex;
uniform sampler2D   samp0;
void main()     {
            gl_FragColor = texture2D(samp0, vtex);
        };

This is the fragment shader that creates a gradient with the texture coordinates that is 1..128,128..255 and not 0..255 as I expected :

precision mediump   float;
varying vec2        vtex;
uniform sampler2D   samp0;
void main()     {
            gl_FragColor = vec4(vtex.x, 0, 0, 1.0);
        };

This is the fragment shader that creates a color for the subpixel values of the texture coordinates and delivers a texture with all values 127 as I expected, and which should be the center of the pixels:

precision mediump   float;
varying vec2        vtex;
uniform sampler2D   samp0;
void main()     {
            gl_FragColor = vec4(fract(vtex.x * 256.0), 0, 0, 1.0);
        };

The rest of the test programs are absolutely identical. I only change the shaders. These are some significant code snippets:

String vertexshader =
    "attribute vec4 a_pos;          \n" +   //  in      position
    "attribute vec2 a_tex;          \n" +   //  in      Texture coordinates
    "varying vec2 vtex;             \n" +   //  out     Texture coordinates
    "void main(void) {              \n" +
    "   gl_Position = vec4(a_pos);  \n" +
    "   vtex = a_tex;               \n" +
    "}";


//  Initialization

    GLES20.glUseProgram(shader);
    //  Vertex
    vdata.position(0);
    GLES20.glVertexAttribPointer(handlepos, 2, GLES20.GL_FLOAT, false, 4*4, vdata);
    GLES20.glEnableVertexAttribArray(handlepos);
    vdata.position(2);
    GLES20.glVertexAttribPointer(handletex, 2, GLES20.GL_FLOAT, false, 4*4, vdata);
    GLES20.glEnableVertexAttribArray(handletex);
    //  Texture
    GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
    GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, maplookupid[1]);
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST );
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_NEAREST );
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_REPEAT );
    GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_REPEAT );
    GLES20.glUniform1i(handlesampler0, 0);
    GLES20.glDisable(GLES20.GL_DITHER);

//  Rendering

    GLES20.glViewport(0, 0, screenx, screeny);
    //
    vdata.put(0*4+0,     0.0f * (2.0f/screenx));
    vdata.put(0*4+1,   256.0f * (2.0f/screeny));
    vdata.put(0*4+2,     0.0f * (1.0f/256.0f));
    vdata.put(0*4+3,   256.0f * (1.0f/256.0f));
    //  
    vdata.put(1*4+0,     0.0f * (2.0f/screenx));
    vdata.put(1*4+1,     0.0f * (2.0f/screeny));
    vdata.put(1*4+2,     0.0f * (1.0f/256.0f));
    vdata.put(1*4+3,     0.0f * (1.0f/256.0f));
    //  
    vdata.put(2*4+0,   256.0f * (2.0f/screenx));
    vdata.put(2*4+1,   256.0f * (2.0f/screeny));
    vdata.put(2*4+2,   256.0f * (1.0f/256.0f));
    vdata.put(2*4+3,   256.0f * (1.0f/256.0f));
    //  
    vdata.put(3*4+0,   256.0f * (2.0f/screenx));
    vdata.put(3*4+1,     0.0f * (2.0f/screeny));
    vdata.put(3*4+2,   256.0f * (1.0f/256.0f));
    vdata.put(3*4+3,     0.0f * (1.0f/256.0f));

    GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);

Chris Dodd Chris Dodd · Accepted Answer · 2012-02-18T05:37:10

Looks like you're running into rounding problems. Since you're apparently using an 8-bit color buffer, colors are converted to pixel values by multiplying them by 255. So your texture value 0.5/256 will get multiplied by 255 to produce 0.498 for the pixel value, which in an ideal world (infinite precision computation on the GPU and round to nearest) would convert to a 0 byte value.

In fact, from the fact you get 1..128,128..255 (128 is the repeated value), it looks like its rounding the conversion up (since (127.5/256)*255 is 127.002 and (128.5/256)*255 is 127.998 -- those being the two pixels that convert to 128). If that is what is happening though, I would expect your 3rd test program to produce 128 in every pixel, rather than 127 (since 0.5*255 == 127.5)

The upshot is that that you can't expect extremely precise computation from a GPU -- there are many sources of rounding error.

Why are the GLSL texture-coordinates in my shader not linear?

2 Answers