OpenGL shading in GPU sometimes throwing malloc error

Question

I have a program that reads in a file, stores it as a triangle mesh and sends it to the GPU using OpenGL. I'm now trying to add shading options to the program so that we can do flat or smooth shading. So, I wrote a function that finds the normals-per-face, and then gets the average per vertex to finally get the normal-per-vertex.

I believe that all should be okay.

However, once I added this, there's this very odd bug that's totally throwing me off which is that about 90% of the time I run the program, I get the following error:

malloc: *** error for object 0x7fb16e942410: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug

The other 10% of the time it draws this monstrosity.

For clarification, it should just draw the bumpy cube in the center in red.

I'm completely lost as to why I sometimes get a malloc error and sometimes get this weird image. I'm fairly certain the bug lies in something to do with the normals, but that doesn't explain the seeming randomness of running.

My vertex shader and fragment shader is

const GLchar* vertex_shader =
        "#version 150 core\n"
                "in vec3 position;"
                "in vec3 color;"
                "in vec3 normals;"

                "out vec3 FragPos;"
                "out vec3 Normal;"
                "out vec3 f_color;"

                "uniform mat4 model;"
                "uniform mat4 view;"
                "uniform mat4 proj;"

                "void main()"
                "{"
                "    FragPos = vec3(model * vec4(position, 1.0));"
                "    Normal = mat3(transpose(inverse(model))) * normals;"
                "    f_color = color;"
                "    gl_Position = vec4(FragPos, 1.0);"
                "}";
const GLchar* fragment_shader =
        "#version 150 core\n"
                "in vec3 Normal;"
                "in vec3 FragPos;"
                "in vec3 f_color;"

                "uniform vec3 lightPos;"
                "uniform vec3 lightColor;"
                "uniform vec3 triangleColor;"

                "out vec4 outColor;"

                "void main()"
                "{"
                "    float ambientStrength = 0.4;"
                "    vec3 ambient = ambientStrength * vec3(1, 0, 0);"

                "    vec3 norm = normalize(Normal);"
                "    vec3 lightDir = normalize(lightPos - FragPos);"
                "    float diff = max(dot(norm, lightDir), 0.0);"
                "    vec3 diffuse = diff * lightColor;"
                "    vec3 shaded_color = (ambient + diffuse) * f_color;"
                "    outColor = vec4(shaded_color, 1.0);"
                "}";

"I get the following error: malloc: *** error for object 0x7fb16e942410" - How is the shader code related to this issue? Probably you write to an array out of bounds in the c++ code. If you would use std::vector then it would be to find with ease. C++ and malloc, that doesn't fit together. — Rabbid76

datenwolf datenwolf · Accepted Answer · 2018-11-23T09:09:48

The bug you've got has nothing to do with shaders. You've got a use-after-free or related bug somewhere in your program. In essence you're either writing out-of-bounds of an allocated memory region, thereby corrupting memory management data, or you're writing to memory after you've freed it.

Example of out-of-bounds:

void out_of_bounds()
{
    char *buf;
#ifdef __cplusplus
    buf = new char[8];
#else
    buf = malloc(8);
#endif
    buf[8] = 'x'; /* out of bounds just one element past the end */
    buf[999] = 'x'; /* out of bounds waaay past the end */
}

Example of use after free:

void use_after_free(char *buf)
{ 
    #ifdef __cplusplus
        delete[] buf;
    #else
        free(buf);
    #endif

    buf[0] = '\0';
}

The error message is pretty clear about the problem:

malloc: *** error for object 0x7fb16e942410: incorrect checksum for freed object - object was probably modified after being freed. *** set a breakpoint in malloc_error_break to debug

Somewhere in your program you've got a bug that's semantically equivalent to the illustrative code snippets above. You've to find and fix that.

Most likely the bug is in the code that reads the files and passes their contents to OpenGL.

OpenGL shading in GPU sometimes throwing malloc error

1 Answers