When configuring OpenGL with glTexParamteri(GL_Texture_2D, GL_TEXTURE_MAG_FILTER, ...) and glTexParamteri(GL_Texture_2D, GL_TEXTURE_MIN_FILTER, ...) how does OpenGL decide which filter to use when accessing a texture in shader with texture(...)?
My only guess it's that it is calculating the pixel footprint but since you could access the texture in either the fragment or vertex shader it can't know on which primitive what texture is projected.
My only guess it's that it is calculating the pixel footprint
Yes, that's what it does. It will approximate the pixel footprint in the texture space by calcualting the derivatives of the texcoords with respect to the window space x and y direction, and it will approximate these derivatives by finite differencing in a 2x2 pixel quad, just like the dFdx and dFdy GLSL functions are working. It will use the longer of the two partial derivative vectors as the size, and calculate the Level-Of-Detail value based on that.
but since you could access the texture in either the fragment or vertex shader it can't know on which primitive what texture is projected.
Correct, that's why the GLSL specification, (Version 4.60) states the following in the beginning of section 8.9 Texture Functions:
Texture lookup functions are available in all shading stages. However, automatic level of detail is computed only for fragment shaders. Other shaders operate as though the base level of detail were computed as zero
Some (ie: most) GLSL texture accessing functions say that they require "implicit derivatives". All such functions only work entirely:
If you call a texture access function that requires implicit derivatives in a non-fragment shader, then it will only access from the base mipmap level. However, if you're in a fragment shader but outside of uniform control flow, then all such functions have undefined behavior.
So if you're not in a fragment shader, you either want to access from the base mipmap level (in which case, MAG_FILTER applies), or you want to use functions that explicitly provide the values used for doing interpolation: Lod (where you explicitly say what mipmap level(s) you fetch from), Grad (where you explicitly specify derivatives used to decide where the pixel footprint is), or any of the texelFetch or textureGather functions (which don't do interpolation at all).
