HLSL Shader: Calling function changes return value

Question

I have been working on a shader program, and one of the weirdest things is happening. I have a method called PositionLightPS, which essentially performs computations for positional lights (omni and spotlights) and returns a struct containing both the intensity and the diffuse colour of the light.

It used to work in a previous revision of the program, however it stopped working for some reason...

Let me explain why. But first, here is the code for the PositionPS method. I have no doubt that the calculations of this method are correct, but more on that later.

struct LightFragment {
    float4 diffuse;
    float intensity;
};

LightFragment PositionLightPS(PositionLightVOut pin, float3 lightToPixelVec)
{
    LightFragment result;

    pin.normal = normalize(pin.normal);

    float4 diffuse = shaderTexture.Sample( textureSampler, pin.tex0 );
    float3 finalDiffuse = float3(0.0f, 0.0f, 0.0f);

    float d = length(lightToPixelVec);
    if( d > plRange )
    {
        result.diffuse = float4(finalDiffuse, 0);
        result.intensity = 0; 
        return result;
    }

    lightToPixelVec /= d; 

    result.intensity = dot(lightToPixelVec, pin.normal);

    if( result.intensity > 0.0f )
    {   
        finalDiffuse = result.intensity * diffuse.xyz;
        finalDiffuse.xyz *= clColour.xyz;
        finalDiffuse.xyz *= clColour.w;

        finalDiffuse /= plAttenuation[0] + (plAttenuation[1] * d) + (plAttenuation[2] * (d*d));
    }   

    result.diffuse = float4(finalDiffuse, diffuse.a) * diffuseColour;
    return result;
}

Here is the omni light pixel shader.

float4 OmniLightDiffusePS(PositionLightVOut pin) : SV_TARGET0
{
    float3 lightToPixelVec = plPosition.xyz - pin.worldPos.xyz;
    return = PositionLightPS(pin, lightToPixelVec).diffuse;
}

For some reason, this does not work, although it had been for a long time. But this is not the focus of my question. What is definitely strange, is that these two methods yield different results.

float4 OmniLightDiffusePS(PositionLightVOut pin) : SV_TARGET0
{
    //return float4(1,1,1,1);
    float3 lightToPixelVec = plPosition.xyz - pin.worldPos.xyz;
    LightFragment fragment = PositionLightPS(pin, lightToPixelVec);
    return float4(1,1,1,1);
}

This one will return a black pixel

float4 OmniLightDiffusePS(PositionLightVOut pin) : SV_TARGET0
{
    return float4(1,1,1,1);
    float3 lightToPixelVec = plPosition.xyz - pin.worldPos.xyz;
    LightFragment fragment = PositionLightPS(pin, lightToPixelVec);
    //return float4(1,1,1,1);
}

This one will return a white pixel

So this means that calling the PositionLightPS method has an influence on the value returned... but why? Why would it? What is happening? Is it a memory error?

(Note: Copying the PositionLightPS method code directly in the OmniLightDiffuse method solves the problem, and the light displays normally. This means that the calculations performed in PositionLightPS are correct. The question is then, does using LightFragment struct have an impact on the shader program somehow?)

Christophe Lionet Christophe Lionet · Accepted Answer · 2012-12-07T20:40:33

I found the cause of the problem. My computer uses CUDA and locked itself into using the integrated graphics chip without my knowledge. Forcing the NVidia card solved the problem.

However, I did find this very useful blog post which details what was essentially my problem. Calling a user define function causes the inlining by the compiler to delete the pixels it treats, when two specific compiler flags are simultaneously active. I do not know if the integrated graphics chip reproduced this exact error, but the symptoms seemed to correspond perfectly (ie calling a function with no incidence on the return value will change what is being returned).

For more details, here is the link to the blog post, by Nico Shertler: http://nicoschertler.wordpress.com/2012/01/02/hlsl-4-calls-to-user-defined-functions-will-crash/

HLSL Shader: Calling function changes return value

3 Answers