After reading and trying to apply the advice from How to enable Hardware Percentage Closer Filtering? into my pixel shader, I got some very strange results when debugging - in that all of the pixels had very low RGB values although the shader returned some greater RGB float4 values.
This issue this only manifested itself while using the hardware (GPU) and was behaving properly when forcing WARP (via the DirectX Control Panel). I thought it was a driver problem, but tested on two different computers, one with a mobile nVidia GPU, the other with a desktop AMD card, and behavior is the same with both - it only works as expected with WARP.
Check out the following screenshots of debugging the Pixel Shader which I think are very informative about my encountered problem:
Debugging the pixel shader with a regular hardware run:
Have a look at the return value - and on the right - in the "Graphics Pixel History" at the Pixel Shader RGB values and Result values - how come they are so different? Nearly division by 3 factor.
Now, take a look at the same debugging - on a similar shadowed pixel - when running the same shader - only now using WARP instead of running on the GPU:
As you can see, the values are nearly identical - and I think the small diferences are because of the render target being 8 bit per channel while the float is 4 bytes (32 bits).
My previous version of the shader was inspired by this blog post and snippets, and : https://takinginitiative.wordpress.com/2011/05/25/directx10-tutorial-10-shadow-mapping-part-2/
I implemented a 16-tap PCF - taking 16 samples with the Sample function instead of using the SampleCmpLevelZero func and doing the comparison with the target value myself, in the shader - for deciding the extent the pixel should be shadowed.
Upon reading the question mentioned above - I modified my shader to use SampleCmpLevelZero and that's when my problems appeared and my shaders only behaved as expected with WARP.
Can someone give me some suggestion on what else to check to further debug this situation?
