CGFX shader extracting data from alpha channel

Question

I posted this as well on Nvidia Developer discussion.

http://forums.developer.nvidia.com/devforum/discussion/9946/extracting-numbers-cgfx-shader

I am learning to write my first cgfx shader- a terrain shader. So I can only use 1 vertex channel (the alpha channel) and I want it to store 3 masks. I programmed a script no problem in the 3d App Maya where I can store 1 mask in the tenths place, 1 in the hundreths, and 1 in the thousandths place.

Each original painted mask is clamped at 0 - .9 (to prevent zeroing out with fmod) so if a vert has:

mask 1 =.857

mask 2 =.345

mask 3 -.948

the combined alpha channel for this vert is = .839

The problem I am having is I cannot get these masks extracted into seperate masks properly in my shader.

If I output the Mask1 I get the correct nicely blended mask. I do this by:

float Mask1 = (fmod((Mcolor  * 10f ), 10f)) *.111f;

If I output the Mask2 I get all these weird artifacts. It looks like the mask1 is pushing itself ontop, even when I am only outputting Mask2. I am positive the math works, I have run the same thing in Maya to store the values to begin with. Can anyone tell me why this is happening?

Even:

float Mask2 = frac(Mcolor *10);

which should just push the mask 2 over into tenths place and erase Mask 1 results in the same artifacts.

float Mcolor = In.VertColor.a;
float Mask1 = (fmod((Mcolor  * 10f ), 10f)) *.111f;
float Mask2 = (fmod((Mcolor * 100f ), 10f)) *.111f;

I posted some images of the issue on the Nvidia forum, but I can't post here becuase of spam prevention.

http://forums.developer.nvidia.com/devforum/discussion/9946/extracting-numbers-cgfx-shader

Help is much appreciated -thanks for reading through! Please explain your response if you can/ have time, I am new to programming so the more descriptive the better!

Okay I found the issue. I need to do this in the vertex shader, not the pixel shader. So now my question is. How can I output something from the vertex shader to the pixel shader multiple times. Eg I need to output the vertex color as a float 4 to the pixel shader 4 times after performing some different math operations on the vertex color in the vert shader? can I do this? I tried creating an empty float4 color2, reading the In.vertcolor from In.color and outputing that... I dont get any errors but my texture is white. Its just reading any empty float 4... — fghajhe

bjorke bjorke · Accepted Answer · 2012-12-05T00:37:14

just add an additional output value to your vertex shader. Give it a 'texcoord' semantic, e.g. float4 Masks: TEXCOORD1;

Then just assign 'OUT.Masks.x' instead of 'Mask1,' 'OUT.Masks.y' instead of 'Mask2' etc

In the fragment/pixel shader, just include the same value in the input (using connector structs makes this easy to manage) and just read 'IN.Masks.x' etc

CGFX shader extracting data from alpha channel

1 Answers