HLSL and XNA: How do I get the position and color of surrounding pixels of the current pixel shader iteration?

Question

Abstract and Goal:

I am trying to make a shader to perform a simple window effect for a game editor. The effect will draw a frame with a low value border color and a high value highlight. I've tried many methods, but overall I have only come up with one possible solution to achieve this using the GPU.

First I create a custom vertex type for storing the XY coordinates of a vector in screen space.

using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;

namespace WindowsGame1 {
    public struct Vertex2D : IVertexType {
        public Vector2 Position;
        public static readonly VertexDeclaration VertexDeclaration = new VertexDeclaration(new VertexElement(0, VertexElementFormat.Vector2, VertexElementUsage.Position, 0));

        public Vertex2D(float x, float y) {
            Position = new Vector2(x, y);
        }

        VertexDeclaration IVertexType.VertexDeclaration {
            get {
                return VertexDeclaration;
            }
        }
    }
}

Next I create an instance of a custom Window class. The constructor sets up the vertex buffer and sets the view, projection, and color parameters in the effect.

Here is the effect file.

float4x4 view;
float4x4 projection;
float4 color;
float shadowPercent = 0.36893203883495145631067961165049;
float highlightPercent = 1.262135922330097087378640776699;
Texture2D targetTexture;

struct FillVertexShaderInput {
    float4 position : POSITION0;
};

struct FillPixelShaderInput {
    float4 position : POSITION0;
};

struct BorderPixelShaderInput {
    float4 position : SV_Position;
};

// Transforms color component range from 0-255 to 0-1.
float4 ClampColor(float4 color) {
    return float4(color[0] / 255, color[1] / 255, color[2] / 255, color[3] / 255);
}

// Shifts the value of a color by a percent to get border color and highlight color from a fill color.
float4 ShiftValue(float4 color, float percent) {
    return float4(clamp(color[0] * percent, 0, 1), clamp(color[1] * percent, 0, 1), clamp(color[2] * percent, 0, 1), clamp(color[3] * percent, 0, 1));
}

FillPixelShaderInput FillVertexShader(FillVertexShaderInput input) {
    FillPixelShaderInput output;
    output.position = mul(mul(input.position, view), projection);
    return output;
}

float4 FillPixelShader(FillPixelShaderInput input) : COLOR0 {
    return color;
}

float4 BorderPixelShader(BorderPixelShaderInput input) : COLOR0 {
    // Get color of pixel above?
    // float4 tempColor = texture.Sample(sampler, (input.position[0], input.position[1] - width));
    return color;
}

technique Frame {
    // Store Texture2D, sampler2D, and others to be stored between passes?
    /*Texture2D texture;
    sampler2D sampler;
    float width;
    float height;
    texture.GetDimensions(width, height);
    color = ClampColor(color);
    float4 shadowColor = ShiftValue(color, shadowPercent);
    float4 highlightColor = ShiftValue(color, highlightPercent);*/

    pass Fill {
        VertexShader = compile vs_2_0 FillVertexShader();
        PixelShader = compile ps_2_0 FillPixelShader();
    }

    pass Border {
        PixelShader = compile ps_4_0 BorderPixelShader();
    }
}

I would like to be able to store the data between passes, but I don't know if that is possible so I tried storing a render target in XNA and using it as a parameter for the next pass.

Here is the Draw code of the Window.

public void Draw(Game1 game) {
    // rectangle is a simple window for this test.
    RenderTarget2D target = new RenderTarget2D(game.GraphicsDevice, rectangle.Width, rectangle.Height);
    game.GraphicsDevice.SetRenderTarget(target);
    game.GraphicsDevice.BlendState = BlendState.AlphaBlend;
    game.GraphicsDevice.Clear(Color.Transparent);
    game.GraphicsDevice.SetVertexBuffer(vertexbuffer);
    effect.Techniques["Frame"].Passes["Fill"].Apply();
    game.GraphicsDevice.DrawPrimitives(PrimitiveType.TriangleStrip, 0, 2);
    game.GraphicsDevice.SetRenderTarget(null);
    effect.Parameters["targetTexture"].SetValue(target);
    effect.Techniques["Frame"].Passes["Border"].Apply();
}

If I can get the position and color of surrounding pixels of the current pixel in a pixel shader, I can determine what color to draw the pixel at the current position. The Fill works fine. I just don't know the best way to go about drawing the border shadow and highlight. Also I am having problems with the alpha blending. Everything except the window is the default dark purple color, even though I set alpha blending and clear the render target buffer to transparent.

Thanks in advance if you decide to help.

For blending, I think you want to set the render target buffer to opaque, instead, though I'm not positive. Also be careful with how you treat the w-component of the color in the pixel shader, because that's the opacity. — Thomas
Just found out XNA content pipeline can only compile Shader Model 3_x and lower, so my idea of using Texture2D.GetDimensions won't work. Also the alpha blending was just because the windows wouldn't always be perfect squares. Sometimes the corners would have transparent rounding. — kkirkfield
Doing a lot of research I found out how XNA uses RenderTargetUsage, and that rendertargets share the buffer of the back buffer by default. By changing render targets my back buffer was being cleared. I just changed it to draw the background and window to preserved render targets and it solved part of my alpha problem. A note to anyone that is also having this problem: make sure the render target format supports alpha channel. Opaque blend option worked just fine, so thanks for that. — kkirkfield
Now I just need to figure out how to get per pixel position in ps_3_0. There is a VPos semantic for getting it, but I have yet to find a good example or tutorial that implements it. — kkirkfield

adenzila adenzila · Accepted Answer · 2013-12-16T01:15:52

I believe it works like this:

PixelShaderOutput PixelShaderFunction(VertexShaderOutput input, float2 vPos : VPOS)
{
// shader code here
}

Then you can just use vPos.x and vPos.y to access the coordinates of current pixel being processed.

HLSL and XNA: How do I get the position and color of surrounding pixels of the current pixel shader iteration?

1 Answers