SKSpriteNode nearest neighbor using SKShader fragment shader

Question

I'm trying to apply palette swapping capabilities in my spritekit 2d pixel art game and it appears that when applying an SKShader the filteringMode on the SKSpriteNode's texture is ignored.

As a result, I believe I need to apply nearest neighbor coloring first, then do the palette swapping logic second.

Based on some code found here on shader toy I've made this attempt which seems like the right direction and the logic seems sound to me if the coordinates are normalized and (0.0, 0.0) is the bottom left and (1.0, 1.0) top right, but the result is coming out WAY too blocky.

https://www.shadertoy.com/view/MllSzX

My adaptation for a shader.fsh file:

void main() {

    float texSize = 48.0;
    vec2 pixel = v_tex_coord * texSize;
    float c_onePixel = 1.0 / texSize;
    pixel = (floor(pixel) / texSize);
    gl_FragColor = texture2D(u_texture, pixel + vec2(c_onePixel/2.0));
}

How can I get nearest neighbor working on my SKShader before I move on to my palette swapping?

Answer 1

Not a perfect answer to my own but I managed to prevent this problem by adding PrefersOpenGL to YES in my info.plist however I understand that it is preferred to use Metal when possible on ios

Answer 2

I ran into a similar issue. I wanted to create an outline around pixels while keeping it pixelated but it was blurring the existing pixels making it look bad. I ended up implementing nearest neighbor then checking the neighbor pixels after calculating the nearest neighbor location to see if the pixel had an alpha greater than 0. If it did I'd fill in the pixel. Here is how I did it:

Outline.fsh:

vec2 nearestNeighbor(vec2 loc, vec2 size) {
    vec2 onePixel = vec2(1.0, 1.0) / size;
    vec2 coordinate = floor(loc * size) / size;
    return coordinate + onePixel / 2.0;
}

void main() {
    vec2 onePixel = vec2(1.0, 1.0) / a_sprite_size;
    vec4 texture = texture2D(u_texture, nearestNeighbor(v_tex_coord, a_sprite_size)); // Nearest neighbor for the current pixel
    if (texture.a == 0.0) {
        // Pixel has no alpha, check if any neighboring pixels have a non 0 alpha
        vec4 outlineColor = vec4(0.9, 0.9, 0.0, 1.0);
        if (texture2D(u_texture, nearestNeighbor(v_tex_coord + vec2(onePixel.x, 0), a_sprite_size)).a > 0.0) {
            // Right neighbor has an alpha > 0
            gl_FragColor = outlineColor;
        } else if (texture2D(u_texture, nearestNeighbor(v_tex_coord + vec2(-onePixel.x, 0), a_sprite_size)).a > 0.0) {
            // Left neighbor has an alpha > 0
            gl_FragColor = outlineColor;
        } else if (texture2D(u_texture, nearestNeighbor(v_tex_coord + vec2(0, onePixel.y), a_sprite_size)).a > 0.0) {
            // Top neighbor has an alpha > 0
            gl_FragColor = outlineColor;
        } else if (texture2D(u_texture, nearestNeighbor(v_tex_coord + vec2(0, -onePixel.y), a_sprite_size)).a > 0.0) {
            // Bottom neighbor has an alpha > 0
            gl_FragColor = outlineColor;
        } else {
            // No neighbors with an alpha > 0, don't change the color
            gl_FragColor = texture;
        }
    } else {
        // Pixel has an alpha > 0
        gl_FragColor = texture;
    }
}

You then need to add the shader to your sprite and set the defined attributes on your sprite and shader so values defined in the shader can be used.

        spriteNode.setValue(SKAttributeValue(vectorFloat2: vector_float2(Float(spriteNode.size.width), Float(spriteNode.size.height))), forAttribute: "a_sprite_size")
        let shader = SKShader(fileNamed: "Outline.fsh")
        shader.attributes = [
            SKAttribute(name: "a_sprite_size", type: .vectorFloat2)
        ]
        spriteNode.shader = shader

Hopefully this helps anyone else that has a similar issue!