How to pass non-texture data to SCNTechnique Metal shaders

Question

I can a pass custom parameter of type sampler2D to the Metal fragment function of an SCNTechnique and I have a working 2nd pass:

PList:

<key>inputs</key>
<dict>
    <key>imageFromPass1</key>
    <string>COLOR</string>
    <key>myCustomImage</key>
    <string>myCustomImage_sym</string>
</dict>

...

<key>symbols</key>
<dict>
    <key>myCustomImage_sym</key>
    <dict>
        <key>type</key>
        <string>sampler2D</string>
    </dict>
</dict>

Relevant Obj-C code:

[technique setValue: UIImagePNGRepresentation(myCustomTexture) forKey:@"myCustomImage_sym"];

Metal function parameters:

fragment half4 myFS(out_vertex_t vert [[stage_in]],
    texture2d<float, access::sample> imageFromPass1 [[texture(0)]],
    texture2d<float, access::sample> myCustomImage [[texture(1)]],
    constant SCNSceneBuffer& scn_frame [[buffer(0)]]) { ...

I access and use all these inputs in the shader function. It Works!

So far so good!

However, when I add another custom parameter of type float ...

<key>blob_pos</key>
<string>blob_pos_sym</string>

...

<key>blob_pos_sym</key>
<dict>
    <key>type</key>
    <string>float</string>
</dict>

[_sceneView.technique setValue:[NSNumber numberWithFloat:0.5f] forKey:@"blob_pos_sym"];

 constant float& blob_pos [[buffer(2)]]

... the passed values never reach the shader function.

I have tried

using different buffer(N) values up to 6
having the custom parameter in the vertex function
type vec3 and float3 instead of type float
different means of encoding my float to NSData

wrapping my float in a struct

[technique setValue:[NSValue valueWithSCNVector3: SCNVector3Make(0.5, 0.5, 0.5)] forKey:@"blob_pos_"];

SCNVector3 xx = SCNVector3Make(0.5, 0.5, 0.5);
[technique setValue:[NSData dataWithBytes:&xx length:sizeof(xx)] forKey:@"blob_pos_"];
[technique setValue:[NSData dataWithBytesNoCopy:&xx length:sizeof(xx)] forKey:@"blob_pos_"];

simd_float3 x = simd_make_float3(0.5, 0.5, 0.5);
[technique setValue:[NSData dataWithBytes:&x length:sizeof(x)] forKey:@"blob_pos_"];

float y = 0.5;
[technique setValue:[NSData dataWithBytes:&y length:sizeof(y)] forKey:@"blob_pos_"];

struct MyStruct {
    float x;
};
struct MyStruct myStruct = {
    0.5
};

[technique setValue:[NSValue valueWithBytes:&myStruct objCType:@encode(struct MyStruct)] forKey:@"blob_pos_"];
[technique setObject:[NSValue valueWithBytes:&myStruct objCType:@encode(struct MyStruct)] forKeyedSubscript:@"blob_pos_"];

... and it all failed.

Then I looked at handleBindingOfSymbol:usingBlock: ... but it is GLSL only.

I found it's Metal counterpart, handleBindingOfBufferNamed:frequency:usingBlock: ... which is not available in SCNTechnique.

I Googled SCNTechnique Metal ... and realized all of the projects used sampler2D parameters only.

Finally I learned that this isn't new but bugs developers for years.

Before I go and encode this float in a texture, let me know the missing bit to make it work the way intended.

FYI this is the workaround I found usable pending a proper solution: stackoverflow.com/questions/51026751/… — diviaki

David Robles David Robles · Accepted Answer · 2018-08-09T07:50:23

You have to use a struct to wrap your input symbols and be sure to use [SCNTechnique setObject:forKeyedSubscript:] to pass in your symbol values to your technique. The documentation for setObject:forKeyedSubscript: mentions Metal however it doesn't explain how to receive the value in a Metal function, which is unfortunate.

Using your example:

Technique Definition:

"inputs": [
    "imageFromPass1": "COLOR",
    "myCustomImage": "myCustomImage_sym",
    "blob_pos": "blob_pos_sym",
],
...

"symbols": [
    "myCustomImage_sym": ["type": "sampler2D"],
    "blob_pos_sym": ["type": "float"]
 ]

Obj-C:

[_sceneView.technique setObject:[NSNumber numberWithFloat:0.5f] forKeyedSubscript:@"blob_pos_sym"];

Metal:

typedef struct {
    float blob_pos; // Must be spelled exactly the same as in inputs dictionary.
} Inputs;

fragment half4 myFS(out_vertex_t vert [[stage_in]],
    texture2d<float, access::sample> imageFromPass1 [[texture(0)]],
    texture2d<float, access::sample> myCustomImage [[texture(1)]],
    constant Inputs& myInputs [[buffer(0)]]) { 

    float blob_pos = myInputs.blob_pos;

    ...

How to pass non-texture data to SCNTechnique Metal shaders

So far so good!

I have tried

1 Answers