I'm new to Metal. I'm rendering a SceneKit scene with Metal using this Apple sample code. TLDR; it calls the SCNRenderer's render function and passes in a command buffer. I'm compiling for Big Sur.
It works, but it is not anti-aliased. I've tried a few ways to achieve it, as you can see in the updates below.
Without Metal, I'd just set isJitteringEnabled to true on the SCNRenderer, and I get beautiful (and slow) 96-ish-pass renderings. If I try to do this with Metal, I get weird pixel format mismatches, so I'm suspecting the two just aren't compatible.
With Metal, as far as I can tell, the simplest way to achieve antialiasing is to enable multi-sampling in the render pipeline (I know how to do that) — and use a multi sampling texture (MTLTextureType.type2DMultisample). This partial answer backs up my assumption.
And that's the problem. I don't know how to change the texture type when I get my texture from CVMetalTextureCache and CVMetalTextureCacheCreateTextureFromImage. It seems this is a limitation in Core Video's Metal support?
My full source is here
That's it. The rest of this post is more details on the stuff I tried.
(I think this might be possible using a shader. I'm open to that solution as well, but I don't know where to start. This example doesn't compile, and this example is for GSLS)
My pixel buffer atts look like this
let pixelbufferAttributes = [
kCVPixelBufferPixelFormatTypeKey : kCVPixelFormatType_32BGRA,
kCVPixelBufferWidthKey: exportSettings.width,
kCVPixelBufferHeightKey : exportSettings.height,
kCVPixelBufferMetalCompatibilityKey: true] as [String: Any]
For each frame, it creates a new pixel buffer from the pool, wraps it in a Metal texture from a cache, like this
let pixelFormat = MTLPixelFormat.bgra8Unorm_srgb
var optionalMetalTexture: CVMetalTexture?
err = CVMetalTextureCacheCreateTextureFromImage(
kCFAllocatorDefault,
metalTextureCache, // object prop
pixelBuffer,
nil, // texture attributes
pixelFormat,
exportSettings.width,
exportSettings.height,
0, // planeIndex
&optionalMetalTexture)
guard err == noErr, let metalTexture = optionalMetalTexture else {
fatalError("Failed to create metal texture wrapper from pixel bufffer \(err)")
}
Attempt: Change the texture descriptor
Since I'm creating my Metal texture from a CVPixelbuffer with CVMetalTextureCacheCreateTextureFromImage, I can't figure out how to set its attributes and make it multi sample.
Attempt: Try H264
Didn't change anything. Also tried changing just the alpha quality, with HEVC with alpha, but no change.
Attempt: Enable multi sampling
I was able to get my pipeline to pick up that I wanted multi sampling, but it crashes due to the texture not being set up for multisampling (more precisely a MTLTexture of type .2DMultisample (docs)
Attempt: Copy the MTLTexture created by Core Video
I tried to use a MTLBlitCommandEncoder to copy the texture I was given by Core Video into a texture I had set up with the right attributes. But it crashes telling me that the attributes don't match.
I'm starting to think there's no solution to this?
