How can I (efficiently) test an SKTexture for visible content?

Question

My app currently animates thousands of "mini textures" with SpriteKit. The conversion process for each object of interest is UIView to UIImage to SKTexture to many, many, very small SKTextures (miniTextures). What I'm having trouble with is how to further process only those miniTextures that are "useful."

Many of the miniTextures, coming from blank parts of an object/view/image/texture, contain no useful visual information (i.e., they're blank). As such, it would be wasteful to animate them. The problem is, I don't know of any good, efficient test for this. I had hoped to somehow be able to examine each texture's pixels, and if every pixel in a texture had an alpha value of 0, that texture would not get processed (i.e., no SKSpriteNode would get created for it).

As a workaround, I tried creating an SKPhysicsBody for each miniTexture (see code below), using SKPhysicsBody(texture: alphaThreshold: size:), which returns nil when nothing in the texture exceeds the alphaThreshold. That then might have at least allowed me to test for and selectively process the useful miniTextures. But when doing that, I ran into two problems: Firstly, the simulator slowed to a virtual standstill. Secondly, when making the textures very small, all of them were determined to be nil, regardless of my alphaThreshold.

Insights? Solutions? Sympathy?

for j in stride(from: 0, to: 1, by: yFraction) {
    for i in stride(from: 0, to: 1, by: xFraction) {

        let miniTexture = SKTexture(rect: CGRect(x: CGFloat(i), 
                                                 y: CGFloat(j), 
                                                 width: CGFloat(xFraction), 
                                                 height: CGFloat(yFraction)), 
                                    in: texture)

        spriteNode.physicsBody = SKPhysicsBody(texture: miniTexture, 
                                               alphaThreshold: 0.5, 
                                               size: CGSize(width: 1, 
                                                            height: 1))

        // BUT WHEN THE miniTextures ARE VERY SMALL, THESE ALL ARE NIL!
        // AND THE ANIMATION COMES TO A VIRTUAL STANDSTILL!
        if spriteNode.physicsBody == 0 { continue }

        // spriteNode.position details here
        // spritNode.run(specificAnimation) details here

        let spriteNode = SKSpriteNode(texture: miniTexture)
        spriteNodes.addChild(spriteNode)
    }
}

Well first of all, why is the view empty? is that because its subviews array is empty? thats the obvious case to skip. the other obvious case to skip is when all of your subviews frames are outside of your bounds. Otherwise its very expensive to do what you are proposing. You can make a shader to check all of the pixels, but by the time you have done that you have still allocated the texture and copied all of that memory, so its much more efficient to just no handle the view in the first place. — Josh Homann
Thanks, @JoshHomann, for the reply. I'm not sure what your first three sentences are about. No view is ever empty. Only parts of it are blank. Imagine, for example, a line-art drawing—lots of black lines, but also lots of plain-white areas. If I were to chop up such a drawing into very small squares and then move them around on a white background, I'd rather not waste resources moving around the white squares. Your final sentence seems to suggest that checking squares, though, would be too costly. By "handle the view," do you mean determine what square will be blank before making textures?? — Optimalist
It sounds like what you are doing is decomposing a single view into many smaller textures. You should not do this; its incredibly inefficient. You should make one texture out of the view and then use the texture coordinates so that each node represents a small subsection of the original texture (ie make it a texture atlas). This is much more efficient since you are not requiring the GPU to allocate additional textures and to constantly switch context. In this case the cost of overdrawing the "blank" areas is likely far less than checking if they are blank. — Josh Homann
Actually, the animation runs pretty well with thousands of textures when I don't bother checking them. I just wanted not to melt my GPU. But are you saying that parts of a texture can be independently moved around—as if they were separate textures?! I just read about SKTextureAtlas, but don't see how. Seems like I'd still have to create thousands of SKTextures, even when using SKTextureAtlas(es). To see the approach I was attempting (using images I create at runtime), see "Attempt 3" at this link. Frankly, I didn't understand all of it.... — Optimalist
You are already doing the same thing that SKTextureAtlas is doing, SKTexture(rect: CGRect(x: CGFloat(i), y: CGFloat(j), width: CGFloat(xFraction), height: CGFloat(yFraction)), in: texture) is creating the subtexture that the atlas would. — Knight0fDragon

Optimalist Optimalist · Accepted Answer · 2018-05-29T08:13:16

As it turns out, SpriteKit already culls invisible and offscreen nodes by default, which may be why my app didn't perform poorly without testing for and removing fully transparent SKSpriteNodes. (But maybe my understanding is wrong, if by "invisible" Apple merely means "set to isHidden.")

Apple's documentation of the shouldCullNonVisibleNodes instance property (default value is true) is here.

A better, upstream solution would be to test for invisibility before creating the images/textures, but that would involve a deeper understanding of low-level iOS graphics manipulation than I currently have.

How can I (efficiently) test an SKTexture for visible content?

1 Answers