ARKit - ARRreferenceImage tracking

Question

I'm playing around with ARReferenceImages in ARKit and I'm trying to add an SCNNode when a reference image is recognised and then leave that node in place regardless of whether the same reference image is then recognised elsewhere.

I can add my SCNode correctly, but if I move my marker it picks it up again and moves my placed node to the position of the marker.

My code to add is as follows:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
        guard let imageAnchor = anchor as? ARImageAnchor else { return }
        let referenceImage = imageAnchor.referenceImage
        print("MAPNODE IS NIL = \(self.mapNode == nil)")
        updateQueue.async {
            if self.mapNode == nil {

            // Create a plane to visualize the initial position of the detected image.
            let plane = SCNPlane(width: 1.2912,
                                 height: 1.2912)
            let planeNode = SCNNode(geometry: plane)
            planeNode.opacity = 1

            /*
             `SCNPlane` is vertically oriented in its local coordinate space, but
             `ARImageAnchor` assumes the image is horizontal in its local space, so
             rotate the plane to match.
             */
            planeNode.eulerAngles.x = -.pi / 2

            self.mapNode = planeNode
            /*
             Image anchors are not tracked after initial detection, so create an
             animation that limits the duration for which the plane visualization appears.
             */
            // Add the plane visualization to the scene.
            node.addChildNode(planeNode)
            }
        }
 }

reading the docs here https://developer.apple.com/documentation/arkit/recognizing_images_in_an_ar_experience#2958517 it states that

Apply Best Practices

This example app simply visualizes where ARKit detects each reference image in the user’s environment, but your app can do much more. Follow the tips below to design AR experiences that use image detection well.

Use detected images to set a frame of reference for the AR scene. Instead of requiring the user to choose a place for virtual content, or arbitrarily placing content in the user’s environment, use detected images to anchor the virtual scene. You can even use multiple detected images. For example, an app for a retail store could make a virtual character appear to emerge from a store’s front door by recognizing posters placed on either side of the door and then calculating a position for the character directly between the posters.

Note

Use the ARSession setWorldOrigin(relativeTransform:) method to redefine the world coordinate system so that you can place all anchors and other content relative to the reference point you choose.

Design your AR experience to use detected images as a starting point for virtual content. ARKit doesn’t track changes to the position or orientation of each detected image. If you try to place virtual content that stays attached to a detected image, that content may not appear to stay in place correctly. Instead, use detected images as a frame of reference for starting a dynamic scene. For example, your app might recognize theater posters for a sci-fi film and then have virtual spaceships appear to emerge from the posters and fly around the environment.

So I tried setting my world transform to be equal to the transform of my image anchor

 self.session.setWorldOrigin(relativeTransform: imageAnchor.transform)

However my mapNode follows the imageAnchor where ever it moves. I haven't implemented the renderer update method so I'm not sure why this keeps moving.

I'm assuming that the setWorldOrigin method is constantly updating to the imageAnchor.transform and not just that moment in time, which is weird as that code is only called once. Any ideas?

BlackMirrorz BlackMirrorz · Accepted Answer · 2018-03-24T02:07:56

If you want to add the mapNode at the position of the ARImageAnchor you could set the position of your mapNode at the transform of the ARImageAnchor and add it to the scene but not linked to the reference image if that makes sense.

This could be done like so:

 func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {

        //1. If Out Target Image Has Been Detected Than Get The Corresponding Anchor
        guard let currentImageAnchor = anchor as? ARImageAnchor else { return }

        //2. An ImageAnchor Is Only Added Once For Each Identified Target
        print("Anchor ID = \(currentImageAnchor.identifier)")

        //3. Add An SCNNode At The Position Of The Identified ImageTarget
        let nodeHolder = SCNNode()

        let nodeGeometry = SCNBox(width: 0.02, height: 0.02, length: 0.02, chamferRadius: 0)
        nodeGeometry.firstMaterial?.diffuse.contents = UIColor.cyan
        nodeHolder.geometry = nodeGeometry

        nodeHolder.position = SCNVector3(currentImageAnchor.transform.columns.3.x,
                                         currentImageAnchor.transform.columns.3.y,
                                         currentImageAnchor.transform.columns.3.z)

        augmentedRealityView?.scene.rootNode.addChildNode(nodeHolder)

 }

In another part of your question you seem to imply that you want to detect multiple occurrences of the same image. I could be wrong but I think the only way to do this is to remove the corresponding ARImageAnchor for the reference image, which can be done like so (by adding it at the end of the last code snippet):

 augmentedRealitySession.remove(anchor: currentImageAnchor)

The issue here is that once the ARImageAnchor is removed, any time it is detected again, you would have to handle whether content should be added, which is tricky since the ARImageAnchor.identifier is always the same for the referenceImage regardless of whether it is removed and then re-added thus making it difficult to store in a dictionary etc. As such depending on your needs you would then need to find a way to determine if content existed at that location and whether to re add it etc.

The last part of your question about setWorldOrigin seems a bit odd like you said, but maybe you could add a Bool to prevent it from potentially changing e.g:

var hasSetWorldOrigin = false

Then based on this you could ensure that it is only set once e.g:

func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {

        //1. If Out Target Image Has Been Detected Than Get The Corresponding Anchor
        guard let currentImageAnchor = anchor as? ARImageAnchor else { return }


        //2. If We Havent Set The World Origin Set It Based On The ImageAnchorTranform
        if !hasSetWorldOrigin{

            self.augmentedRealitySession.setWorldOrigin(relativeTransform: currentImageAnchor.transform)
            hasSetWorldOrigin = true

            //3. Create Two Nodes To Add To The Scene And Distribute Them
            let nodeHolderA = SCNNode()
            let nodeGeometryA = SCNBox(width: 0.04, height: 0.04, length: 0.04, chamferRadius: 0)
            nodeGeometryA.firstMaterial?.diffuse.contents = UIColor.green
            nodeHolderA.geometry = nodeGeometryA

            let nodeHolderB = SCNNode()
            let nodeGeometryB = SCNBox(width: 0.04, height: 0.04, length: 0.04, chamferRadius: 0)
            nodeGeometryB.firstMaterial?.diffuse.contents = UIColor.red
            nodeHolderB.geometry = nodeGeometryB

            if let cameraTransform = augmentedRealitySession.currentFrame?.camera.transform{
                nodeHolderA.simdPosition =  float3(cameraTransform.columns.3.x,
                                                   cameraTransform.columns.3.y,
                                                   cameraTransform.columns.3.z)

                nodeHolderB.simdPosition =  float3(cameraTransform.columns.3.x + 0.2,
                                                   cameraTransform.columns.3.y,
                                                   cameraTransform.columns.3.z)
            }


            augmentedRealityView?.scene.rootNode.addChildNode(nodeHolderA)
            augmentedRealityView?.scene.rootNode.addChildNode(nodeHolderB)
        }

}

Hopefully my answer will provide a useful starting point to you assuming of course I have interpreted your question correctly,

ARKit - ARRreferenceImage tracking

1 Answers