Where is the .camera AnchorEntity located?

Question

When adding a child to my AnchorEntity(.camera), it appears as if the child is spawning behind my camera (meaning I can only see my child when I turn around). I have also tried to add a mesh to my Anchor directly but unfortunately ARKit / RealityKit does not render the mesh when you are inside of it (which because its centered around the camera, is theoretically always the case. However, it could also be the case that its always located behind the screen [where the user is] and I'm never able to see it).

Also, oddly enough the child entity does not move with the camera AnchorEntity despite setting the translation transform to (0,0,0).

My two questions are:

Is the .camera anchor actually located right where the physical iPad / camera is located or is it located further back (perhaps where the user would normally hold the iPad)?
How do you get a child entity of the AnchorEntity(.camera) to move as the iPad / camera moves in real space?

Andy Jazz Andy Jazz · Accepted Answer · 2020-02-26T14:52:44

Question I. Is the .camera anchor actually located right where the physical iPad / iPhone camera is located or is it located further back (perhaps where the user would normally hold the iPad / iPhone)?

Answer I

In RealityKit and ARKit frameworks ARCamera has a pivot point like other entities (nodes) have, and it's located at the point where lens is attached to the camera body (at bayonet level). This pivot can tether AnchorEntity(.camera). In other words, virtual camera and real-world camera have that pivot point approximately at the same place.

So, if you attach RealityKit's AnchorEntity to a camera's pivot, you place it to the coordinates where camera's bayonet is located. And this AnchorEntity(.camera) will be tracked automatically without a need to implement session(_:didUpdate:) method.

However, if attach ARKit's ARAnchor to the camera's pivot, you have to implement session(_:didUpdate:) method to constantly update a position and orientation of that anchor for every ARFrame.

Question II. How do you get a child entity of the AnchorEntity(.camera) to move as the iPad / camera moves in real space?

Answer II

If you want to constantly update model's position in RealityKits at 60 fps (when ARCamera moves and rotates) you need to use the following approach:

import ARKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let box = MeshResource.generateBox(size: 0.25)            
        let material = SimpleMaterial(color: .systemPink, isMetallic: true)
        let boxEntity = ModelEntity(mesh: box, materials: [material])
        
        let cameraAnchor = AnchorEntity(.camera)       // ARCamera anchor
        cameraAnchor.addChild(boxEntity)
        arView.scene.addAnchor(cameraAnchor)
        
        boxEntity.transform.translation = [0, 0,-1]    // Box offset 1 m
    }
}

...Or you could implement ARKits currentFrame property inside session(_:didUpdate:) method:

extension ViewController: ARSessionDelegate {

    func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {

        guard let transform = arView.session.currentFrame?.camera.transform
        else { return }
    
        let arkitAnchor = ARAnchor(transform: transform)

        let anchor = AnchorEntity(anchor: arkitAnchor)
        anchor.addChild(boxEntity)
        arView.scene.addAnchor(anchor)
    }
}

class ViewController: UIViewController {

    @IBOutlet var arView: ARView!
    var boxEntity = ModelEntity(...)

    override func viewDidLoad() {
        super.viewDidLoad()
        arView.session.delegate = self                 // Session's delegate
    }
}

Where is the .camera AnchorEntity located?

1 Answers

Answer I

Answer II