ARKit: Plot a Node at a specific pixel at a specific Z distance from Camera

Question

Referring to the image above. I have a Red Node at the center of the screen with a distance of 1.0 unit (1 meter away) [See iPhone Portrait Top View]

What I do is I capture a screenshot of the iPhone screen and the resulting image is 750 x 1334 pixels [See iPhone Portrait Front View]

sceneView.snapshot()

What I want to do is put 4 Red Square Nodes located on the four sides of the iPhone screen relative to the Red Circle (at the dead center of the screen). I am making this to mark where I did a snapshot. What I want to know is how can I plot a box node precisely at a certain x,y point given z distance. (The value of Z is not fixed, I just used 1.0 as a sample scenario).. I want to plot (0,0), (750,0), (0, 1334) and (750, 1334) at a given z of 1.0 and assuming I am on a tripod, the plotted nodes would appear on the four sides of my iPhone screen.

I am very terrible at math and this problem is so complicated for me to solve alone with my current math skills. Can anyone help? Please?

Are you just trying to make sure the red node is exactly 1m away? — BlackMirrorz
Your question is not clear enough. What exactly is it that you are trying to achieve. You stated that What I want to do is plot a frame between the screenshot I took. What do you mean by plot a frame? plot a frame between the screenshot and what? Please update your question with some clarifications. Also, please share what you have tried so far — Malik
@JoshRobbins i updated my question. on my example its 1 meter away, but it may vary depending on the object... (maybe 0.5 or even 2 meters away etc) — Chinchan Zu
@Malik hello, I updated my question to clarify. What im trying now is doing I hard code numbers and do tests but sometimes its off so i think im doing my computations wrong so im asking if you guys know the math to solve this problem. thanks! — Chinchan Zu

Jose Paredes Jose Paredes · Accepted Answer · 2018-06-26T07:55:08

Since you need ARnchor nodes (to mark where a snapshot was done) using the real time information from the camera instead of a snapshot would be probably easier. In special due to pixels in a 2D image are referenced from left to right and from top to bottom (with "0,0" coords located in the top left side) ... and we know the AR nodes are referenced in 3D coordinates with the center of the local node as the 0,0,0 coords.

I haven't yet code to test but I think the following properties should help:

    let pPoint  = sceneView.projectPoint(self.centerBall.position)
    let fieldW  = sceneView.session.currentFrame?.camera.imageResolution.width
    let fieldH  = sceneView.session.currentFrame?.camera.imageResolution.height

The "pPoint" should return the 2D coords corresponding to the (0,0,0) 3D coords of "centerBall" from there it should be just add or subtract calculations to obtain all 4 corners in 2D

Finally passing the 2D coords of every corner to the "unprojectPoint(:)" method should provide the 3D "world" coords and that can be converted to "centerBall" coordinates with the "convertPosition( position: SCNVector3, from node: SCNNode?)" method

It seems interesting so will try to code this before weekend

At the end I suspect ARanchor nodes may not be 100% stables

ARKit: Plot a Node at a specific pixel at a specific Z distance from Camera

1 Answers