Finding a fixed point in mesh on Microsoft HoloLens

3
votes

I have been trying to find a fixed point in a mesh during runtime of HoloLens application (mesh is generated by Microsoft HoloLens) that I can relate to after relaunching the application, but with no success. My goal is to find a fixed point in a mesh that I will relate to when placing prefabs saved from last application run. I will use that point as origin for placing the prefabs at their previous positions in the mesh, thus displaying them where they were originally placed in the mesh.

I have tried to do the following, with incorrect results:

  • save a LocalToWorld / WorldToLocal Unity4x4 Matrix of the transofrm component of SpatialMapping GameObject and perform Matrix4x4.MulitplyPoint() on positions of prefabs
  • transform positions of prefabs relative to local center of the generated mesh
  • transform positions of prefabs relative to local center of the first submesh

Note that I won't change the mesh (disabling update of spatial surfaces) once I place the prefabs, even in future application runs.

The only constraint is that I can't use WorldAnchors, since I need to be able to edit the position of those prefabs outside of MixedReality application (specifically in a WebGL application) and then display them at the correct edited position in the HoloLens application in the mesh.

1
unity3dmeshhololensunity-webglwindows-mixed-reality
Is this mesh constructed from an object in the physical space? If it is not there is no way you can find a transformation between two sessions. I have never used Hololens but i am assuming it changes its coordinate system every time you run the app. Therefore, you need something which is common in both sessions like a marker. You can then place your objects relative to that marker in local space. – Ali Kanat
Yes,@Ali Kanat, the mesh is constructed from an object in a physical space - it is a room / hall scanned by HoloLens. Origin of coordinate system is defined on camera (user) position on every application run as you said. Making a marker (or in general a common point between sessions) would work, but as I said, I can't find such a fixed point. – n0DruidsH3r3
Well you can use the marker as fixed point then. You can create an empty game object at marker position and rotation and place your model as a child to that. The only thing you need to be careful is pivot point of your model. If you want a better solution you can calculate a Transformation matrix between two Sessions using common points on Marker(lets say corners of the marker) Then transform each point using Matrix4x4.MultiplyPoint(). I would prefer second option because with that one you can scan, create model and transform it in Run time. – Ali Kanat
What do you mean by a marker and how to find a place for it? The problem is that HoloLens during runtime don't have the model (scanned mesh) saved as a whole piece. Instead, the SpatialMappingSource contains list of SurfaceObjects and each has a mesh associated with it, so they together form the model. – n0DruidsH3r3
As far as i know you can use markers in Hololens. So you have to place the marker on a wall or any surface in the place you scan. Then in another app or another scene you walk to that marker detect it or track it however it works in Hololens, Calculate transformation and then instantiate the model. Then translate and rotate the mesh. You can do this in two ways you can either translate vertices in the meshes or you can translate and rotate the whole 3D model. In option two be careful with the pivot though. The reason for using a marker is having common points in scanning and showing model part – Ali Kanat

1 Answers

1
votes

It's precisely an anchoring task. There is no fixed point in the mesh.

  1. You need to place anchors in world. Then get GetNativeSpatialAnchorPtr
  2. SpatialAnchor has TryCreateRelativeTo() method. Use it for creating edited location.
  3. Assign SetNativeSpatialAnchorPtr() from the edited location.