How can I simulate hand rays on HoloLens 1?

Question

I'm setting a new project which is intended to deploy to both HoloLens 1 and 2, and I'd like to use hand rays in both, or at least be able to simulate them on HoloLens 1 in preparation for HoloLens 2.

As far as I have got is:

Customizing the InputSimulationService to be gesture only (so I can test in editor)
Adding the GGVHand Controller Type to DefaultControllerPointer Options in the MRTK/Pointers section.

This gets it to show up and respond to clicks both in editor and device, but it does not use the hand coordinates and instead raycasts forward from 0,0,0, which suggests that the GGV Hand Controller is providing a GripPosition (of course with no rotation due to HL1) but not providing a Pointer Pose.

I imagine the cleanest way to do this would be to add a pointer pose to the GGV Hand controller, or add (estimated) rotation to the GripPosition and use this as the Pose Action in the ShellHandRayPointer. I can't immediately see where to customize/insert this in the MRTK.

Alternatively, I could customize the DefaultControllerPointer prefab but I am hesitant to do so as the MRTK seems to still be undergoing frequent changes and this would likely lead to upgrade headaches.

Julia Schwarz Julia Schwarz · Accepted Answer · 2019-06-19T10:03:46

You could create a custom pointer that would set the pointer's rotation to be inferred based on the hand position, and then like you suggested use Grip Pose instead of Pointer Pose for the Pose Action.

The code of your custom pointer would look something like this:

// Note you could extend ShellHandRayPointer if you wanted the beam bending, 
// however configuring that pointer requires careful setup of asset.
public class HL1HandRay : LinePointer
{
    public override Quaternion Rotation  
    {
        get
        {
            // Set rotation to be line from head to head, rotated a bit
            float sign = Controller.ControllerHandedness == Handedness.Right ? -1f : 1f;
            return Quaternion.Euler(0, sign * 35, 0) * Quaternion.LookRotation(Position - CameraCache.Main.transform.position, Vector3.up);
        }
    }

    // We cannot use the base IsInteractionEnabled
    // Because  HL1 hands are always set to have their "IsInPointing pose" field as false
    // You may want to do more thorough checks here, following BaseControllerPointer implementation
    public override bool IsInteractionEnabled => IsFocusLocked || IsTracked;
}

Then create a new pointer prefab and configure your pointer profile to use the new pointer prefab. Creating your own prefab instead of modifying MRTK prefabs has advantage of ensuring that MRTK updates will not overwrite your prefabs.

Here's some captures of the simple pointer prefab I made to test this with relevant changes highlighted:

And then the components I used:

How can I simulate hand rays on HoloLens 1?

1 Answers