Pico Motion Trackers in Unity

As of 25-11-2024, there is a mounting solution available for Pico Motion Trackers for the SenseGlove Nova 2. This solution allows one to mount Motion Trackers consistently onto the glove to track the wrist movement.

../_images/picoMT-Nova2.jpg

As per Pico’s documentation at the time of writing, Pico Motion Trackers are compatible with the following headsets:

  • Pico Neo 3

  • Pico 4

  • Pico 4 Ultra Devices

For the most up-to-date information, refer to Pico’s official website .

Using Pico Motion Trackers in your Unity Project

As of writing this guide, there is no way to collect the location of a Pico Motion Tracker via the UnityEngine XR Plugin Management. Unlike Vive (Wrist) Trackers, Pico Motion trackers do not show up in the UnityEngine.InputDevices.GetDevices(List<InputDevice> inputDevices); list.

As such, wrist tracking using Pico Motion Trackers does NOT work ‘out of the box’ within the SenseGlove Unity Plugin. You will need to import the “PICO Unity Integration SDK” package into your unity project, which is available here.

PXR_Manager and Body Tracking

Create a new XR Project using Unity’s XR System.

Add the PXR_Manager script component to a GameObject within your level, and set “Body Tracking” to true

../_images/PXR_Manager.png

Move GameObjects using PXR_MotionTracking

Pico provides an API through which one can obtain tracker locations. You can use your own script to manage their updates, or you can use the following code provided by SenseGlove:

Note

The built-in offsets used by the SenseGlove Plug-in are hard-coded, and assume you are using the PXR_MotionTracking API as provided by Pico on their Motion Tracker reference page. If you’re using a different API to track your Motion Trackers, we cannot guarantee that the wrist location will be correct.

Create a new script called SG_PicoMTObjects.cs, and code below for its content.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
using Unity.XR.PXR;
using UnityEngine;

/*
 * Using reference code from Pico XR, this script updates (up to) two Pico Trackers for a left and right hand tracking, for use with SenseGlove Hand Tracking.
 * https://developer.picoxr.com/document/unity/object-tracking/
 *
 * author: max@senseglove.com
 */

/// <summary> A script that animates two GameObjects based on Pico's Motion Tracker API. For use in Wrist Tracking for SenseGlove. </summary>
public class SG_PicoMTObjects : MonoBehaviour
{

    //---------------------------------------------------------------------------------------------
    // Variables

    /// <summary> If this variable is assigned, we will move the TrackingDevice's relative to this one. Otherwise, we set the LocalPostion instead. </summary>
    public Transform xrOrigin;

    [Header("Left Wrist Tracking")]
    public Transform leftHandTrackingDevice;
    public MotionTrackerNum trackerIndexLeftHand = MotionTrackerNum.ONE;

    [Header("Right Wrist Tracking")]
    public Transform rightHandTrackingDevice;
    public MotionTrackerNum trackerIndexRightHand = MotionTrackerNum.TWO;

    //---------------------------------------------------------------------------------------------
   // Functions

    public void UpdateTrackerLocations()
    {
        MotionTrackerMode trackingMode = PXR_MotionTracking.GetMotionTrackerMode();
        if (trackingMode == MotionTrackerMode.MotionTracking)
        {
            MotionTrackerConnectState mtConnectStates = new MotionTrackerConnectState();
            PXR_MotionTracking.GetMotionTrackerConnectStateWithSN(ref mtConnectStates);
            if (mtConnectStates.trackerSum > 0) //there is at least 1 tracker
            {
                UpdateTransform(leftHandTrackingDevice, trackerIndexLeftHand, xrOrigin, ref mtConnectStates);
                UpdateTransform(rightHandTrackingDevice, trackerIndexRightHand, xrOrigin, ref mtConnectStates);
            }
        }
    }

    public static void UpdateTransform(Transform obj, MotionTrackerNum trackerNum, Transform xrOrigin, ref MotionTrackerConnectState mtConnectState)
    {
        if (trackerNum == MotionTrackerNum.NONE)
            return;
        if (obj == null)
            return;
        int trackingIndex = ToTrackingIndex(trackerNum);
        if (trackingIndex < 0 || trackingIndex >= mtConnectState.trackerSum)
            return; //In case there's less trackers or ToTrackingIndex somewhow still returns an invalid index

        string sn = mtConnectState.trackersSN[trackingIndex].value.ToString().Trim();
        if (string.IsNullOrEmpty(sn))
            return;

        MotionTrackerLocations locations = new MotionTrackerLocations();
        MotionTrackerConfidence confidence = new MotionTrackerConfidence();
        // Retrieve the location of PICO Motion Tracker
        if (PXR_MotionTracking.GetMotionTrackerLocations(mtConnectState.trackersSN[trackingIndex], ref locations, ref confidence) == 1)
            return; //if this function returns 1, it was a failure. (A bit intuitive because it's usualy 0 = false, 1 = true)

        MotionTrackerLocation location = locations.localLocation; //This is relative to the Headset, but not to the player...
        Vector3 localPosition = location.pose.Position.ToVector3();
        Quaternion localRotation = location.pose.Orientation.ToQuat();
        if (xrOrigin == null)
        {
            obj.localRotation = localRotation;
            obj.localPosition = localPosition;
        }
        else
        {
            obj.localRotation = xrOrigin.rotation * localRotation;
            obj.localPosition = xrOrigin.position + (xrOrigin.rotation * localPosition);
        }
    }


    public static int ToTrackingIndex(MotionTrackerNum num)
    {
        return ((int)num) - 1;
    }


    //---------------------------------------------------------------------------------------------
    // Monobehaviour


    private void Start()
    {
        if (trackerIndexLeftHand != MotionTrackerNum.NONE)
            PXR_MotionTracking.CheckMotionTrackerModeAndNumber(MotionTrackerMode.MotionTracking, trackerIndexLeftHand);
        if (trackerIndexRightHand != MotionTrackerNum.NONE)
            PXR_MotionTracking.CheckMotionTrackerModeAndNumber(MotionTrackerMode.MotionTracking, trackerIndexRightHand);
    }

    private void Update()
    {
        UpdateTrackerLocations();
    }
}

You will need to assign two individual GameObjects as the leftHandTrackingDevice and rightHandTrackingDevice via the Inspector. These will represent the Pico Motion Trackers assigned to the left hand and right hand.

The “trackerIndex” variables control which of Pico’s trackers to access for the corresponding hand. Unfortunately, the Pico API does not allow you to assign a left and right hand to a tracker. The reccomended indices are “ONE” and “TWO”. If you accidentally assign the wrong tracker index, it is easy to disconnect and swap the hardware on your glove.

The PXR_MotionTracking API gives us the tracker location in the “play area”. You should assign this script an xrOrigin so the GameObjects move with your XR Rig. Alternatively, you can make sure your GameObject(s) are children of the XRRig instead.

../_images/PicoMTObjects_inspector.png