Unity Wrist Tracking

SenseGlove devices have no built-in wrist tracking. So we have different ways of linking controllers and/or trackers to the SenseGlove tracking scripts within Unity.

There are two parts to tracking our gloves in VR:

  1. Determining the (world) position & rotation of a tracked device.

  2. Determining which offsets to use to calculate the wrist location.

Step 1 is achieved using the plugins or packages relevant for the specific tracked device. If you can get your controller or tracker to show up at the appropriate location in your Scene, you can get the SenseGlove wrist tracking up and running.

Step 2; determining which offsets to use, is dependent on which tracked device you have, as every one of these has a different shape and therefore a different tracker mount. For the most part, SenseGlove can automatically determine which device you are using, based on the device name (“Auto Offsets”). Alternatively, you can hard-code which device offsets to use (“Manual Offsets”).

Wrist Tracking Options

Method

Description

UnityXR

Use UnityXR’s positioning System with automatically offsets. Assign an “Origin” to move the hands with the play area.

UnityXRManualOffsets

Use UnityXR’s positioning system, but use predetermined offsets.

FollowObjectAutoOffsets

Follow a specific transform, with automatically detected offsets.

FollowObjectManualOffsets

Follow a specific transform, but using predetermined offsets.

FollowObjectNoOffsets

Follow a GameObject without using any offsets. Useful to override any wrist location(s) in example scenes.

MyGameObject

This GameObject’s own Transform represents the wrist rotation & position. Useful when using parenting.

Setting up wrist tracking

Your main interface as of Unity Plugin v2.0+ to set up these options is the SG_HapticGlove script, which is where hand tracking from the gloves is combined with wrist tracking through 3rd party devices.

../_images/wristTracking.png

Using Unity XR Plugin Management

This is the easiest way to set up hand tracking, provided your project uses Unity’s built-in XR Plugin Management package. You can check its installation instructions here. Do note that the Unity XR Plugin Management package is available in Unity Editor 2019.1 and higher.

If this package is installed along with the appropriate XR Device package, the SenseGlove Plugin can automatically detect any XR devices it is compatible with. For Oculus, this is as simple as checking the ‘oculus’ option in the Player Settings > XR Plugin Management.

../_images/xrPluginManagement.png

For other XR devices, you’ll need to either place files inside your project’s Packages folder, or link to an online registry. There’s packages available for:

Note

Unity’s XR Plugin Management has different tabs for each platforms. If you are developing on Windows, ensure you have the appropriate XR package selected in the windows tab. If you are deploying to Android, make sure you also have the appropriate XR package selected in the Android tab.

If you have both the XR Plugin management and your device’s package installed, you can use any of the following Wrist Tracking Methods on your SG_HapticGlove:

  • Unity XR (Auto Offsets) (Default)

  • Unity XR Manual Offsets

Assign the Origin!

By “Origin” we mean the GameObject that represents your player or play area location inside your scene, usually called the “XRrig”. When using UnityXR to track devices, we unfortunately only have access to the position and rotation inside your play area. If your XR Rig does not have a position of [0,0,0] and rotation of [0, 0, 0], or when the play area moves through the scene, your hands will not end up in the correct location. By assigning the Origin GameObject of your SG_HapticGlove script, the tracking will move with your play area.

Using OpenXR to track devices

When you are using OpenXR to handle your input, and not your device’s appropriate XR package, you will need to take a few thing into account:

  • OpenXR obscures the name of your device(s) (for example “Quest 2 HMD” becomes “OpenXR Head Tracking”), so the SenseGlove API is unable to automatically detect what XR device is being used. Until we have a workaround for this, you will need to manually assign which tracking offsets to use for your build(s).

  • OpenXR’s origin on controllers may not be the same as those on the device-specific Package, resulting on your hands ending up in the wrong location despite the correct device offsets being used. in this case, you will need to determine the appropriate offsets yourself - for example, by making the object a Child of your controllers.

Make it follow a tracked GameObject

Instead of relying on UnityXR to handle wrist tracking, you can also link the SG_HapticGlove to a GameObject in the scene. This method is useful if you already have your XR Rig and controller tracking up and running. Especially useful if you are not using UnityXR Plugin Management to track your devices. Since we assume that your controllers are set up to move with your play area, there’s no need to assign an Origin like in the UnityXR wrist tracking methods.

The difference between this method and making the script a child of your tracked object is that you can dynamically change offsets: There’s no need to have the script at the correct location on startup. It lends itself well for Automatic Offsets.

  • Follow Object Auto Offsets

  • Follow Object Manual Offsets

  • Follow Object No Offsets [deprecated - use My GameObject instead].

Make it a child of tracked object

The easiest, but least scalable solution: Using the position & rotation of the GameObject that the SG_TrackedHand is attached to. You will need to defermine your own hand location relative to the controllers.

Alternatively, you can also use this method to manually move your hands around in editor, when you do not have access to an XR Device.

  • My GameObject

Tips & Tricks

Meta Quest Pro

In as late as Unity Editor version 2020, a Meta Quest pro will falsely identify itself as an “Oculus Quest 2” to Unity’s XR Plugin Management package, but only when building for Android. This is either due to a bug in the Meta API, or because this Unity Editor version (and the versions before it) use an older version of their API (before a Meta Quest Pro was launched). Either way, this will likely result in your hands ending in the wrong location.

The way to solve this is to eplicitly tell the SG_HapticGlove script that you are building for Meta Quest Pro. To do so, set your Wrist Tracking Mode to anything ending in “ManualOffsets” (UnityXR_ManualOffsets or FollowObject_ManualOffsets). Then, set your wristTrackingOffsets parameter to QuestProController.

Until Meta fixes this bug, you will unfortunately have to make a build for Quest 2 and Quest Pro separately.

Vive Focus 3 Trackers

When using the Vive Wave SDK for UnityXR Plugin management to set up your Trackers, you will need to change your project settings: If you have the Vive Wave SDK imported, you should find a “WaveXRSettings” menu under the “XR Plugiin Management”.

  • Under “Tracker”, make sure “Enable Tracker” is checked. If it is not, you won’t be able to use the Wrist Trackers.

  • Under “Natural Hand”, make sure “Natural Hand” is checked. If it is not, your app will demand you switch to controllers to start it.

../_images/viveWaveSDK.png

HTC Vive Trackers

This is a known issue with the OpenVR plugin for UnityXR: When your trackers are set to the “held in hand” role, introduced as the new default in SteamVR v1.22, the plugin returns a different rotation for all other settings. Setting your Vive Tracker Role to any other role fixes this issue.

With your tracker(s) connected and SteamVR running, right-click on any tracker icon, and select the “Manage Trackers” option. This will open an options window. In this window, click on a second “Manage Trackers” button to open the tracker roles window. Click on the “Tracker Role” dropdown menu, and assign any role other than “Held in Hand”. You might need to restart SteamVR for this change to be applied.