Learning Exercises

These tasks cover topics that require a basic scripting- and editor experience with the Unity Engine, and is meant for those who have already imported the SenseGlove Unity Plugin, and set up their project for Virtual Reality. If these do not apply to you, we recommend you check out our getting started guide and/or some basic Unity tutorials.

Note

These examples were made for plugin version v2.4.1 and above. If you are getting compiler errors, you’re likely using an older version of our Unity plugin.

Task 1: Grab and feel objects

Task 1.1: Make a basic Grabable

Create a simple cube in your scene (GameObject > 3D Object > Cube). Adjust its scale to make it a comfortable size to grab. A 6cm cube with a scale of [0.06, 0.06, 0.06], for example.

Add a SG_Grabable script Component to the cube via the Inspector. This component marks the cube as something that can be picked up and moved around by our GrabScripts. Leave the parameters unchanged for now.

Next, add a SG_Material script Component to the cube via the Inspector. This component defines the cube as something that provides force-feedback, which is picked up by our HandFeedback Layer. Leave its parameters at their default values.

Optionally, you can add a RigidBody component to the object, so it becomes affected by Gravity.

If all want well, your object will look something like this:

Tip

If you have a Unity GameObject made up of multiple GameObjects with (primitive) colliders, addding a Rigidbody to their parent GameObject will have the physics engine consider them as one single collider / body. Add your SenseGlove Scripts to this parent object to apply material- and grab components to all of them at once. See also: Compound Colliders.

If your Unity project is set up with your desired VR plugin and the blue SenseGlove prefab hands, you should now be able to grab the cube by putting it between your thumb and your index- or middle finger. You will feel the Force-Feedback activate when your virtual fingers touch it.

Tip

Don’t have access to a VR headset? Or do you want precise control over hand and finger tracking(s)? The SG_ManualPoser script allows you to control the blue prefab model(s) with simple sliders. Add a ManualPoser to the scene, assign it to the SG_TrackedHand’s trackingSource, and you’re ready to go!

This is the basic process by which you make objects grabable and by which to add force-feedback to objects. Play around with the SG_Material parameters and see how changing the maxForce and force-Response change the density of the cube. You can also adjust the moveSpeed and rotationSpeed parameters of the SG_Grabable script to change how your cube moves when you hold onto it.

Note

The SG_Grabable and SG_Breakable scripts can be attached to any Unity Collider. This includes mesh colliders as well! But due to limitations in the Unity Engine, this can only be done with convex mesh colliders. See also: Mesh Collider Reference.

Task 1.2: Make something breakable

A Breakable object is a special type of Grabable where, if you squeeze hard enough, the “Unbroken” gameobject is disabled, and optionally replaced with a “Broken” version. This change in object will disable the force-feedback, creating a surprising experience for your users.

Making an object breakable requires you to create an “Unbroken” and (optionally) a “Broken” version of your object. In this case, let’s re-use the Cube from the previous task: Create a copy of this cube, and scale it down to about half the size of the original; [0.03, 0.03, 0.03]. This will be our “Broken” version of the cube.

Create a new, empty GameObject to house our Breakable logic, and name it something sensible, like “BreakLogic”. Make both cubes children of this BreakLogic object. This is not madatory, but is good practice: It will allow you to more easily see which break logic belongs to which object(s).

Attach an SG_Breakable script to this empty GameObject. Assign the bigger, “Unbroken” cube to the wholeObject parameter, and assign the smaller, “Broken” cube to the brokenObject parameter. It should look something like this:

Tip

Disabling a GameObject also disables the scripts that are attached to it. Attaching the SG_Breakable script to the “Unbroken” or “Broken” GameObject will cause it to disable itself when changing states, meaning it will no longer respond to events. If you’re only using the Breakable logic once, you can attach the SG_Breakable script to the same object. Otherwise, use a separate GameObject like we did just now.

The SG_Breakable logic will fire when the SG_Material script attached to the wholeObject fires a MaterialBreak event. We control if and when this event is fired through the SG_Material’s Breakable Material Settings.

Set the breakable parameter of the SG_Material on the “Unbroken” cube to true. For this exercise, we’ll also set the mustBeGrabbed parameter to true as well. Doing so means the object will not break unless you’re holding it. Leave the other settings as they are for now.

Run the program. The “Broken” cube will dissapear. You should be able to pick up and feel the “Unbroken” cube like before. When you squeeze through the cube, it will “break”, and in its place will be its smaller, broken version.

The yieldDistance parameter of the SG_Material script determines how far inside the collider your fingers can push before a MaterialBreak event is fired. It’s given in meters [m], so the default value of 0.03 indicates you can push your fingers in for about 3cm before the material “breaks”.

Tip

If no “Broken” object is assigned to the SG_Breakable script, the “Unbroken” object will simply dissapear.

Play around with the breakable settings of your “Unbroken” object, and see how this affects the breaking logic. You can also opt to not add a “Broken” object to the SG_Breakable script, which causes the original to simple dissappear.

The SenseGlove Unity Plugin comes with a pre-rigged Breakable object. It can be found by following the path “Assets/SenseGlove/Prefabs/Interactions/BreakableObject”.

Task 1.3: Deforming a “Squishy” object

You can use the SG_Material’s maxForceDist and forceResponse parameters to make the force-feedback ramp up as you squeeze into the object’s collider. This will make it feel squishy, but it will visually stay the same. Adding an SG_MeshDeform script to an object with a SG_Material script will visually deform the mesh of the object as your fingers pass through it, creating the illusion of a “squishy” material, such as that of a stress ball.

To create a squishy object, we need to have a grabable object. We can re-use the cube from the previous exercise(s), or create a new Sphere object with an SG_Material and SG_Grabable script attached.

Attach an SG_MeshDeform script to this new grabable, and leave the parameters as they are for now. When running the simulation, you’ll notice the object deforms as you squeeze in it! Whether you’re holding on to it or not.

Note

The only deformation method available at this time is called “Plane”. This method flattens all vertices that lie ‘above’ the finger(s), as though a glass pane presses down on them. This MeshDeform may therefore not be suited for large objects.

Task 2: Haptics and Scripting

So far, we’ve covered how to work with the Unity plugin using the Inspector only. Let’s look at some scripting next.

In this task, we will create a grabable object that sends a vibration (and for Nova2 squeezes the active strap) to all hands that are holding it. To do so, we will create a separate script to observe the Grabable’s state and send a vibration command. First, create a new C# Script and name it “VibratingObject”. Place the following lines of code inside:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

using SG; //All SenseGlove Unity Plugin scripts are embedded in the SG namespace.

public class VibratingObject : MonoBehaviour
{
   /// <summary> The Interactable object that we will be sending vibration commands to. </summary>
   /// <remarks> Since SG_Grabable derives from SG_Interactable, this will work for grabables, as well as any other script that derives from SG_Interactable. </remarks>
   [SerializeField]
   private SG_Interactable objectToVibrate;
   [SerializeField]
   private SG_CustomWaveform hapticsWaveform;
   [SerializeField]
   private SG_CustomWaveform stopVibrationWaveform;
   [SerializeField]
   private SG.VibrationLocation vibrationLocation = SG.VibrationLocation.WholeHand;

   //assign this to the on grabbed event of the grabable
   //This will loop until explicitly told to stop
   public void StartVibration()
   {
      objectToVibrate.SendCustomWaveform(hapticsWaveform, vibrationLocation);
   }

   //assign this to the on dropped event of the grabable
   public void StopVibration()
   {
      objectToVibrate.SendCustomWaveform(stopVibrationWaveform, vibrationLocation);
   }
}

Create two new custom waveforms somewhere in your project by right clicking and navigating to Create>SenseGlove and selecting “Custom Waveform” and name them something descriptive like “ConstantVibration” and “StopVibration”. For “ConstantVibration” you want to turn on “Repeat Infinite” and for “StopVibration” you want to set Amplitude to 0.

Create a new Grabable gameObject. You can re-use the object you’ve made in Task 1, or create an entirely new one.

Next, attach the new VibratingObject script to your Grabable object.

Assign the grabable to the objectToVibrate field of the VibratingObject script.

Assign the custom waveforms to the waveforms field of the VibratingObject script.

Now, when you grab the object, you’ll feel vibration on your hand! Change the magnitude in the custom waveform you made and vibrationLocation in the VibratingObject script to adjust the intensity and location(s) of the signal. Note that a Nova Glove only has vibration motors on the Thumb and Index finger.

Note

We could embed the vibration logic in the SG_Grabable class or create a script that derives from SG_Grabable. However, doing so would limit this feature to one specific class only. Placing this logic in a separate script allows us to add this behaviour to any kind of interactable object (drawers, hinges etc) without changing any of their internal logic.

Task 3 : Create a simple power tool

A simple power tool, such as a drill, is a grabable which “snaps” the hand in the right place. We can also do something a little bit more complicated by mapping the index finger to the drill’s (haptics) behaviour.

First, you’ll need a model of a tool. You’re free to model your own, though for this tutorial, we’ll assume you use the model found in “Assets/SenseGlove/Prefabs/Interactions/DummyDrill” It is already set up with physics colliders and “Snap Points”. Use it as a reference for your own tool(s).

Place your tool inside the Unity Scene, and add an SG_Grabable script to the root of the DummyDrill object. Let’s add a RigidBody to the root of the object as well, which will ensure that any grab events on its colliders are passed to the SG_Grabable script. Right now, you should be able to grab your tool like the cube in Task 1.1. But what if we want it to snap to the hand in a fixed location?

Add a SG_SnapOptions to the gameobject. This component tells our SG_Grabable script to snap the hand to a specific point on the object. Assign the “SnapPoint_Right” and “SnapPoint_Left” of the dummy drill to the script’s rightHandSnapPoint and leftHandSnapPoint respectively. This lets the script where to place the left- and right hands’ Grab References (VirtualGrabRef) on the tool.

With the snap behaviour in place, let’s press Play. When you grab your tool, the left- and right hands should now snap to the assigned location!

Tip

When creating custom Snap Points for your tool handles, the general rule of thumb is to have their X-axis align ‘foward’ (parallel to the hand palm). For the right hand, the Y-Axis of the snap point should be aligned “upwards” (from the bottom of your handle to the top). For the left hand, this Y-Axis should run “downwards” (from the top of your handle to the bottom).

You now have a tool that snaps to your hand. But you can’t really do anything with the trigger yet. The simplest way to add this functionality is through either a ‘collision-based’ approach or a ‘gesture-based’ approach. In a collision-based approach, you add a collider to the trigger and implement an OnTriggerEnter() function that responds to the rigidbodies of the hand. While simple to set up, this approach only fires once, and is dependent on the correct placement of your SnapPoints. The ‘gesture-based’ approach uses the flexion of a specific finger to map force- and vibrotactile feedback on the hand. In this exercise, we’ll be using this approach:

Create a new C# script and give it a descriptive name, such as TriggerLogic. Add the following code to it:

using SG;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class TriggerLogic : MonoBehaviour
{
   /// <summary> Grabable linked to this Script </summary>
   public SG.SG_Grabable grabable;

   /// <summary> The trigger logic will respond to the flexion of this finger, which is given as value between 0..1. </summary>
   public SGCore.Finger respondsTo = SGCore.Finger.Index;

   [Range(0, 1)] public float startFlexion = 0.2f; //when finger flexion is above this value, trigger pressure will ramp up from 0%
   [Range(0, 1)] public float endFlexion = 0.8f; //when finger flexion is at this value, the trigger pressure it at 100%

   /// <summary> The last calculated pressure </summary>
   private float latestPressure = 0.0f;

   private Coroutine hapticsLoop;

   [SerializeField]
   private SG_CustomWaveform hapticsWaveform;
   [SerializeField]
   private SG_CustomWaveform stopVibrationWaveform;
   [SerializeField]
   private SG.VibrationLocation vibrationLocation = SG.VibrationLocation.WholeHand;


   /// <summary> The Trigger Pressure as calculated by this script.. </summary>
   public float TriggerPressure
   {
      get { return grabable.IsGrabbed() ? latestPressure : 0.0f; } //For outside scripts; When you're not being grabbed, return 0.
   }

   // Update is called once per frame
   void Update()
   {
      if (grabable.IsGrabbed()) //indicated there is at least one script grabbing onto grabable
      {
         SG.SG_TrackedHand firstHand = grabable.ScriptsGrabbingMe()[0].TrackedHand; //grab the first hand

         //update the latest pressure
         float[] flexions;
         if (firstHand.GetNormalizedFlexion(out flexions)) //attempt to get the latest normalized finger flexions: values between 0...1
         {
            float currFlex = flexions[(int)this.respondsTo]; //we're using an enumerator to index the array (0..4, where 0 = thumb and 4 = pinky).
            if (startFlexion == endFlexion)  //if these are equal, mapping would result into a div/0.
            {
               latestPressure = currFlex >= startFlexion ? 1.0f : 0.0f;
            }
            else
            {
               latestPressure = SG.Util.SG_Util.Map(currFlex, startFlexion, endFlexion, 0.0f, 1.0f, true); //free function that comes with unity plugin; map a value from one range to another.
            }
         }
      }
   }

   //assign this to the on grabbed event of the grabable
   //This will loop until explicitly told to stop
   public void StartVibration()
   {
      if (hapticsLoop == null)
      {
         hapticsLoop = StartCoroutine(UpdateHaptics());
      }

   }

   //assign this to the on dropped event of the grabable
   public void StopVibration()
   {
      if (hapticsLoop != null)
      {
         StopCoroutine(hapticsLoop);
         hapticsLoop = null;
      }

   }

   IEnumerator UpdateHaptics()
   {
      for (; ; )
      {

         hapticsWaveform.amplitude = latestPressure;
         //start vibration
         grabable.SendCustomWaveform(hapticsWaveform, vibrationLocation);
         //start squeeze
         grabable.QueueWristSqueeze(latestPressure);//For now its just a static 1, TODO: Remove magic number

         yield return new WaitForSeconds(0.1f);
      }
   }
}

Create two new custom waveforms somewhere in your project by right clicking and navigating to Create>SenseGlove and selecting “Custom Waveform” and name them something descriptive like “ConstantVibration” and “StopVibration”. For “ConstantVibration” you want to turn on “Repeat Infinite” and for “StopVibration” you want to set Amplitude to 0.

Create a new Grabable gameObject. You can re-use the object you’ve made in Task 1, or create an entirely new one.

Next, attach the new VibratingObject script to your Grabable object.

Assign the grabable to the objectToVibrate field of the VibratingObject script.

Assign the custom waveforms to the waveforms field of the VibratingObject script.

This code is similar to the one in Task #2. However, instead of using a constant command, we vary its intensity based on the latestPressure. You’ll notice that as you flex your index finger further, the vibration increases in intensity. Because the startFlexion is greater than 0, the vibration does not immediately start when you grab onto the tool.

You can re-use the TriggerPressure in other scripts. For example, you could have the drill activate something within your scene when the pressure is greater than a certain value. You can also use this value it to control the animation of your drill trigger (and head).

You now know how to set up a simple tool! Play around with the SG_SnapOptions and SG_TriggerLogic parameters to get a feel for what each of them does.

Tip

If your tool changes its manipulation behaviour throughout the scene (by snapping to a set axis when close to a screw, for example), you could implement a new script that extends off SG_Grabable, and override its UpdateLocation method with (for example) a State pattern. Alternatively, you could have an external script that ‘snaps’ your tool to the correct location, and sets the moveSpeed and/or rotateSpeed to 0, which will prevent SenseGlove scripts from moving it.