Scroll to top

Dark

Light

Dark

Light

Get In Touch
London
United Kingdom
Work Inquiries
hello@hillierforshaw.co.uk
@hillierforshaw

VR HandTracking Gestures

Having a look at the implementation of handtracking for the Oculus Quest. There can be so many benefits to removing controllers from the equation especially when creating experiences with accessibility in mind. Although it’s in its infancy I wanted to check out the current  functionalities and uses pre Oculus Quest 2 using the unity engine and C#.

Gesture tracking

To start I initially looked at the integration of the hands and built a little rig that show’s the interaction for each gesture.My findings from this stated that tracking may be lost or hand confidence may become low when one hand occludes with another and the general lighting of an area can weaken the confidence of the gestures.

Gesture triggered UI

Next I decided to look at the integration of gesture triggered user interfaces as this is where my mind is heading in regards to hand tracking gesture use and accessibility. I created a super simple UI menu that appears on right hand Index pinch and hides on middle finger pinch.

 

Custom hand gestures & triggers

Next up is a look at the implementation of custom poses that that trigger events when made.

 

 

Notes:

Pinch

Pinch is the basic interaction primitive for UI interactions using hands. A successful pinch of the index finger can be considered the same as a normal select or trigger action for a controller, i.e., the action that activates a button or other control on a UI.

To detect whether the finger is currently pinching and to check the pinch’s strength, call the GetFingerIsPinching() and GetFingerPinchStrength() methods from OVRHand.cs. Pass the relevant finger constant defined in the HandFinger enum for the finger that you want to query. The finger constants are: Thumb, Index, Middle, Ring, and Pinky.

var hand = GetComponent<OVRHand>();
bool isIndexFingerPinching = hand.GetFingerIsPinching(HandFinger.Index);
float ringFingerPinchStrength = hand.GetFingerPinchStrength(HandFinger.Ring);

 

The progression of a pinch gesture is indicated by the returned float value. For each finger pinch, the corresponding value ranges from 0 to 1, where 0 indicates no pinch and 1 indicates full pinch with the finger touching the thumb.

In addition to the pinch strength, OVRHand.cs also provides the GetFingerConfidence() method to measure the confidence level of the finger pose. It’s measured in terms of low or high, which indicates the amount of confidence that the tracking system has for the finger pose.

To retrieve the confidence level of a finger pose, call the GetFingerConfidence() method and pass the finger constant for which you want to track the confidence level.

var hand = GetComponent<OVRHand>();
TrackingConfidence confidence = hand.GetFingerConfidence(HandFinger.Index);

 

Handtracking    
Output/tested Oculus Quest  
Engine Unity 2020.1.0f1  
Creative Unity  
Git    

I Use Anyltics to provide a better User Experience.