Plugin/Handbook/src/getting-started/enabling-xr-ext-hand-tracking-vr-headsets/pcvr-mode/README.md

Enabling XR_EXT_hand_tracking OpenXR Extension on VR Headsets in PCVR Mode

Starting from version v2.1.0, the SenseGlove Unreal Engine Plugin requires the XR_EXT_hand_tracking OpenXR extension to function. If you are streaming your immersive 3D VR application from your PC to your VR headset, enabling XR_EXT_hand_tracking support, requires additional plugins and settings depending on the HMD's vendor or model.

Enabling OpenXR and OpenXRHandTracking Plugins

Regardless of the type or vendor of the HMD you have in mind for development or deployment purposes, OpenXR and OpenXRHandTracking are required as a prerequisite.

Though enabling the SenseGlove Unreal Engine Plugin should enable the OpenXR and OpenXRHandTracking plugins automatically, it is recommended to ensure these plugins are enabled by navigating to Edit > Plugins in the Unreal Editor menus.

The OpenXRHandTracking plugin implements the XR_EXT_hand_tracking OpenXR extension.

Meta Quest

To set up XR_EXT_hand_tracking support on Meta Quest HMDs in PCVR mode, depending on your project requirements (e.g. whether you rely on the Meta XR plugin or not), additional setup steps are required.

Meta Quest Link App

For Meta Quest headsets, enable the Developer runtime features under the Settings > Beta section inside the Meta Quest Link app:

Important

Enabling Developer runtime features in the Meta Quest Link requires a Meta Developer Account. If you are not signed in using a Meta Developer Account, this option won't be shown to you inside Meta Quest Link.

Caution

Streaming to Meta Quest headsets from SteamVR is no longer supported because the migration to OpenXR has caused controller offsets for Meta Quest HMDs to break on SteamVR. One possible reason is that SteamVR lists XR_FB_hand_tracking as an unsupported feature. Further investigation is needed to identify the exact underlying cause.

Meta XR Plugin

Caution

By default, enabling the Meta XR plugin breaks and disrupts the SenseGlove Unreal Engine Plugin implementation of XR_EXT_hand_tracking. Thus, rendering both glove and hand-tracking data broken. However, it's possible to fix this issue with proper Meta XR plugin configuration.

By default, the Meta XR plugin settings located in Edit > Project Settings > Plugins > Meta XR look something like these:

The following settings control the hand-tracking behavior when the Meta XR plugin is enabled:

• XR API: This selects the XR API that the engine will use. If set to Oculus OVRPlugin + OpenXR backend (current recommended), which is the default, will break the SenseGlove Unreal Engine Plugin's glove and hand-tracking data. In order to make the Meta XR and SenseGlove Unreal Engine plugins compatible you should choose Epic Native OpenXR with Oculus vendor extensions here. This disables Legacy Oculus in favor of the native OpenXR implementation, with Oculus vendor extensions. This will be where Epic focuses XR development going forward. Changing the XR API settings requires an editor restart to take effect.

• Hand Tracking Support: Determines whether controllers and/or hands can be used within the current appication. Possible values are ControllersOnly, ControllersAndHands, and HandsOnly. We recommend to set this to ControllersAndHands to be able to use both controllers (as a wrist-tracker device) and hand-tracking. See also bFallbackToHandTrackingIfNoGloveDetected.

• HandTrackingFrequency: Determines the hand-tracking frequency, whether LOW or HIGH. Please note that a higher tracking frequency will reserve some performance headroom from the application's budget.

• HandTrackingVersion: Determines the version of hand tracking algorithm. Default, V1, and V2 are valid options. If unsure, leave it as Default and the SenseGlove glove and hand-tracking should work as expected.

HTC VIVE

To set up XR_EXT_hand_tracking support on HTC VIVE HMDs in PCVR mode, additional plugins or configuration steps are required.

OpenXRViveTracker Plugin

To enable VIVE Trackers support ensure OpenXRViveTracker is enabled by navigating to Edit > Plugins in the Unreal Editor menus. This plugin should be enabled, or wrist tracking won't function on VIVE devices at all.

The OpenXRViveTracker plugin implements the XR_HTCX_vive_tracker_interaction OpenXR extension.

VIVE Business Streaming App

For VIVE headsets relying on the VIVE Business Streaming application, ensure the Hand Tracking settings under Input are enabled:

Note

Tracking and accessing FXRMotionControllerData output from SenseGlove devices do not require Hand and Body Tracking to be enabled on the HMD device. Enabling this feature is only necessary if you wish to use hand-tracking as a fallback option when no glove is connected to your PC.