diff --git a/Handbook/src/advanced-topics/openxr/README.md b/Handbook/src/advanced-topics/openxr/README.md index 8ca24724..4fb279ac 100644 --- a/Handbook/src/advanced-topics/openxr/README.md +++ b/Handbook/src/advanced-topics/openxr/README.md @@ -4,15 +4,126 @@ The SenseGlove Unreal Engine Plugin has provided OpenXR-compatible hand tracking Typically a user does not need to know anything about OpenXR to use the plugin, so this section of the handbook is for advanced users who are looking for a way to directly consume the OpenXR data coming directly from either a SenseGlove device or if enabled in the plugin settings from hand-tracking. -Since the SenseGlove Unreal Engine Plugin registers itself as an `OpenXRHandTracking` motion controller device it becomes a hand-tracking provider for Unreal Engine, thus the OpenXR data from SenseGlove could always be retrieved from the Unreal Engine's `IXTrackingSystem` with one caveat. The caveat is if another OpenXR-compatible hand-tracking plugin, e.g. Epic's own OpenXRHandTracking, is enabled simultaneously it's not guaranteed that the `FXRHandTrackingState` structs retrieved from the `IXTrackingSystem::GetMotionControllerData()` and `IXTrackingSystem::GetHandTrackingState()` methods are coming from SenseGlove, as these methods return the first hand-tracking plugin they could find. Thus, SenseGlove provides its own implementation of `GetMotionControllerData()` and `GetHandTrackingState()` which guarantee the retrieved `FXRHandTrackingState` are coming from the SenseGlove Unreal Engine Plugin; and this is the preferred way to that. +Since the SenseGlove Unreal Engine Plugin registers itself as an `OpenXRHandTracking` device it becomes a hand-tracking provider for Unreal Engine, thus the OpenXR data from SenseGlove could always be retrieved from the Unreal Engine's `IXTrackingSystem` with two caveats: -> [!NOTE] +- The first caveat is, if another OpenXR-compatible hand-tracking plugin, e.g. Epic's own OpenXRHandTracking, is enabled simultaneously it's not guaranteed that the `FXRHandTrackingState` struct retrieved from the `IXTrackingSystem::GetHandTrackingState()` method is coming from SenseGlove, as these methods return the first hand-tracking plugin they could find. Thus, SenseGlove provides its own implementation of `GetHandTrackingState()` which guarantees the retrieved `FXRHandTrackingState` is coming from the SenseGlove Unreal Engine Plugin; and this is the preferred way to that. +- The second caveat is, Unreal `IXTrackingSystem::GetHandTrackingState()` method is blind to SenseGlove's [wrist-tracking settings and offsets](../../plugin-configuration/plugin-settings/tracking/wrist-tracking/), therefore the returned `FXRHandTrackingState` retrieved using this engine function won't take into account the wrist-tracker offsets, and the hands end up at the wrong orientation or position. In contrast, + +> [!IMPORTANT] > In order to retrieve the latest `FXRHandTrackingState` available, The > SenseGlove Unreal Engine Plugin provides an alternative implementation for -> `IXTrackingSystem::GetMotionControllerState()` as well . However, since this -> method does not rely on the `OpenXRHandTracking` provider, it may become -> redundant. As a result, we might consider removing this functionality in -> future updates in favor of the one that Unreal Engine provides. +> `IXTrackingSystem::GetHandTrackingState()`, which guarantees the OpenXR +> hand-tracking data is coming from a SenseGlove device and also takes into +> account the SenseGlove's wrist-tracker offsets automatically. +> This is the recommended approach over Unreal Engine's own +> `IXTrackingSystem::GetHandTrackingState()` or you have to ensure the data +> received is coming from a SenseGlove device yourself, and also take into +> account the SenseGlove's wrist-tracker settings and calculate the wrist +> offsets either using one of the `SGHapticGlove::GetWristLocation()` variants, +> or manually. +> In short `IXTrackingSystem::GetHandTrackingState()` does not respect the +> offsets from +> `Project Settings > SenseGlove > Tracking > Wrist-Tracking Settings` + +> [!CAUTION] +> In order to retrieve the latest `FXRHandTrackingState` available, The +> SenseGlove Unreal Engine Plugin provides an alternative implementation for +> `IXTrackingSystem::GetHandTrackingState()`, which guarantees the OpenXR +> hand-tracking data is coming from a SenseGlove device and also takes into +> account the SenseGlove's wrist-tracker offsets automatically. +> This is the recommended approach over Unreal Engine's own +> `IXTrackingSystem::GetHandTrackingState()` or you have to ensure the data +> received is coming from a SenseGlove device yourself, and also take into +> account the SenseGlove's wrist-tracker settings and calculate the wrist +> offsets either using one of the `GetWristLocation()` variants, e.g., +> `SGHandLayer::GetWristLocation()`, `SGHapticGlove::GetWristLocation()`, etc, +> or manually. +> In short `IXTrackingSystem::GetHandTrackingState()` does not respect the +> offsets from +> `Project Settings > SenseGlove > Tracking Settings > Wrist Tracking Settings`. + +> [!TIP] +> Sometimes, using `IXTrackingSystem::GetHandTrackingState()` is unavoidable, +> e.g., when using a third-party hand manipulation system such as +> [**VR Expansion Plugin (VRE)**](https://vreue4.com/), which internally relies +> on `IXTrackingSystem::GetHandTrackingState()` to retrieve the hand-tracking +> data. In that case SenseGlove provides methods such as +> `SGHandLayer::GetWristLocation()`, `SGHapticGlove::GetWristLocation()`, etc, +> which you can use to reliably calculate the wrist offsets: +> +> ```cpp +> // Get the OpenXR hand-tracking data for the right hand +> FXRHandTrackingState HandTrackingState; +> IXTrackingSystem::GetHandTrackingState( +> GetWorld(), +> EXRSpaceType::UnrealWorldSpace, +> EControllerHand::Right, +> HandTrackingState); +> +> // Return if the struct data is invalid! +> if (!bGotHandTrackingState || !HandTrackingState.bValid) +> { +> return; +> } +> +> // Return if the device is not being tracked! +> if (HandTrackingState.TrackingStatus == ETrackingStatus::NotTracked) +> { +> return; +> } +> +> // Ensure that HandTrackingState.HandKeyLocations has the location data +> // for 26 joints! +> if (!ensureAlwaysMsgf(HandTrackingState.HandKeyLocations.Num() +> == EHandKeypointCount, +> TEXT("Invalid HandKeyLocations count!"))) +> { +> return; +> } +> +> // Ensure that HandTrackingState.HandKeyRotations has the rotation data +> // for 26 joints! +> if (!ensureAlwaysMsgf(HandTrackingState.HandKeyRotations.Num() +> == EHandKeypointCount, +> TEXT("Invalid HandKeyRotations count!"))) +> { +> return; +> } +> +> { +> // FQuat variant of FSGHandLayer::GetWristLocation() +> // Get the OpenXR hand-tracking data with the correct wrist offsets for +> // Meta Quest 3 Controllers +> FVector WristLocation; +> FQuat WristRotation; +> FSGHandLayer::GetWristLocation( +> true, // true for a right-handed glove and false for a left-handed one +> HandTrackingState.HandKeyLocations[0], // 0 is the Wrist joint location +> HandTrackingState.HandKeyRotations[0], // 0 is the wrist joint rotation +> ESGPositionalTrackingHardware::Quest3Controller, +> WristLocation, WristRotation); +> +> // WristLocation and WristRotation variables now contain the correct +> // OpenXR hand-tracking data with the wrist offsets applied correctly +> } +> +> { +> // FRotator variant of FSGHandLayer::GetWristLocation() +> // Get the OpenXR hand-tracking data with the correct wrist offsets for +> // Meta Quest 3 Controllers +> FVector WristLocation; +> FRotator WristRotation; +> FSGHandLayer::GetWristLocation( +> true, // true for a right-handed glove and false for a left-handed one +> HandTrackingState.HandKeyLocations[0], // 0 is the Wrist joint location +> HandTrackingState.HandKeyRotations[0].Rotator, // 0 is the wrist joint rotation +> ESGPositionalTrackingHardware::Quest3Controller, +> WristLocation, WristRotation); +> +> // WristLocation and WristRotation variables now contain the correct +> // OpenXR hand-tracking data with the wrist offsets applied correctly +> } +> ``` In the next sections we'll see: diff --git a/Handbook/src/appendix/changelog.md b/Handbook/src/appendix/changelog.md index 7be5c166..66e87836 100644 --- a/Handbook/src/appendix/changelog.md +++ b/Handbook/src/appendix/changelog.md @@ -32,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Dropped support for Unreal Engine `5.4`, which was already deprecated via the `v2.7.x` release series. - Dropped support for Epic Native/Cross Toolchains `v22` (previously used for building UE `5.3` and `5.4` Linux dependencies), as they were already deprecated via previous releases. - Removed support for the deprecated `FXRMotionControllerData`. The plugin now exclusively uses `FXRHandTrackingState` (introduced in Unreal Engine 5.5+ and supported by The SenseGlove Unreal Engine Plugin since [`v2.2.0`](#220---2024-10-22) for OpenXR hand tracking. This affects only projects that directly consume `FXRMotionControllerData` from the SenseGlove plugin in their own custom hand-tracking or interaction systems. Please see [the v2.7.x to v2.8.x migration guide](../misc/upgrade-guide/#upgrading-from-v27x-to-v28x) for more details. +- Removed SenseGlove's `GetMotionControllerData()`; the alternative implementation to `IXTrackingSystem::GetMotionControllerData()`. You can now use SenseGlove's `GetHandTrackingState()` instead of Unreal's `IXTrackingSystem::GetHandTrackingState()` which guarantees the OpenXR hand-tracking data is coming from a SenseGlove device and also takes into account the SenseGlove's wrist-tracker offsets automatically. ### Documentation @@ -39,6 +40,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Updated [Enabling XR_EXT_hand_tracking on VR Headsets](../getting-started/enabling-xr-ext-hand-tracking-vr-headsets/) documentation, replacing `FXRMotionControllerData` usage with `FXRHandTrackingState`. - Updated [Setup the Virtual Hand Meshes](../getting-started/setup-virtual-hand-meshes/) documentation, replacing `FXRMotionControllerData` usage with `FXRHandTrackingState`. - Updated [Plugin Configuration > Plugin Settings > Virtual Hand > Mesh](../plugin-configuration/plugin-settings/virtual-hand/mesh.md) documentation, replacing `FXRMotionControllerData` usage with `FXRHandTrackingState`. +- Updated [Advanced Topics > OpenXR](../advanced-topics/openxr/) documentation section with more relevant and plenty of useful information reflecting the recent changes. - Removed the **Consuming FXRMotionControllerData** documentation section as it's no longer relevant. - Removed the **Consuming FXRMotionControllerData > Blueprint** documentation section as it's no longer relevant. - Removed the **Consuming FXRMotionControllerData > C++** documentation section as it's no longer relevant.