migrate away from the deprecated FXRMotionControllerData in favor of FXRMotionControllerState and FXRHandTrackingState on UE 5.5+

This commit is contained in:
Mamadou Babaei
2024-10-22 06:40:08 +02:00
parent c290b87f44
commit 29026c5d65
22 changed files with 744 additions and 51 deletions
+34
View File
@@ -5,6 +5,40 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
This is a minor release with some breaking API and ABI changes, focusing mainly on migrating away from the deprecated `FXRMotionControllerData` in favor of `FXRMotionControllerState` and `FXRHandTrackingState` on Unreal Engine `5.5+`.
### Added
- Completed support for the upcoming Unreal Engine `5.5` release.
- Added `USGVirtualHandComponent::GetMotionControllerState()` and the equivalent Blueprint function`UVirtualHandComponentKismetLibrary::GetMotionControllerState` on UE `5.5+`.
- Added `USGVirtualHandComponent::GetHandTrackingState()` and the equivalent Blueprint function `UVirtualHandComponentKismetLibrary::GetHandTrackingState` on UE `5.5+`.
- Added `USGWristTrackerComponent::GetMotionControllerState()` and the equivalent Blueprint function `UWristTrackerComponentKismetLibrary::GetMotionControllerState` on UE `5.5+`.
- Added `USGWristTrackerComponent::GetHandTrackingState()` and the equivalent Blueprint function `UWristTrackerComponentKismetLibrary::GetHandTrackingState` on UE `5.5+`.
- Added a variant of `FSGDebugVirtualHand::Draw()` and the equivalent Blueprint function `USGDebugVirtualHandKismetLibrary::Draw_FXRHandTrackingState()` which accept `FXRHandTrackingState` on UE `5.5+`.
- Added the new member `bTracked` to the `FSGXRHandState` struct.
- Added `FSGXRTracker::GetMotionControllerState()` and the equivalent Blueprint function `USGXRTrackerKismetLibrary::GetMotionControllerState()`.
- Added `FSGXRTracker::GetHandTrackingState()` and the equivalent Blueprint function `USGXRTrackerKismetLibrary::GetHandTrackingState()`.
### Fixed
- Additional minor fixes and improvements that may not be listed here.
### Changed
- Replaced all internal usages of the `FXRMotionControllerData` struct with either `FXRMotionControllerState` or `FXRHandTrackingState` on UE `5.5+`.
- Deprecated `USGVirtualHandComponent::GetMotionControllerData()` on UE `5.5+`.
- Deprecated `USGWristTrackerComponent::GetMotionControllerData()` on UE `5.5+`.
- Deprecated the variant of `FSGDebugVirtualHand::Draw()` which accepts `FXRMotionControllerData` as a parameter on UE `5.5+`.
- Renamed `USGDebugVirtualHandKismetLibrary::Draw` to `USGDebugVirtualHandKismetLibrary::Draw_FXRMotionControllerData` for more clarification.
- Renamed an `FSGXRHandState` member from `bReceivedJointPoses` to `bHasReceivedJointPoses`.
- Changed the `FSGXRTracker::GetAllKeypointStates()` signature on UE `5.5+` to match the `IHandTracker` interface API changes.
- The animation system on UE `5.5+` has been revamped to utilize `FXRHandTrackingState` instead of `FXRMotionControllerData`.
- The wrist tracking system on UE `5.5+` has been revamped to utilize `FXRHandTrackingState` instead of `FXRMotionControllerData`.
- The hand interaction manipulation on UE `5.5+` has been revamped to utilize `FXRHandTrackingState`.
- The virtual hand debugging system on UE `5.5+` has been revamped to utilize `FXRHandTrackingState`.
## [2.1.3] - 2024-10-11 ## [2.1.3] - 2024-10-11
### Added ### Added
@@ -1,6 +1,6 @@
## Planned Features Completion Status ## Planned Features Completion Status
### Implemented as of v2.1.x ### Implemented as of v2.2.x
- [X] Full SenseGlove low-level core API access through Unreal C++. - [X] Full SenseGlove low-level core API access through Unreal C++.
- [X] Full SenseGlove low-level core API access through Blueprint. - [X] Full SenseGlove low-level core API access through Blueprint.
@@ -28,17 +28,18 @@
- [X] C++/Blueprint interaction events such as OnGrabStateUpdated, OnTouchStateUpdated, OnActorGrabbed, OnActorReleased, OnActorBeginTouch, and OnActorEndTouch. - [X] C++/Blueprint interaction events such as OnGrabStateUpdated, OnTouchStateUpdated, OnActorGrabbed, OnActorReleased, OnActorBeginTouch, and OnActorEndTouch.
- [X] A fall back to HMD and wrist tracker hardware auto-detection mechanism when automatic detection of the wrist tracker hardware is desired. - [X] A fall back to HMD and wrist tracker hardware auto-detection mechanism when automatic detection of the wrist tracker hardware is desired.
- [X] OpenXR-compatible hand tracking (XR_EXT_hand_tracking) support. - [X] OpenXR-compatible hand tracking (XR_EXT_hand_tracking) support.
- [X] FXRMotionControllerData compatible hand animation system. - [X] `FXRMotionControllerData` compatible hand animation system on UE versions `5.2`, `5.3`, and `5.4`.
- [X] FXRMotionControllerData compatible wrist tracking system. - [X] `FXRHandTrackingState` compatible hand animation system on UE versions `5.5+`.
- [X] FXRMotionControllerData compatible hand interaction manipulation system. - [X] `FXRMotionControllerData` compatible wrist tracking system on UE versions `5.2`, `5.3`, and `5.4`.
- [X] `FXRHandTrackingState` compatible wrist tracking system on UE versions `5.5+`.
- [X] `FXRMotionControllerData` compatible hand interaction manipulation system on UE versions `5.2`, `5.3`, and `5.4`.
- [X] `FXRHandTrackingState` compatible hand interaction manipulation system on UE versions `5.5+`.
- [X] Ability to fallback to hand tracking when a glove is not present and use the bare hands for interactions, or a combination of glove and hand tracking if no motion controller input is detected. - [X] Ability to fallback to hand tracking when a glove is not present and use the bare hands for interactions, or a combination of glove and hand tracking if no motion controller input is detected.
- [X] The SenseGlove grab/touch sockets one-click-setup ability on any Epic-compliant virtual hand mesh from within the Unreal Editor's Content Browser, Skeleton Editor, or Skeletal Mesh Editor. - [X] The SenseGlove grab/touch sockets one-click-setup ability on any Epic-compliant virtual hand mesh from within the Unreal Editor's Content Browser, Skeleton Editor, or Skeletal Mesh Editor.
- [X] A flexible virtual hand animation system that can take the mesh bone's transforms into account for a more reliable hand animation. - [X] A flexible virtual hand animation system that can take the mesh bone's transforms into account for a more reliable hand animation.
- [X] Ability to manage the Engine Scalability Settings through the SenseGlove plugin in order to change the graphics settings on the fly. - [X] Ability to manage the Engine Scalability Settings through the SenseGlove plugin in order to change the graphics settings on the fly.
### Upcoming features planned for the v2.2.x release ### Upcoming features planned for the v2.3.x release
- [ ] Migrating away from the deprecated `FXRMotionControllerData` in favor of `FXRMotionControllerState` and `FXRHandTrackingState`.
### Planned features long-term ### Planned features long-term
@@ -47,7 +47,6 @@
#include "Runtime/Launch/Resources/Version.h" #include "Runtime/Launch/Resources/Version.h"
#include "SenseGlove/Components/SGVirtualHandComponent.h" #include "SenseGlove/Components/SGVirtualHandComponent.h"
#include "SGLog/SGLog.h"
FSGVirtualHandAnimInstanceProxy::FSGVirtualHandAnimInstanceProxy() FSGVirtualHandAnimInstanceProxy::FSGVirtualHandAnimInstanceProxy()
: FAnimInstanceProxy{} : FAnimInstanceProxy{}
@@ -108,6 +107,16 @@ void FSGVirtualHandAnimInstanceProxy::PreEvaluateAnimation(UAnimInstance* InAnim
return; return;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
const bool bGotHandTrackingState = VirtualHand->GetHandTrackingState(
EXRSpaceType::UnrealWorldSpace, HandTrackingState);
if (!bGotHandTrackingState || !HandTrackingState.bValid)
{
return;
}
ensureAlwaysMsgf(HandTrackingState.HandKeyLocations.Num() == HandTrackingState.HandKeyRotations.Num(),
TEXT("Mismatched hand tracking state locations and rotations!"));
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
const bool bGotMotionControllerData = VirtualHand->GetMotionControllerData(MotionControllerData); const bool bGotMotionControllerData = VirtualHand->GetMotionControllerData(MotionControllerData);
if (!bGotMotionControllerData || !MotionControllerData.bValid) if (!bGotMotionControllerData || !MotionControllerData.bValid)
{ {
@@ -115,10 +124,18 @@ void FSGVirtualHandAnimInstanceProxy::PreEvaluateAnimation(UAnimInstance* InAnim
} }
ensureAlwaysMsgf(MotionControllerData.HandKeyPositions.Num() == MotionControllerData.HandKeyRotations.Num(), ensureAlwaysMsgf(MotionControllerData.HandKeyPositions.Num() == MotionControllerData.HandKeyRotations.Num(),
TEXT("Mismatched motion controller data positions and rotations!")); TEXT("Mismatched motion controller data positions and rotations!"));
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
BoneNamesMap.Empty(); BoneNamesMap.Empty();
for (TArray<FQuat>::SizeType HandKeyIndex = 0; for (TArray<FQuat>::SizeType HandKeyIndex = 0;
HandKeyIndex < MotionControllerData.HandKeyRotations.Num(); ++HandKeyIndex) HandKeyIndex <
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
HandTrackingState.HandKeyRotations.Num()
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
MotionControllerData.HandKeyRotations.Num()
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
; ++HandKeyIndex)
{ {
const FName& BoneName{VirtualHand->GetHandBoneName(HandKeyIndex)}; const FName& BoneName{VirtualHand->GetHandBoneName(HandKeyIndex)};
BoneNamesMap.Add(HandKeyIndex, BoneName); BoneNamesMap.Add(HandKeyIndex, BoneName);
@@ -136,10 +153,17 @@ bool FSGVirtualHandAnimInstanceProxy::Evaluate(FPoseContext& Output)
return FAnimInstanceProxy::Evaluate(Output); return FAnimInstanceProxy::Evaluate(Output);
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
if (!HandTrackingState.bValid)
{
return FAnimInstanceProxy::Evaluate(Output);
}
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
if (!MotionControllerData.bValid) if (!MotionControllerData.bValid)
{ {
return FAnimInstanceProxy::Evaluate(Output); return FAnimInstanceProxy::Evaluate(Output);
} }
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FComponentSpacePoseContext PoseContext{Output.AnimInstanceProxy}; FComponentSpacePoseContext PoseContext{Output.AnimInstanceProxy};
PoseContext.Pose.InitPose(Output.Pose); PoseContext.Pose.InitPose(Output.Pose);
@@ -164,9 +188,16 @@ bool FSGVirtualHandAnimInstanceProxy::Evaluate(FPoseContext& Output)
BoneReference.BoneIndex = BoneIndex; BoneReference.BoneIndex = BoneIndex;
BoneReference.bUseSkeletonIndex = true; BoneReference.bUseSkeletonIndex = true;
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
const FVector& HandKeyLocation{HandTrackingState.HandKeyLocations[HandKeyIndex]};
const FQuat& HandKeyRotation{HandTrackingState.HandKeyRotations[HandKeyIndex]};
const FTransform HandKeyTransform{HandKeyRotation, HandKeyLocation};
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
const FVector& HandKeyPosition{MotionControllerData.HandKeyPositions[HandKeyIndex]}; const FVector& HandKeyPosition{MotionControllerData.HandKeyPositions[HandKeyIndex]};
const FQuat& HandKeyRotation{MotionControllerData.HandKeyRotations[HandKeyIndex]}; const FQuat& HandKeyRotation{MotionControllerData.HandKeyRotations[HandKeyIndex]};
const FTransform HandKeyTransform{HandKeyRotation, HandKeyPosition}; const FTransform HandKeyTransform{HandKeyRotation, HandKeyPosition};
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
const FTransform BoneTransform{AnimationBoneCorrectionOffsetTransform * HandKeyTransform}; const FTransform BoneTransform{AnimationBoneCorrectionOffsetTransform * HandKeyTransform};
FRotator BoneRotation{BoneTransform.GetRotation().Rotator()}; FRotator BoneRotation{BoneTransform.GetRotation().Rotator()};
@@ -452,8 +452,10 @@ bool USGVirtualHandComponent::GetMotionControllerData(FXRMotionControllerData& O
OutMotionControllerData.bValid = false; OutMotionControllerData.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left; const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
PRAGMA_DISABLE_DEPRECATION_WARNINGS
const bool bGotMotionControllerData = const bool bGotMotionControllerData =
FSGXRTracker::GetMotionControllerData(this, Hand, OutMotionControllerData); FSGXRTracker::GetMotionControllerData(this, Hand, OutMotionControllerData);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
if (!bGotMotionControllerData || !OutMotionControllerData.bValid) if (!bGotMotionControllerData || !OutMotionControllerData.bValid)
{ {
return false; return false;
@@ -462,6 +464,46 @@ bool USGVirtualHandComponent::GetMotionControllerData(FXRMotionControllerData& O
return OutMotionControllerData.bValid; return OutMotionControllerData.bValid;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool USGVirtualHandComponent::GetMotionControllerState(
const EXRSpaceType XRSpaceType, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
OutMotionControllerState.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
const bool bGotMotionControllerState =
FSGXRTracker::GetMotionControllerState(
this, XRSpaceType, Hand, XRControllerPoseType, OutMotionControllerState);
if (!bGotMotionControllerState || !OutMotionControllerState.bValid)
{
return false;
}
return OutMotionControllerState.bValid;
}
bool USGVirtualHandComponent::GetHandTrackingState(
const EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState)
{
OutHandTrackingState.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
const bool bGotHandTrackingState =
FSGXRTracker::GetHandTrackingState(
this, XRSpaceType, Hand, OutHandTrackingState);
if (!bGotHandTrackingState || !OutHandTrackingState.bValid)
{
return false;
}
return OutHandTrackingState.bValid;
}
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
const TArray<FName>& USGVirtualHandComponent::GetHandBoneNames() const const TArray<FName>& USGVirtualHandComponent::GetHandBoneNames() const
{ {
const TArray<FName>& Names{ const TArray<FName>& Names{
@@ -747,12 +789,25 @@ void USGVirtualHandComponent::FImpl::SetVisibility(const bool bInVisible)
void USGVirtualHandComponent::FImpl::UpdateVisibility() void USGVirtualHandComponent::FImpl::UpdateVisibility()
{ {
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
FXRHandTrackingState HandTrackingState;
const bool bGotHandTrackingState =
Owner->GetHandTrackingState(EXRSpaceType::UnrealWorldSpace, HandTrackingState);
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FXRMotionControllerData MotionControllerData; FXRMotionControllerData MotionControllerData;
const bool bGotMotionControllerData = Owner->GetMotionControllerData(MotionControllerData); const bool bGotMotionControllerData = Owner->GetMotionControllerData(MotionControllerData);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
const bool bHandDataAvailable = const bool bHandDataAvailable =
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bGotHandTrackingState
&& HandTrackingState.bValid
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
bGotMotionControllerData bGotMotionControllerData
&& MotionControllerData.bValid && MotionControllerData.bValid
&& MotionControllerData.DeviceVisualType == EXRVisualType::Hand; && MotionControllerData.DeviceVisualType == EXRVisualType::Hand
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
;
const bool bHandVisible = Owner->IsVisible(); const bool bHandVisible = Owner->IsVisible();
const bool bShouldBeVisibleWhenHandDataUnavailable = const bool bShouldBeVisibleWhenHandDataUnavailable =
Owner->GetVirtualHandSettings().bVisibleWhenHandDataUnavailable; Owner->GetVirtualHandSettings().bVisibleWhenHandDataUnavailable;
@@ -811,6 +866,16 @@ void USGVirtualHandComponent::FImpl::DrawDebugVirtualHand() const
return; return;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
FXRHandTrackingState HandTrackingState;
const bool bGotHandTrackingState = Owner->GetHandTrackingState(EXRSpaceType::UnrealWorldSpace, HandTrackingState);
if (!bGotHandTrackingState || !HandTrackingState.bValid)
{
return;
}
FSGDebugVirtualHand::Draw(World, HandTrackingState, Owner->GetVirtualHandSettings().DebuggingSettings);
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FXRMotionControllerData MotionControllerData; FXRMotionControllerData MotionControllerData;
const bool bGotMotionControllerData = Owner->GetMotionControllerData(MotionControllerData); const bool bGotMotionControllerData = Owner->GetMotionControllerData(MotionControllerData);
if (!bGotMotionControllerData || !MotionControllerData.bValid) if (!bGotMotionControllerData || !MotionControllerData.bValid)
@@ -819,6 +884,7 @@ void USGVirtualHandComponent::FImpl::DrawDebugVirtualHand() const
} }
FSGDebugVirtualHand::Draw(World, MotionControllerData, Owner->GetVirtualHandSettings().DebuggingSettings); FSGDebugVirtualHand::Draw(World, MotionControllerData, Owner->GetVirtualHandSettings().DebuggingSettings);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
} }
void USGVirtualHandComponent::FImpl::TriggerHandVisibilityChangedEvent(const bool bNewVisibility) const void USGVirtualHandComponent::FImpl::TriggerHandVisibilityChangedEvent(const bool bNewVisibility) const
@@ -246,8 +246,11 @@ bool USGWristTrackerComponent::GetMotionControllerData(FXRMotionControllerData&
OutMotionControllerData.bValid = false; OutMotionControllerData.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left; const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
PRAGMA_DISABLE_DEPRECATION_WARNINGS
const bool bGotMotionControllerData = const bool bGotMotionControllerData =
FSGXRTracker::GetMotionControllerData(this, Hand, OutMotionControllerData); FSGXRTracker::GetMotionControllerData(this, Hand, OutMotionControllerData);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
if (!bGotMotionControllerData || !OutMotionControllerData.bValid) if (!bGotMotionControllerData || !OutMotionControllerData.bValid)
{ {
return false; return false;
@@ -256,6 +259,46 @@ bool USGWristTrackerComponent::GetMotionControllerData(FXRMotionControllerData&
return OutMotionControllerData.bValid; return OutMotionControllerData.bValid;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool USGWristTrackerComponent::GetMotionControllerState(
const EXRSpaceType XRSpaceType, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
OutMotionControllerState.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
const bool bGotMotionControllerState =
FSGXRTracker::GetMotionControllerState(
this, XRSpaceType, Hand, XRControllerPoseType, OutMotionControllerState);
if (!bGotMotionControllerState || !OutMotionControllerState.bValid)
{
return false;
}
return OutMotionControllerState.bValid;
}
bool USGWristTrackerComponent::GetHandTrackingState(
const EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState)
{
OutHandTrackingState.bValid = false;
const EControllerHand Hand = IsRight() ? EControllerHand::Right : EControllerHand::Left;
const bool bGotHandTrackingState =
FSGXRTracker::GetHandTrackingState(
this, XRSpaceType, Hand, OutHandTrackingState);
if (!bGotHandTrackingState || !OutHandTrackingState.bValid)
{
return false;
}
return OutHandTrackingState.bValid;
}
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
USGWristTrackerComponent::FImpl::FImpl(USGWristTrackerComponent* InOwner) USGWristTrackerComponent::FImpl::FImpl(USGWristTrackerComponent* InOwner)
: bOverridePluginSettingsPropertyAlreadyModified(false), : bOverridePluginSettingsPropertyAlreadyModified(false),
Owner(InOwner) Owner(InOwner)
@@ -67,7 +67,12 @@ private:
bool bVirtualHandVisible; bool bVirtualHandVisible;
bool bMarkRenderStateDirty; bool bMarkRenderStateDirty;
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
FXRHandTrackingState HandTrackingState;
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FXRMotionControllerData MotionControllerData; FXRMotionControllerData MotionControllerData;
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
TMap<TArray<FQuat>::SizeType, FName> BoneNamesMap; TMap<TArray<FQuat>::SizeType, FName> BoneNamesMap;
FTransform AnimationBoneCorrectionOffsetTransform; FTransform AnimationBoneCorrectionOffsetTransform;
@@ -103,10 +108,17 @@ protected:
return bMarkRenderStateDirty; return bMarkRenderStateDirty;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
FORCEINLINE const FXRHandTrackingState& GetHandTrackingState() const
{
return HandTrackingState;
}
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FORCEINLINE const FXRMotionControllerData& GetMotionControllerData() const FORCEINLINE const FXRMotionControllerData& GetMotionControllerData() const
{ {
return MotionControllerData; return MotionControllerData;
} }
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FORCEINLINE const TMap<TArray<FQuat>::SizeType, FName>& GetBoneNamesMap() const FORCEINLINE const TMap<TArray<FQuat>::SizeType, FName>& GetBoneNamesMap() const
{ {
@@ -175,8 +175,19 @@ protected:
public: public:
bool IsGloveConnected() const; bool IsGloveConnected() const;
USGHapticGlove* GetConnectedGlove() const; USGHapticGlove* GetConnectedGlove() const;
UE_DEPRECATED(5.5, "Deprecated by UE 5.5 in favor of GetMotionControllerState and GetHandTrackingState")
bool GetMotionControllerData(FXRMotionControllerData& OutMotionControllerData); bool GetMotionControllerData(FXRMotionControllerData& OutMotionControllerData);
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool GetMotionControllerState(
EXRSpaceType XRSpaceType, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
bool GetHandTrackingState(
EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
public: public:
const TArray<FName>& GetHandBoneNames() const; const TArray<FName>& GetHandBoneNames() const;
const FName& GetHandBoneName(int32 Joint) const; const FName& GetHandBoneName(int32 Joint) const;
@@ -164,5 +164,15 @@ protected:
FActorComponentTickFunction* ThisTickFunction); FActorComponentTickFunction* ThisTickFunction);
public: public:
UE_DEPRECATED(5.5, "Deprecated by UE 5.5 in favor of GetMotionControllerState and GetHandTrackingState")
bool GetMotionControllerData(FXRMotionControllerData& OutMotionControllerData); bool GetMotionControllerData(FXRMotionControllerData& OutMotionControllerData);
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool GetMotionControllerState(
EXRSpaceType XRSpaceType, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
bool GetHandTrackingState(
EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
}; };
@@ -73,3 +73,35 @@ void FSGDebugVirtualHand::Draw(const UWorld* World,
} }
} }
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
void FSGDebugVirtualHand::Draw(const UWorld* World,
const FXRHandTrackingState& HandTrackingState,
const FSGVirtualHandDebuggingSettings& Settings)
{
ensureAlwaysMsgf(Settings.DrawingMode != ESGDebugVirtualHandDrawingMode::None,
TEXT("Invalid virtual hand drawing mode!"));
ensureAlwaysMsgf(HandTrackingState.bValid,
TEXT("Invalid hand tracking state!"));
ensureAlwaysMsgf(HandTrackingState.HandKeyLocations.Num() == HandTrackingState.HandKeyRotations.Num(),
TEXT("Mismatched hand tracking state locations and rotations!"));
for (TArray<FVector>::SizeType HandKeyIndex = 0;
HandKeyIndex < HandTrackingState.HandKeyLocations.Num(); ++HandKeyIndex)
{
const FVector& HandKeyLocation{HandTrackingState.HandKeyLocations[HandKeyIndex]};
const FQuat& HandKeyRotation{HandTrackingState.HandKeyRotations[HandKeyIndex]};
if (Settings.DrawingMode == ESGDebugVirtualHandDrawingMode::CubicJoints)
{
FSGDebugCube::Draw(World, HandKeyLocation, HandKeyRotation, Settings.DebugCubicHandSettings);
}
else if (Settings.DrawingMode == ESGDebugVirtualHandDrawingMode::GizmoJoints)
{
FSGDebugGizmo::Draw(World, HandKeyLocation, HandKeyRotation, Settings.DebugGizmoHandSettings);
}
}
}
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
@@ -47,11 +47,19 @@ class UWorld;
class SENSEGLOVEDEBUG_API FSGDebugVirtualHand final class SENSEGLOVEDEBUG_API FSGDebugVirtualHand final
{ {
public: public:
UE_DEPRECATED(5.5, "Deprecated in favor of the variant that accepts FXRHandTrackingState")
static void Draw( static void Draw(
const UWorld* World, const UWorld* World,
const FXRMotionControllerData& MotionControllerData, const FXRMotionControllerData& MotionControllerData,
const FSGVirtualHandDebuggingSettings& Settings); const FSGVirtualHandDebuggingSettings& Settings);
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
static void Draw(
const UWorld* World,
const FXRHandTrackingState& HandTrackingState,
const FSGVirtualHandDebuggingSettings& Settings);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
public: public:
FSGDebugVirtualHand() = delete; FSGDebugVirtualHand() = delete;
virtual ~FSGDebugVirtualHand() = delete; virtual ~FSGDebugVirtualHand() = delete;
@@ -37,11 +37,24 @@
#include "SGDebug/SGDebugVirtualHand.h" #include "SGDebug/SGDebugVirtualHand.h"
void USGDebugVirtualHandKismetLibrary::Draw( void USGDebugVirtualHandKismetLibrary::Draw_FXRMotionControllerData(
UObject* WorldContextObject, UObject* WorldContextObject,
const FXRMotionControllerData& MotionControllerData, const FSGVirtualHandDebuggingSettings& Settings) const FXRMotionControllerData& MotionControllerData,
const FSGVirtualHandDebuggingSettings& Settings)
{ {
ensureAlwaysMsgf(WorldContextObject, TEXT("Invalid world context object!")); ensureAlwaysMsgf(WorldContextObject, TEXT("Invalid world context object!"));
PRAGMA_DISABLE_DEPRECATION_WARNINGS
FSGDebugVirtualHand::Draw(WorldContextObject->GetWorld(), MotionControllerData, Settings); FSGDebugVirtualHand::Draw(WorldContextObject->GetWorld(), MotionControllerData, Settings);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
void USGDebugVirtualHandKismetLibrary::Draw_FXRHandTrackingState(
UObject* WorldContextObject,
const FXRHandTrackingState& HandTrackingState,
const FSGVirtualHandDebuggingSettings& Settings)
{
ensureAlwaysMsgf(WorldContextObject, TEXT("Invalid world context object!"));
FSGDebugVirtualHand::Draw(WorldContextObject->GetWorld(), HandTrackingState, Settings);
} }
@@ -35,8 +35,10 @@
#pragma once #pragma once
#include "UObject/ObjectMacros.h"
#include "HeadMountedDisplayTypes.h"
#include "SGKismet/SGBlueprintFunctionLibrary.h" #include "SGKismet/SGBlueprintFunctionLibrary.h"
#include "SGDebug/SGDebugVirtualHand.h"
#include "SGDebugVirtualHandKismetLibrary.generated.h" #include "SGDebugVirtualHandKismetLibrary.generated.h"
@@ -48,10 +50,17 @@ class SENSEGLOVEDEBUGKISMET_API USGDebugVirtualHandKismetLibrary final : public
GENERATED_BODY() GENERATED_BODY()
public: public:
UFUNCTION(BlueprintCallable, Category="SenseGlove | Debug | Virtual Hand", UFUNCTION(BlueprintCallable, DisplayName="Draw", Category="SenseGlove | Debug | Virtual Hand",
meta=(WorldContext="WorldContextObject")) meta=(WorldContext="WorldContextObject"))
static void Draw( static void Draw_FXRMotionControllerData(
UObject* WorldContextObject, UObject* WorldContextObject,
const FXRMotionControllerData& MotionControllerData, const FXRMotionControllerData& MotionControllerData,
const FSGVirtualHandDebuggingSettings& Settings); const FSGVirtualHandDebuggingSettings& Settings);
UFUNCTION(BlueprintCallable, DisplayName="Draw", Category="SenseGlove | Debug | Virtual Hand",
meta=(WorldContext="WorldContextObject"))
static void Draw_FXRHandTrackingState(
UObject* WorldContextObject,
const FXRHandTrackingState& HandTrackingState,
const FSGVirtualHandDebuggingSettings& Settings);
}; };
@@ -113,7 +113,23 @@ USGHapticGlove* UVirtualHandComponentKismetLibrary::GetConnectedGlove(
bool UVirtualHandComponentKismetLibrary::GetMotionControllerData( bool UVirtualHandComponentKismetLibrary::GetMotionControllerData(
USGVirtualHandComponent* VirtualHandComponent, FXRMotionControllerData& OutMotionControllerData) USGVirtualHandComponent* VirtualHandComponent, FXRMotionControllerData& OutMotionControllerData)
{ {
PRAGMA_DISABLE_DEPRECATION_WARNINGS
return VirtualHandComponent->GetMotionControllerData(OutMotionControllerData); return VirtualHandComponent->GetMotionControllerData(OutMotionControllerData);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
bool UVirtualHandComponentKismetLibrary::GetMotionControllerState(USGVirtualHandComponent* VirtualHandComponent,
const EXRSpaceType XRSpaceType, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
return VirtualHandComponent->GetMotionControllerState(
XRSpaceType, XRControllerPoseType, OutMotionControllerState);
}
bool UVirtualHandComponentKismetLibrary::GetHandTrackingState(USGVirtualHandComponent* VirtualHandComponent,
const EXRSpaceType XRSpaceType, FXRHandTrackingState& OutHandTrackingState)
{
return VirtualHandComponent->GetHandTrackingState(XRSpaceType, OutHandTrackingState);
} }
const TArray<FName>& UVirtualHandComponentKismetLibrary::GetHandBoneNames( const TArray<FName>& UVirtualHandComponentKismetLibrary::GetHandBoneNames(
@@ -93,5 +93,24 @@ const FRotator& UWristTrackerComponentKismetLibrary::GetWristRotation(
bool UWristTrackerComponentKismetLibrary::GetMotionControllerData( bool UWristTrackerComponentKismetLibrary::GetMotionControllerData(
USGWristTrackerComponent* WristTrackerComponent, FXRMotionControllerData& OutMotionControllerData) USGWristTrackerComponent* WristTrackerComponent, FXRMotionControllerData& OutMotionControllerData)
{ {
PRAGMA_DISABLE_DEPRECATION_WARNINGS
return WristTrackerComponent->GetMotionControllerData(OutMotionControllerData); return WristTrackerComponent->GetMotionControllerData(OutMotionControllerData);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
bool UWristTrackerComponentKismetLibrary::GetMotionControllerState(
USGWristTrackerComponent* WristTrackerComponent,
const EXRSpaceType XRSpaceType, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
return WristTrackerComponent->GetMotionControllerState(
XRSpaceType, XRControllerPoseType, OutMotionControllerState);
}
bool UWristTrackerComponentKismetLibrary::GetHandTrackingState(
USGWristTrackerComponent* WristTrackerComponent,
const EXRSpaceType XRSpaceType, FXRHandTrackingState& OutHandTrackingState)
{
return WristTrackerComponent->GetHandTrackingState(
XRSpaceType, OutHandTrackingState);
} }
@@ -106,6 +106,16 @@ public:
static bool GetMotionControllerData(UPARAM(ref) USGVirtualHandComponent* VirtualHandComponent, static bool GetMotionControllerData(UPARAM(ref) USGVirtualHandComponent* VirtualHandComponent,
FXRMotionControllerData& OutMotionControllerData); FXRMotionControllerData& OutMotionControllerData);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Virtual Hand Component")
static bool GetMotionControllerState(UPARAM(ref) USGVirtualHandComponent* VirtualHandComponent,
EXRSpaceType XRSpaceType, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Virtual Hand Component")
static bool GetHandTrackingState(UPARAM(ref) USGVirtualHandComponent* VirtualHandComponent,
EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState);
UFUNCTION(BlueprintPure, Category="SenseGlove | Components | Virtual Hand Component") UFUNCTION(BlueprintPure, Category="SenseGlove | Components | Virtual Hand Component")
static const TArray<FName>& GetHandBoneNames(const USGVirtualHandComponent* VirtualHandComponent); static const TArray<FName>& GetHandBoneNames(const USGVirtualHandComponent* VirtualHandComponent);
@@ -86,4 +86,14 @@ public:
UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Wrist Tracker Component") UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Wrist Tracker Component")
static bool GetMotionControllerData(UPARAM(ref) USGWristTrackerComponent* WristTrackerComponent, static bool GetMotionControllerData(UPARAM(ref) USGWristTrackerComponent* WristTrackerComponent,
FXRMotionControllerData& OutMotionControllerData); FXRMotionControllerData& OutMotionControllerData);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Wrist Tracker Component")
static bool GetMotionControllerState(UPARAM(ref) USGWristTrackerComponent* WristTrackerComponent,
EXRSpaceType XRSpaceType, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Components | Wrist Tracker Component")
static bool GetHandTrackingState(UPARAM(ref) USGWristTrackerComponent* WristTrackerComponent,
EXRSpaceType XRSpaceType,
FXRHandTrackingState& OutHandTrackingState);
}; };
@@ -43,7 +43,8 @@ FSGXRHandState::FSGXRHandState()
Locations{XR_TYPE_HAND_JOINT_LOCATIONS_EXT}, Locations{XR_TYPE_HAND_JOINT_LOCATIONS_EXT},
KeypointTransforms{}, KeypointTransforms{},
Radii{}, Radii{},
bReceivedJointPoses(false) bHasReceivedJointPoses(false),
bTracked(false)
{ {
Velocities.jointCount = XR_HAND_JOINT_COUNT_EXT; Velocities.jointCount = XR_HAND_JOINT_COUNT_EXT;
Velocities.jointVelocities = JointVelocities; Velocities.jointVelocities = JointVelocities;
@@ -60,7 +61,7 @@ bool FSGXRHandState::GetTransform(const EHandKeypoint Keypoint, FTransform& OutT
check(static_cast<int32>(Keypoint) < EHandKeypointCount); check(static_cast<int32>(Keypoint) < EHandKeypointCount);
OutTransform = KeypointTransforms[static_cast<uint32>(Keypoint)]; OutTransform = KeypointTransforms[static_cast<uint32>(Keypoint)];
return bReceivedJointPoses; return bHasReceivedJointPoses;
} }
const FTransform& FSGXRHandState::GetTransform(const EHandKeypoint Keypoint) const const FTransform& FSGXRHandState::GetTransform(const EHandKeypoint Keypoint) const
@@ -998,7 +998,9 @@ FTransform FSGXRTracker::FImpl::GetMeshFingerStartBoneRefTransform(
} }
bool FSGXRTracker::GetMotionControllerData( bool FSGXRTracker::GetMotionControllerData(
UObject* WorldContextObject, const EControllerHand Hand, FXRMotionControllerData& OutMotionControllerData) UObject* WorldContextObject,
const EControllerHand Hand,
FXRMotionControllerData& OutMotionControllerData)
{ {
OutMotionControllerData.DeviceName = NAME_None; OutMotionControllerData.DeviceName = NAME_None;
OutMotionControllerData.ApplicationInstanceID = FApp::GetInstanceId(); OutMotionControllerData.ApplicationInstanceID = FApp::GetInstanceId();
@@ -1163,16 +1165,14 @@ bool FSGXRTracker::GetMotionControllerData(
{ {
OutMotionControllerData.DeviceVisualType = EXRVisualType::Hand; OutMotionControllerData.DeviceVisualType = EXRVisualType::Hand;
/// TODO
// This is a just a temporary workaround to silence a UE 5.5 deprecation warning.
// Should be refactored with a proper implementation when migrating away from the deprecated
// `FXRMotionControllerData` in favor of newly introduced
// `FXRMotionControllerState` and `FXRHandTrackingState` structs.
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 #if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool bTracked = false; bool bTracked = false;
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */ #endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
OutMotionControllerData.bValid = HandTracker->GetAllKeypointStates( OutMotionControllerData.bValid = HandTracker->GetAllKeypointStates(
Hand, OutMotionControllerData.HandKeyPositions, OutMotionControllerData.HandKeyRotations, Hand,
OutMotionControllerData.HandKeyPositions,
OutMotionControllerData.HandKeyRotations,
OutMotionControllerData.HandKeyRadii OutMotionControllerData.HandKeyRadii
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 #if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
, bTracked , bTracked
@@ -1180,10 +1180,25 @@ bool FSGXRTracker::GetMotionControllerData(
); );
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 #if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
if (!bTracked) // Unused!
(void) bTracked;
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
// TODO
// Begin backward compatibility with the deprecated function, remove this block along with the deprecated
// GetAllKeypointStates (though it should be time to remove this entire function as well)!
if (!OutMotionControllerData.bValid)
{ {
return false; PRAGMA_DISABLE_DEPRECATION_WARNINGS
OutMotionControllerData.bValid = HandTracker->GetAllKeypointStates(
Hand,
OutMotionControllerData.HandKeyPositions,
OutMotionControllerData.HandKeyRotations,
OutMotionControllerData.HandKeyRadii);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
} }
// End backward compability.
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */ #endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
if (!ensureAlwaysMsgf( if (!ensureAlwaysMsgf(
@@ -1204,13 +1219,293 @@ bool FSGXRTracker::GetMotionControllerData(
OutMotionControllerData.TrackingStatus = ETrackingStatus::Tracked; OutMotionControllerData.TrackingStatus = ETrackingStatus::Tracked;
} }
/// TODO
// This is reportedly a WMR-specific convenience function for rapid prototyping. Not sure if it's useful for OpenXR.
OutMotionControllerData.bIsGrasped = false;
return OutMotionControllerData.bValid; return OutMotionControllerData.bValid;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool FSGXRTracker::GetMotionControllerState(
UObject* WorldContextObject,
const EXRSpaceType XRSpaceType, const EControllerHand Hand, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
auto ToMotionSourceName = [](
const EControllerHand Hand, const EXRControllerPoseType XRControllerPoseType) -> FLazyName
{
static FLazyName LeftAim = "LeftAim";
static FLazyName LeftGrip = "LeftGrip";
static FLazyName LeftPalm = "LeftPalm";
static FLazyName RightAim = "RightAim";
static FLazyName RightGrip = "RightGrip";
static FLazyName RightPalm = "RightPalm";
if (Hand == EControllerHand::Left)
{
switch (XRControllerPoseType)
{
case EXRControllerPoseType::Aim:
return LeftAim;
case EXRControllerPoseType::Grip:
return LeftGrip;
case EXRControllerPoseType::Palm:
return LeftPalm;
default:
check(false);
return LeftGrip;
}
}
else
{
switch (XRControllerPoseType)
{
case EXRControllerPoseType::Aim:
return RightAim;
case EXRControllerPoseType::Grip:
return RightGrip;
case EXRControllerPoseType::Palm:
return RightPalm;
default:
check(false);
return RightGrip;
}
}
};
OutMotionControllerState.DeviceName = NAME_None;
OutMotionControllerState.ApplicationInstanceID = FApp::GetInstanceId();
OutMotionControllerState.TrackingStatus = ETrackingStatus::NotTracked;
OutMotionControllerState.Hand = Hand;
OutMotionControllerState.XRSpaceType = XRSpaceType;
OutMotionControllerState.bValid = false;
if (!IsValid(WorldContextObject))
{
return false;
}
if (!IsValid(WorldContextObject))
{
return false;
}
IXRTrackingSystem* XRTrackingSystem{GEngine ? GEngine->XRSystem.Get() : nullptr};
if (!XRTrackingSystem)
{
return false;
}
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4
IOpenXRHMD* OpenXRHMD{XRTrackingSystem->GetIOpenXRHMD()};
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4 */
FOpenXRHMD* OpenXRHMD{static_cast<FOpenXRHMD*>(GEngine->XRSystem.Get())};
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4 */
if (!OpenXRHMD)
{
return false;
}
TArray<int32> Devices;
const int32 HandIndex = static_cast<int32>(Hand);
const bool bGotDevices =
XRTrackingSystem->EnumerateTrackedDevices(Devices, EXRTrackedDeviceType::Controller);
if (bGotDevices && Devices.IsValidIndex(HandIndex))
{
const XrSession Session{OpenXRHMD->GetSession()};
if (Session)
{
XrInteractionProfileState Profile = {XR_TYPE_INTERACTION_PROFILE_STATE};
const XrPath TopLevelUserPath{OpenXRHMD->GetTrackedDevicePath(Devices[HandIndex])};
const XrResult CurrentInteractionProfile{
xrGetCurrentInteractionProfile(Session, TopLevelUserPath, &Profile)
};
if (XR_SUCCEEDED(CurrentInteractionProfile) && Profile.interactionProfile != XR_NULL_PATH)
{
OutMotionControllerState.DeviceName = FOpenXRPath(Profile.interactionProfile);
}
}
}
if ((Hand == EControllerHand::Left) || (Hand == EControllerHand::Right))
{
static const FName MotionControllerName("OpenXR");
TArray<IMotionController*> MotionControllers{
IModularFeatures::Get().GetModularFeatureImplementations<IMotionController>(
IMotionController::GetModularFeatureName())
};
IMotionController* MotionController{nullptr};
for (IMotionController* MotionControllerIterator: MotionControllers)
{
if (!MotionControllerIterator)
{
continue;
}
const FName DeviceTypeName{MotionControllerIterator->GetMotionControllerDeviceTypeName()};
if (DeviceTypeName != MotionControllerName)
{
continue;
}
MotionController = MotionControllerIterator;
break;
}
if (MotionController)
{
{
// Handle the pose that is actually being requested.
const FName MotionSource{ToMotionSourceName(Hand, XRControllerPoseType)};
const FTransform TrackingToWorld{
XRSpaceType == EXRSpaceType::UnrealWorldSpace
? XRTrackingSystem->GetTrackingToWorldTransform()
: FTransform::Identity
};
const float WorldToMeters = XRSpaceType == EXRSpaceType::UnrealWorldSpace
? XRTrackingSystem->GetWorldToMetersScale()
: 100.0f;
FVector Position{FVector::ZeroVector};
FRotator Rotation{FRotator::ZeroRotator};
bool bGotMotionControllerOrientationAndPosition = MotionController->GetControllerOrientationAndPosition(
0, MotionSource, Rotation, Position, WorldToMeters);
if (bGotMotionControllerOrientationAndPosition)
{
OutMotionControllerState.ControllerLocation = TrackingToWorld.TransformPosition(Position);
OutMotionControllerState.ControllerRotation = TrackingToWorld.TransformRotation(FQuat{Rotation});
}
OutMotionControllerState.bValid |= bGotMotionControllerOrientationAndPosition;
OutMotionControllerState.TrackingStatus =
MotionController->GetControllerTrackingStatus(0, MotionSource);
}
{
// We always provide the grip transform in Unreal space for XRVisualizationFunctionLibrary;
// The bValid and TrackingStatus above are also valid for this pose.
const FName MotionSource = ToMotionSourceName(Hand, EXRControllerPoseType::Grip);
const FTransform TrackingToWorld{XRTrackingSystem->GetTrackingToWorldTransform()};
const float WorldToMeters = XRTrackingSystem->GetWorldToMetersScale();
FVector Position = FVector::ZeroVector;
FRotator Rotation = FRotator::ZeroRotator;
bool bGotMotionControllerOrientationAndPosition = MotionController->GetControllerOrientationAndPosition(
0, MotionSource, Rotation, Position, WorldToMeters);
if (bGotMotionControllerOrientationAndPosition)
{
OutMotionControllerState.GripUnrealSpaceLocation = TrackingToWorld.TransformPosition(Position);
OutMotionControllerState.GripUnrealSpaceRotation =
TrackingToWorld.TransformRotation(FQuat(Rotation));
}
}
}
}
return OutMotionControllerState.bValid;
}
bool FSGXRTracker::GetHandTrackingState(
UObject* WorldContextObject,
const EXRSpaceType XRSpaceType, const EControllerHand Hand,
FXRHandTrackingState& OutHandTrackingState)
{
OutHandTrackingState.ApplicationInstanceID = FApp::GetInstanceId();
OutHandTrackingState.TrackingStatus = ETrackingStatus::NotTracked;
OutHandTrackingState.Hand = Hand;
OutHandTrackingState.XRSpaceType = XRSpaceType;
OutHandTrackingState.bValid = false;
if (!IsValid(WorldContextObject))
{
return false;
}
IXRTrackingSystem* XRTrackingSystem{GEngine ? GEngine->XRSystem.Get() : nullptr};
if (!XRTrackingSystem)
{
return false;
}
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4
IOpenXRHMD* OpenXRHMD{XRTrackingSystem->GetIOpenXRHMD()};
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4 */
FOpenXRHMD* OpenXRHMD{static_cast<FOpenXRHMD*>(GEngine->XRSystem.Get())};
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 4 */
if (!OpenXRHMD)
{
return false;
}
FName HandTrackerName("OpenXRHandTracking");
TArray<IHandTracker*> HandTrackers = IModularFeatures::Get().GetModularFeatureImplementations<IHandTracker>(
IHandTracker::GetModularFeatureName());
IHandTracker* HandTracker = nullptr;
for (auto Itr: HandTrackers)
{
if (Itr->GetHandTrackerDeviceTypeName() == HandTrackerName)
{
HandTracker = Itr;
break;
}
}
if ((Hand == EControllerHand::Left) || (Hand == EControllerHand::Right))
{
const bool bHandTrackingStateValid = HandTracker && HandTracker->IsHandTrackingStateValid();
if (bHandTrackingStateValid)
{
bool bTracked = false;
OutHandTrackingState.bValid = HandTracker->GetAllKeypointStates(
Hand,
OutHandTrackingState.HandKeyLocations,
OutHandTrackingState.HandKeyRotations,
OutHandTrackingState.HandKeyRadii,
bTracked);
/// TODO
// Begin backward compatibility with the deprecated function, remove this block along with the deprecated
// GetAllKeypointStates.
if (!OutHandTrackingState.bValid)
{
PRAGMA_DISABLE_DEPRECATION_WARNINGS
OutHandTrackingState.bValid = HandTracker->GetAllKeypointStates(
Hand,
OutHandTrackingState.HandKeyLocations,
OutHandTrackingState.HandKeyRotations,
OutHandTrackingState.HandKeyRadii);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
bTracked = OutHandTrackingState.bValid;
// The inability to distinguish this is the reason for the deprecation.
}
// End backward compability.
if (!ensureAlwaysMsgf(
!OutHandTrackingState.bValid
|| (OutHandTrackingState.HandKeyLocations.Num() == EHandKeypointCount
&& OutHandTrackingState.HandKeyRotations.Num() == EHandKeypointCount
&& OutHandTrackingState.HandKeyRadii.Num() == EHandKeypointCount),
TEXT("Corrupted OutHandTrackingState!")))
{
OutHandTrackingState.bValid = false;
return false;
}
if (OutHandTrackingState.bValid)
{
OutHandTrackingState.TrackingStatus = bTracked ? ETrackingStatus::Tracked : ETrackingStatus::NotTracked;
}
}
}
return OutHandTrackingState.bValid;
}
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
FSGXRTracker::FSGXRTracker(const TSharedRef<FGenericApplicationMessageHandler>& InMessageHandler) FSGXRTracker::FSGXRTracker(const TSharedRef<FGenericApplicationMessageHandler>& InMessageHandler)
: Pimpl(TUniquePtr<FImpl, FImplDeleter>(new FImpl{this}, PimplDeleter)) : Pimpl(TUniquePtr<FImpl, FImplDeleter>(new FImpl{this}, PimplDeleter))
{ {
@@ -1480,7 +1775,7 @@ ETrackingStatus FSGXRTracker::GetControllerTrackingStatus(const int32 Controller
if (bFallbackToHandTrackingIfNoGloveDetected) if (bFallbackToHandTrackingIfNoGloveDetected)
{ {
const FSGXRHandState& HandState{bRight ? GetRightHandState() : GetLeftHandState()}; const FSGXRHandState& HandState{bRight ? GetRightHandState() : GetLeftHandState()};
return HandState.bReceivedJointPoses ? ETrackingStatus::Tracked : ETrackingStatus::NotTracked; return HandState.bTracked ? ETrackingStatus::Tracked : ETrackingStatus::NotTracked;
} }
return ETrackingStatus::NotTracked; return ETrackingStatus::NotTracked;
@@ -1606,9 +1901,17 @@ bool FSGXRTracker::GetKeypointState(
return bGotTransform; return bGotTransform;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
bool FSGXRTracker::GetAllKeypointStates(
const EControllerHand Hand,
TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii,
bool& OutIsTracked) const
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
bool FSGXRTracker::GetAllKeypointStates( bool FSGXRTracker::GetAllKeypointStates(
const EControllerHand Hand, const EControllerHand Hand,
TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii) const TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii) const
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5*/
{ {
if (!IsHandTrackingStateValid()) if (!IsHandTrackingStateValid())
{ {
@@ -1629,11 +1932,17 @@ bool FSGXRTracker::GetAllKeypointStates(
Hand == EControllerHand::Left ? GetLeftHandState() : GetRightHandState() Hand == EControllerHand::Left ? GetLeftHandState() : GetRightHandState()
}; };
if (!HandState.bReceivedJointPoses) if (!HandState.bHasReceivedJointPoses)
{ {
return false; return false;
} }
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
OutIsTracked = HandState.bTracked;
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5*/
// If we ever received poses we return the last data we received, even if not currently tracking to avoid poping
// back to (0.0f, 0.0f, 0.0f) and the like.
OutPositions.Empty(EHandKeypointCount); OutPositions.Empty(EHandKeypointCount);
OutRotations.Empty(EHandKeypointCount); OutRotations.Empty(EHandKeypointCount);
OutRadii.Empty(EHandKeypointCount); OutRadii.Empty(EHandKeypointCount);
@@ -1765,8 +2074,9 @@ bool FSGXRTracker::FImpl::UpdateXRHandTrackingData(
XR_ENSURE(XRLocateHandJointsEXT(OutHandState.HandTracker, &LocateInfo, &OutHandState.Locations)); XR_ENSURE(XRLocateHandJointsEXT(OutHandState.HandTracker, &LocateInfo, &OutHandState.Locations));
OutHandState.bReceivedJointPoses = OutHandState.Locations.isActive == XR_TRUE; OutHandState.bTracked = OutHandState.Locations.isActive == XR_TRUE;
if (!OutHandState.bReceivedJointPoses) OutHandState.bHasReceivedJointPoses |= HandStates->bTracked;
if (!OutHandState.bTracked)
{ {
return false; return false;
} }
@@ -1893,21 +2203,16 @@ bool FSGXRTracker::FImpl::GetControllerTransform(
if (!bTracked && bFallbackToHandTrackingIfNoGloveDetected) if (!bTracked && bFallbackToHandTrackingIfNoGloveDetected)
{ {
const FSGXRHandState& HandState{bRight ? Owner->GetRightHandState() : Owner->GetLeftHandState()}; const FSGXRHandState& HandState{bRight ? Owner->GetRightHandState() : Owner->GetLeftHandState()};
if (!HandState.bReceivedJointPoses) if (!HandState.bHasReceivedJointPoses)
{ {
return false; return false;
} }
const EHandKeypoint KeyPoint = KeyPointInfoPtr->Key; const EHandKeypoint KeyPoint = KeyPointInfoPtr->Key;
const FTransform& ControllerTransform{ const FTransform& ControllerTransform{HandStates->GetTransform(KeyPoint)};
bRight
? GetRightHandState().GetTransform(KeyPoint)
: GetLeftHandState().GetTransform(KeyPoint)
};
OutTransform = ControllerTransform; OutTransform = ControllerTransform;
bOutMotionSourceHandTracking = true; bOutMotionSourceHandTracking = true;
bTracked = true; bTracked = HandState.bTracked;
} }
return bTracked; return bTracked;
@@ -2404,7 +2709,8 @@ bool FSGXRTracker::FImpl::UpdateSGHandState(const bool bRight, FSGXRHandState& O
return false; return false;
} }
OutHandState.bReceivedJointPoses = true; OutHandState.bHasReceivedJointPoses = true;
OutHandState.bTracked = true;
const TArray<TArray<FVector>> JointLocations{HandPose->GetJointPositions()}; const TArray<TArray<FVector>> JointLocations{HandPose->GetJointPositions()};
const TArray<TArray<FQuat>> JointRotations{HandPose->GetJointRotationsQ()}; const TArray<TArray<FQuat>> JointRotations{HandPose->GetJointRotationsQ()};
@@ -51,12 +51,16 @@ public:
XrHandJointVelocitiesEXT Velocities; XrHandJointVelocitiesEXT Velocities;
XrHandJointLocationsEXT Locations; XrHandJointLocationsEXT Locations;
// Transforms are cached in Unreal Tracking Space // Transforms are cached in Unreal Tracking Space.
FTransform KeypointTransforms[EHandKeypointCount]; FTransform KeypointTransforms[EHandKeypointCount];
float Radii[EHandKeypointCount]; float Radii[EHandKeypointCount];
bool bReceivedJointPoses; // true if this hand has ever been tracked, and therefore its poses are valid.
bool bHasReceivedJointPoses;
// true if tracked right now.
bool bTracked;
public: public:
FSGXRHandState(); FSGXRHandState();
@@ -55,9 +55,23 @@ class SENSEGLOVETRACKING_API FSGXRTracker final : public IOpenXRExtensionPlugin,
public IHandTracker public IHandTracker
{ {
public: public:
static bool GetMotionControllerData(UObject* WorldContextObject, EControllerHand Hand, UE_DEPRECATED(5.5, "Deprecated by UE 5.5 in favor of GetMotionControllerState and GetHandTrackingState")
static bool GetMotionControllerData(
UObject* WorldContextObject,
EControllerHand Hand,
FXRMotionControllerData& OutMotionControllerData); FXRMotionControllerData& OutMotionControllerData);
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
static bool GetMotionControllerState(
UObject* WorldContextObject,
EXRSpaceType XRSpaceType, EControllerHand Hand, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
static bool GetHandTrackingState(
UObject* WorldContextObject,
EXRSpaceType XRSpaceType, EControllerHand Hand,
FXRHandTrackingState& OutHandTrackingState);
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
private: private:
struct FImpl; struct FImpl;
@@ -158,7 +172,15 @@ public:
virtual bool GetKeypointState( virtual bool GetKeypointState(
EControllerHand Hand, EControllerHand Hand,
EHandKeypoint Keypoint, FTransform& OutTransform, float& OutRadius) const override; EHandKeypoint Keypoint, FTransform& OutTransform, float& OutRadius) const override;
#if ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5
virtual bool GetAllKeypointStates(
EControllerHand Hand,
TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii,
bool& OutIsTracked) const override;
#else /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
virtual bool GetAllKeypointStates( virtual bool GetAllKeypointStates(
EControllerHand Hand, EControllerHand Hand,
TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii) const override; TArray<FVector>& OutPositions, TArray<FQuat>& OutRotations, TArray<float>& OutRadii) const override;
#endif /* ENGINE_MAJOR_VERSION >= 5 && ENGINE_MINOR_VERSION >= 5 */
}; };
@@ -42,5 +42,24 @@
bool USGXRTrackerKismetLibrary::GetMotionControllerData( bool USGXRTrackerKismetLibrary::GetMotionControllerData(
UObject* WorldContextObject, const EControllerHand Hand, FXRMotionControllerData& OutMotionControllerData) UObject* WorldContextObject, const EControllerHand Hand, FXRMotionControllerData& OutMotionControllerData)
{ {
PRAGMA_DISABLE_DEPRECATION_WARNINGS
return FSGXRTracker::GetMotionControllerData(WorldContextObject, Hand, OutMotionControllerData); return FSGXRTracker::GetMotionControllerData(WorldContextObject, Hand, OutMotionControllerData);
PRAGMA_ENABLE_DEPRECATION_WARNINGS
}
bool USGXRTrackerKismetLibrary::GetMotionControllerState(
UObject* WorldContextObject,
const EXRSpaceType XRSpaceType, const EControllerHand Hand, const EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState)
{
return FSGXRTracker::GetMotionControllerState(
WorldContextObject, XRSpaceType, Hand, XRControllerPoseType, OutMotionControllerState);
}
bool USGXRTrackerKismetLibrary::GetHandTrackingState(
UObject* WorldContextObject,
const EXRSpaceType XRSpaceType, const EControllerHand Hand,
FXRHandTrackingState& OutHandTrackingState)
{
return FSGXRTracker::GetHandTrackingState(WorldContextObject, XRSpaceType, Hand, OutHandTrackingState);
} }
@@ -53,6 +53,22 @@ class SENSEGLOVETRACKINGKISMET_API USGXRTrackerKismetLibrary final : public USGB
public: public:
UFUNCTION(BlueprintCallable, Category="SenseGlove | Tracking | XR Tracker", UFUNCTION(BlueprintCallable, Category="SenseGlove | Tracking | XR Tracker",
meta=(WorldContext="WorldContextObject")) meta=(WorldContext="WorldContextObject"))
static bool GetMotionControllerData(UObject* WorldContextObject, EControllerHand Hand, static bool GetMotionControllerData(
UObject* WorldContextObject,
EControllerHand Hand,
FXRMotionControllerData& OutMotionControllerData); FXRMotionControllerData& OutMotionControllerData);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Tracking | XR Tracker",
meta=(WorldContext="WorldContextObject"))
static bool GetMotionControllerState(
UObject* WorldContextObject,
EXRSpaceType XRSpaceType, EControllerHand Hand, EXRControllerPoseType XRControllerPoseType,
FXRMotionControllerState& OutMotionControllerState);
UFUNCTION(BlueprintCallable, Category="SenseGlove | Tracking | XR Tracker",
meta=(WorldContext="WorldContextObject"))
static bool GetHandTrackingState(
UObject* WorldContextObject,
EXRSpaceType XRSpaceType, EControllerHand Hand,
FXRHandTrackingState& OutHandTrackingState);
}; };