replace inline code blocks with back tilde
This commit is contained in:
Mamadou Babaei
@@ -1,9 +1,9 @@
|
||||
# OpenXR
|
||||
|
||||
The SenseGlove Unreal Engine Plugin has provided OpenXR-compatible hand tracking by implementing <code>XR_EXT_hand_tracking</code> since <code>v2.1.0</code>.
|
||||
The SenseGlove Unreal Engine Plugin has provided OpenXR-compatible hand tracking by implementing `XR_EXT_hand_tracking` since `v2.1.0`.
|
||||
|
||||
Typically a user does not need to know anything about OpenXR to use the plugin, so this section of the handbook is for advanced users who are looking for a way to directly consume the OpenXR data coming directly from either a SenseGlove device or if enabled in the plugin settings from hand-tracking.
|
||||
|
||||
Since the SenseGlove Unreal Engine Plugin registers itself as an <code>OpenXRHandTracking</code> motion controller device it becomes a hand-tracking provider for Unreal Engine, thus the OpenXR data from SenseGlove could always be retrieved from the Unreal Engine's <code>IXTrackingSystem</code> with one caveat. The caveat is if another OpenXR-compatible hand-tracking plugin, e.g. Epic's own OpenXRHandTracking, is enabled simultaneously it's not guaranteed that the <code>FXRMotionControllerData</code> retrieved from the <code>IXTrackingSystem::GetMotionControllerData()</code> method is coming from SenseGlove, as this method returns the first hand-tracking plugin it could find. Thus, SenseGlove provides its own implementation of <codE>GetMotionControllerData()</code> which guarantees the retrieved <code>FXRMotionControllerData</code> is coming from the SenseGlove Unreal Engine Plugin; and this is the preferred way to that.
|
||||
Since the SenseGlove Unreal Engine Plugin registers itself as an `OpenXRHandTracking` motion controller device it becomes a hand-tracking provider for Unreal Engine, thus the OpenXR data from SenseGlove could always be retrieved from the Unreal Engine's `IXTrackingSystem` with one caveat. The caveat is if another OpenXR-compatible hand-tracking plugin, e.g. Epic's own OpenXRHandTracking, is enabled simultaneously it's not guaranteed that the `FXRMotionControllerData` retrieved from the `IXTrackingSystem::GetMotionControllerData()` method is coming from SenseGlove, as this method returns the first hand-tracking plugin it could find. Thus, SenseGlove provides its own implementation of `GetMotionControllerData()` which guarantees the retrieved `FXRMotionControllerData` is coming from the SenseGlove Unreal Engine Plugin; and this is the preferred way to that.
|
||||
|
||||
In the next sections we'll see [how we can directly consume the <code>FXRMotionControllerData</code>](consuming-fxrmotioncontrollerdata/README.md) to draw and animate debug virtual hands in both [Blueprint](consuming-fxrmotioncontrollerdata/blueprint.md) and [C++](consuming-fxrmotioncontrollerdata/cpp.md).
|
||||
In the next sections we'll see [how we can directly consume the `FXRMotionControllerData`](consuming-fxrmotioncontrollerdata/README.md) to draw and animate debug virtual hands in both [Blueprint](consuming-fxrmotioncontrollerdata/blueprint.md) and [C++](consuming-fxrmotioncontrollerdata/cpp.md).
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Consuming FXRMotionControllerData
|
||||
|
||||
Taking a closer look at the <code>FXRMotionControllerData</code> declaration inside the Unreal Engine's <code>HeadMountedDisplay</code> module at <code>[Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h)</code>, figuring out the data structure might not seem very straightforward:
|
||||
Taking a closer look at the `FXRMotionControllerData` declaration inside the Unreal Engine's `HeadMountedDisplay` module at `[Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h)`, figuring out the data structure might not seem very straightforward:
|
||||
|
||||
```cpp
|
||||
USTRUCT(BlueprintType)
|
||||
@@ -132,15 +132,15 @@ But, fear not, we've got you covered!
|
||||
|
||||
### Organization of FXRMotionControllerData
|
||||
|
||||
The structure is organized to encapsulate all relevant data needed for motion controller tracking in a coherent and accessible manner. Boolean flags (`bValid` and `bIsGrasped`) provide quick checks on the state of the controller data. Identifiers (`DeviceName` and `ApplicationInstanceID`) ensure the correct association of data. Positional and rotational data (`GripPosition`, `GripRotation`, `AimPosition`, and `AimRotation`) offer precise tracking of the controller's movement. Arrays (`HandKeyPositions`, `HandKeyRotations`, and `HandKeyRadii`) allow detailed hand tracking, which is critical for immersive VR experiences. Lastly, the tracking status (`TrackingStatus`) informs the system of the reliability of the data being processed and whether the motion controller is actively being tracked or it's inactive at the moment.
|
||||
The structure is organized to encapsulate all relevant data needed for motion controller tracking in a coherent and accessible manner. Boolean flags `bValid` and `bIsGrasped` provide quick checks on the state of the controller data. Identifiers `DeviceName` and `ApplicationInstanceID` ensure the correct association of data. Positional and rotational data `GripPosition`, `GripRotation`, `AimPosition`, and `AimRotation` offer precise tracking of the controller's movement. Arrays `HandKeyPositions`, `HandKeyRotations`, and `HandKeyRadii` allow detailed hand tracking, which is critical for immersive VR experiences. Lastly, the tracking status `TrackingStatus` informs the system of the reliability of the data being processed and whether the motion controller is actively being tracked or it's inactive at the moment.
|
||||
|
||||
### Processing the Data for Drawing and Animating a Virtual Hand
|
||||
|
||||
In order to draw and animate a virtual hand in real-time whether the data is coming from hand-tracking or a SenseGlove device, we could consume the data from the <code>HandKeyPositions</code> and <code>HandKeyRotations</code> fields of the <code>FXRMotionControllerData</code> struct.
|
||||
In order to draw and animate a virtual hand in real-time whether the data is coming from hand-tracking or a SenseGlove device, we could consume the data from the `HandKeyPositions` and `HandKeyRotations` fields of the `FXRMotionControllerData` struct.
|
||||
|
||||
Both <code>HandKeyPositions</code> and <code>HandKeyRotations</code> contain 26 elements as defined by OpenXR's [<code>XR_HAND_JOINT_COUNT_EXT</code>](https://registry.khronos.org/OpenXR/specs/1.1/man/html/XR_HAND_JOINT_COUNT_EXT.html) and [<code>XrHandJointLocationsEXT</code>](https://registry.khronos.org/OpenXR/specs/1.0/man/html/XrHandJointLocationsEXT.html), etc.
|
||||
Both `HandKeyPositions` and `HandKeyRotations` contain 26 elements as defined by OpenXR's [`XR_HAND_JOINT_COUNT_EXT`](https://registry.khronos.org/OpenXR/specs/1.1/man/html/XR_HAND_JOINT_COUNT_EXT.html) and [`XrHandJointLocationsEXT`](https://registry.khronos.org/OpenXR/specs/1.0/man/html/XrHandJointLocationsEXT.html), etc.
|
||||
|
||||
Unreal Engine also provides an enum called <code>EHandKeypoint</code> naming the 26 joints, and the equivalent of <code>XR_HAND_JOINT_COUNT_EXT</code> as <code>EHandKeypointCount</code> inside <code>[Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h)</code> as follows:
|
||||
Unreal Engine also provides an enum called `EHandKeypoint` naming the 26 joints, and the equivalent of `XR_HAND_JOINT_COUNT_EXT` as `EHandKeypointCount` inside `[Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h](https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Source/Runtime/HeadMountedDisplay/Public/HeadMountedDisplayTypes.h)` as follows:
|
||||
|
||||
```cpp
|
||||
/**
|
||||
@@ -187,7 +187,7 @@ So, getting the any joint's position or rotation is as easy as casting the enum
|
||||
```cpp
|
||||
FXRMotionControllerData MotionControllerData;
|
||||
const bool bGotMotionControllerData = FSGXRTracker::GetMotionControllerData(
|
||||
this, EControllerHand::Left, MotionControllerData);
|
||||
GetWorld(), EControllerHand::Left, MotionControllerData);
|
||||
|
||||
// Return if the struct data is invalid!
|
||||
if (!bGotMotionControllerData || !MotionControllerData.bValid)
|
||||
|
||||
+10
-10
@@ -12,40 +12,40 @@ Before continuing this section, please ensure you've studied the [Consuming FXRM
|
||||
|
||||
3. You could use either hand-tracking or a SenseGlove device as the input data, or both of the inside the same project. Whether you would like to use hand-tracking or a SenseGlove device, please make sure the required steps are taken for each of those first.
|
||||
|
||||
4. You could add the required Blueprint code for drawing virtual hands to either your Level Buleprint or the VRPawn Blueprint Class located at <code>/Content/VRTemplate/Blueprints/VRPawn</code>. In this guide we are going to add the code to our VRPawn.
|
||||
4. You could add the required Blueprint code for drawing virtual hands to either your Level Buleprint or the VRPawn Blueprint Class located at `/Content/VRTemplate/Blueprints/VRPawn`. In this guide we are going to add the code to our VRPawn.
|
||||
|
||||
5. Add a new function named <code>Draw Hand</code> with an input parameter of type <code>EController Hand</code> named <code>Hand</code>.
|
||||
5. Add a new function named `Draw Hand` with an input parameter of type `EController Hand` named `Hand`.
|
||||
|
||||
|
||||
|
||||
6. Inside this function's event graph add a <code>Get Motion Controller Data</code> node from <code>SenseGlove > Tracking > XR Tracker > Get Motion Controller Data</code>.
|
||||
6. Inside this function's event graph add a `Get Motion Controller Data` node from `SenseGlove > Tracking > XR Tracker > Get Motion Controller Data`.
|
||||
|
||||
|
||||
|
||||
7. Then connect the functions <code>Hand</code> input parameter to the <code>Get Motion Controller Data</code>'s <code>Hand</code> input and right-click on the <code>OutMotionControllerData</code> parameter and use the <code>Break XRMotionControllerData</code> node to break the struct to it's fields.
|
||||
7. Then connect the functions `Hand` input parameter to the `Get Motion Controller Data`'s `Hand` input and right-click on the `OutMotionControllerData` parameter and use the `Break XRMotionControllerData` node to break the struct to it's fields.
|
||||
|
||||
|
||||
|
||||
8. After this, we need to perform data validation by checking the return status of the <code>Get Motion Controller Data</code> function and <code>FXRMotionControllerData</code>'s <code>Valid</code> field. Then, we check if the motion controller device is being tracked and indeed coming from a hand-tracking source. And, finally, we check whether we have the positions and rotations for exactly <code>26</code> joints or not.
|
||||
8. After this, we need to perform data validation by checking the return status of the `Get Motion Controller Data` function and `FXRMotionControllerData`'s `Valid` field. Then, we check if the motion controller device is being tracked and indeed coming from a hand-tracking source. And, finally, we check whether we have the positions and rotations for exactly `26` joints or not.
|
||||
|
||||
|
||||
|
||||
9. OK, now it's time to draw the joints! If we check out the SenseGlove Debug module's draw option, we notice there are various ways to draw the debug virtual hand. Drawing a cube or a gizmo per joint, or draw the whole hand all at once by passing the retrieved <code>FXRMotionControllerData</code> to the <code>DebugVirtualHand::Draw</code> function! But, since the point of this tutorial is to learn how to consume the <code>FXRMotionControllerData</code> we ignore the last option. Between the debug cubes or gizmos, we are going to choose the gizmos since they better represent the rotations than the cubes.
|
||||
9. OK, now it's time to draw the joints! If we check out the SenseGlove Debug module's draw option, we notice there are various ways to draw the debug virtual hand. Drawing a cube or a gizmo per joint, or draw the whole hand all at once by passing the retrieved `FXRMotionControllerData` to the `DebugVirtualHand::Draw` function! But, since the point of this tutorial is to learn how to consume the `FXRMotionControllerData` we ignore the last option. Between the debug cubes or gizmos, we are going to choose the gizmos since they better represent the rotations than the cubes.
|
||||
|
||||
|
||||
|
||||
10. In the last step inside the <code>Draw Hand</code> function, in order to draw a virtual hand with <code>26</code> joints, we have to first iterate through either of the <code>Hand Key Positions</code> or <code>Hand Key Rotations</code> arrays from the <code>FXRMotionControllerData</code> struct. Since we made sure both arrays have <code>26</code> elements before we reached this step, it's safe to just iterate over one and use the <code>Array Index</code> inside a <code>For Each Loop</code> or a <code>For Loop</code> to access the position and rotation of every joint. Then we use each array <code>Get (a ref)</code> method to access the position and rotation data inside the loop and call the <code>Draw</code> function from <code>SenseGlove > Debug > Gizmo</code> per every joint. Please note that there are two <code>Draw</code> functions and the only difference between the two is that one accepts an <code>FQuat</code> and the other a <code>FRotator</code> for its <code>Rotation</code> input parameter. In this case, we use the <code>FQuat</code> variant to avoid an extra conversion to <code>FRotator</code>. Also, please adjust the <code>Thickness</code> option for the <code>Settings</code> parameter from <code>1.0</code> to <code>0.2</code>, as the default value might be too thick for drawing a joint gizmo.
|
||||
10. In the last step inside the `Draw Hand` function, in order to draw a virtual hand with `26` joints, we have to first iterate through either of the `Hand Key Positions` or `Hand Key Rotations` arrays from the `FXRMotionControllerData` struct. Since we made sure both arrays have `26` elements before we reached this step, it's safe to just iterate over one and use the `Array Index` inside a `For Each Loop` or a `For Loop` to access the position and rotation of every joint. Then we use each array `Get (a ref)` method to access the position and rotation data inside the loop and call the `Draw` function from `SenseGlove > Debug > Gizmo` per every joint. Please note that there are two `Draw` functions and the only difference between the two is that one accepts an `FQuat` and the other a `FRotator` for its `Rotation` input parameter. In this case, we use the `FQuat` variant to avoid an extra conversion to `FRotator`. Also, please adjust the `Thickness` option for the `Settings` parameter from `1.0` to `0.2`, as the default value might be too thick for drawing a joint gizmo.
|
||||
|
||||
|
||||
|
||||
11. Well, now the full implementation for the <code>Draw Hand</code> function insde the <code>VRPawn</code> should look something like this:
|
||||
11. Well, now the full implementation for the `Draw Hand` function insde the `VRPawn` should look something like this:
|
||||
|
||||
|
||||
|
||||
12. Finally, go back to <code>VRPawn</code>'s event graph and the following code to the <code>Tick</code> event. Basically what we do here is call our newly implemented <code>Draw Hand</code> twice, once for each hand.
|
||||
12. Finally, go back to `VRPawn`'s event graph and the following code to the `Tick` event. Basically what we do here is call our newly implemented `Draw Hand` twice, once for each hand.
|
||||
|
||||
|
||||
|
||||
13. Now, go back to the <code>VRTemplateMap</code> and use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation.
|
||||
13. Now, go back to the `VRTemplateMap` and use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation.
|
||||
|
||||
|
||||
@@ -12,33 +12,33 @@ Before continuing this section, please ensure you've first studied the [Consumin
|
||||
|
||||
3. You could use either hand-tracking or a SenseGlove device as the input data, or both of the inside the same project. Whether you would like to use hand-tracking or a SenseGlove device, please make sure the required steps are taken for each of those first.
|
||||
|
||||
4. From the <code>Tools</code> menu choose <code>New C++ class...</code>.
|
||||
4. From the `Tools` menu choose `New C++ class...`.
|
||||
|
||||
|
||||
|
||||
5. Choose the Unreal Engine's <code>APawn</code> class as the parent class for the new C++ pawn class.
|
||||
5. Choose the Unreal Engine's `APawn` class as the parent class for the new C++ pawn class.
|
||||
|
||||
|
||||
|
||||
6. Name the new pawn class <code>DebugPawn</code>.
|
||||
6. Name the new pawn class `DebugPawn`.
|
||||
|
||||
|
||||
|
||||
7. Since we have created a new C++ class, this converts the current Blueprint VRTemplateMap project to a C++ one. That's why the Unreal Editor will give us a few prompts regarding opening the project in the default IDE and rebuilding the code. It might be simpler to just close the editor, then rebuild the source code inside your favorite IDE, and then start the editor with the converted project again.
|
||||
|
||||
8. Find and open the VRPawn Blueprint Class located at <code>/Content/VRTemplate/Blueprints/VRPawn</code> inside the Blueprint Editor and from the <code>File</code> menu choose the <code>Reparent Blueprint</code> class.
|
||||
8. Find and open the VRPawn Blueprint Class located at `/Content/VRTemplate/Blueprints/VRPawn` inside the Blueprint Editor and from the `File` menu choose the `Reparent Blueprint` class.
|
||||
|
||||
|
||||
|
||||
9. In the new <code>Reparent blueprint</code> window choose <code>DebugPawn</code> as the new parent.
|
||||
9. In the new `Reparent blueprint` window choose `DebugPawn` as the new parent.
|
||||
|
||||
|
||||
|
||||
10. By looking at the <code>Parent Class</code> label located under the Blueprint Editor window control buttons verify that the <code>ADebugPawn</code> class has been set as the new parent.
|
||||
10. By looking at the `Parent Class` label located under the Blueprint Editor window control buttons verify that the `ADebugPawn` class has been set as the new parent.
|
||||
|
||||
|
||||
|
||||
11. Locate the project's main Build file, in our case <code>VirtualHandCpp/Source/VirtualHandCpp/VirtualHandCpp.Build.cs</code> and add the <code>InputDevice</code>, <code>OpenXRHMD</code>, <code>SenseGloveBuildHacks</code>, <code>SenseGloveDebug</code>, <code>SenseGloveSettings</code>, and <code>SenseGloveTracking</code> modules as either a private or public dependency.
|
||||
11. Locate the project's main Build file, in our case `VirtualHandCpp/Source/VirtualHandCpp/VirtualHandCpp.Build.cs` and add the `InputDevice`, `OpenXRHMD`, `SenseGloveBuildHacks`, `SenseGloveDebug`, `SenseGloveSettings`, and `SenseGloveTracking` modules as either a private or public dependency.
|
||||
|
||||
```csharp
|
||||
// Fill out your copyright notice in the Description page of Project Settings.
|
||||
@@ -74,9 +74,9 @@ public class VirtualHandCpp : ModuleRules
|
||||
}
|
||||
```
|
||||
|
||||
12. Locate the C++ header and source file for the <code>ADebugPawn</code> inside the project in your C++ IDE. In our case they are located at <code>VirtualHandCpp/Source/VirtualHandCpp/DebugPawn.h</code> and <code>VirtualHandCpp/Source/VirtualHandCpp/DebugPawn.cpp</code>.
|
||||
12. Locate the C++ header and source file for the `ADebugPawn` inside the project in your C++ IDE. In our case they are located at `VirtualHandCpp/Source/VirtualHandCpp/DebugPawn.h` and `VirtualHandCpp/Source/VirtualHandCpp/DebugPawn.cpp`.
|
||||
|
||||
13. Modify the <code>DebugPawn.h</code> header file to look like this:
|
||||
13. Modify the `DebugPawn.h` header file to look like this:
|
||||
|
||||
```cpp
|
||||
// Fill out your copyright notice in the Description page of Project Settings.
|
||||
@@ -121,7 +121,7 @@ private:
|
||||
};
|
||||
```
|
||||
|
||||
14. Modify the <code>DebugPawn.cpp</code> implementation file to look like this:
|
||||
14. Modify the `DebugPawn.cpp` implementation file to look like this:
|
||||
|
||||
|
||||
```cpp
|
||||
@@ -246,6 +246,6 @@ void ADebugPawn::DrawHand(const EControllerHand Hand) const
|
||||
}
|
||||
```
|
||||
|
||||
15. Now, rebuild the source code and go back to the <code>VRTemplateMap</code>, then use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation.
|
||||
15. Now, rebuild the source code and go back to the `VRTemplateMap`, then use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user