complete advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint tutorial

This commit is contained in:
Mamadou Babaei
2024-08-16 18:01:44 +02:00
parent 50f2278a87
commit 7bd4f85d61
16 changed files with 70 additions and 12 deletions
@@ -61,9 +61,9 @@ struct FXRMotionControllerData
Which on the Blueprint side it looks like this:
![FXRMotionControllerData Blueprint representation](fxrmotioncontrollerdata-blueprint-representation.png "FXRMotionControllerData Blueprint representation")
![FXRMotionControllerData Blueprint representation](consuming-fxrmotioncontrollerdata-blueprint-representation.png "FXRMotionControllerData Blueprint representation")
But, fear not, we got you covered!
But, fear not, we've got you covered!
## FXRMotionControllerData in Unreal Engine
@@ -208,7 +208,7 @@ So, getting the any joint's position or rotation is as easy as casting the enum
{
}
// Ensure that MotionControllerData.HandKeyPositions has the location data
// Ensure that MotionControllerData.HandKeyPositions has the position data
// for 26 joints!
if (!ensureAlwaysMsgf(MotionControllerData.HandKeyPositions.Num()
== EHandKeypointCount,
@@ -217,7 +217,7 @@ So, getting the any joint's position or rotation is as easy as casting the enum
return;
}
// Ensure that MotionControllerData.HandKeyPositions has the location data
// Ensure that MotionControllerData.HandKeyPositions has the rotation data
// for 26 joints!
if (!ensureAlwaysMsgf(MotionControllerData.HandKeyRotations.Num()
== EHandKeypointCount,
@@ -238,6 +238,6 @@ So, getting the any joint's position or rotation is as easy as casting the enum
The equivalent Blueprint code for the above looks something like this:
![Get a joint position and rotation from FXRMotionControllerData in Blueprint](fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png "Get a joint position and rotation from FXRMotionControllerData in Blueprint")
![Get a joint position and rotation from FXRMotionControllerData in Blueprint](consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png "Get a joint position and rotation from FXRMotionControllerData in Blueprint")
OK, now that we've got a glimpse of how the virtual hand's joint data could be processed we are going to draw and animate a virtual hand in both [Blueprint](blueprint.md) and [C++](cpp.md) in the upcoming sections.
@@ -6,7 +6,9 @@ Before continuing this section, please ensure you've studied the [Consuming FXRM
1. Create a new Virtual Reality project based [the Unreal VR Template](https://dev.epicgames.com/documentation/en-us/unreal-engine/vr-template-in-unreal-engine).
2. Make sure the [SenseGlove UnrealEngine is installed and enabled](/getting-started/installation.md) inside your new project.
2. Make sure the [SenseGlove UnrealEngine plugin is installed and enabled](/getting-started/installation.md) inside your new project.
![Enabling the SenseGlove Unreal Engine Plugin](enabling-senseglove-unrealp-engine-plugin.png "Enabling the SenseGlove Unreal Engine Plugin")
3. You could use either hand-tracking or a SenseGlove device as the input data, or both of the inside the same project. Whether you would like to use hand-tracking or a SenseGlove device, please make sure the required steps are taken for each of those first.
@@ -14,10 +16,36 @@ Before continuing this section, please ensure you've studied the [Consuming FXRM
5. Add a new function named <code>Draw Hand</code> with an input parameter of type <code>EController Hand</code> named <code>Hand</code>.
![Adding the Blueprint Draw Hand function](consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png "Adding the Blueprint Draw Hand function")
6. Inside this function's event graph add a <code>Get Motion Controller Data</code> node from <code>SenseGlove > Tracking > XR Tracker > Get Motion Controller Data</code>.
![Adding the Get Motion Controller Data node](consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png "Adding the Get Motion Controller Data node")
7. Then connect the functions <code>Hand</code> input parameter to the <code>Get Motion Controller Data</code>'s <code>Hand</code> input and right-click on the <code>OutMotionControllerData</code> parameter and use the <code>Break XRMotionControllerData</code> node to break the struct to it's fields.
![Breaking the XR Motion Controller Data node](consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png "Breaking the XR Motion Controller Data node")
8. After this, we need to perform data validation by checking the return status of the <code>Get Motion Controller Data</code> function and <code>FXRMotionControllerData</code>'s <code>Valid</code> field. Then, we check if the motion controller device is being tracked and indeed coming from a hand-tracking source. And, finally, we check whether we have the positions and rotations for exactly <code>26</code> joints or not.
![FXRMotionControllerData validation](consuming-fxrmotioncontrollerdata-blueprint-validation.png "FXRMotionControllerData validation")
9. OK, now it's time to draw the joints! If we check out the SenseGlove Debug module's draw option, we notice there are various ways to draw the debug virtual hand. Drawing a cube or a gizmo per joint, or draw the whole hand all at once by passing the retrieved <code>FXRMotionControllerData</code> to the <code>DebugVirtualHand::Draw</code> function! But, since the point of this tutorial is to learn how to consume the <code>FXRMotionControllerData</code> we ignore the last option. Between the debug cubes or gizmos, we are going to choose the gizmos since they better represent the rotations than the cubes.
![Some options for drawing a debug virtual hand](consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png "Some options for drawing a debug virtual hand")
10. In the last step inside the <code>Draw Hand</code> function, in order to draw a virtual hand with <code>26</code> joints, we have to first iterate through either of the <code>Hand Key Positions</code> or <code>Hand Key Rotations</code> arrays from the <code>FXRMotionControllerData</code> struct. Since we made sure both arrays have <code>26</code> elements before we reached this step, it's safe to just iterate over one and use the <code>Array Index</code> inside a <code>For Each Loop</code> or a <code>For Loop</code> to access the position and rotation of every joint. Then we use each array <code>Get (a ref)</code> method to access the position and rotation data inside the loop and call the <code>Draw</code> function from <code>SenseGlove > Debug > Gizmo</code> per every joint. Please note that there are two <code>Draw</code> functions and the only difference between the two is that one accepts an <code>FQuat</code> and the other a <code>FRotator</code> for its <code>Rotation</code> input parameter. In this case, we use the <code>FQuat</code> variant to avoid an extra conversion to <code>FRotator</code>. Also, please adjust the <code>Thickness</code> option for the <code>Settings</code> parameter from <code>1.0</code> to <code>0.2</code>, as the default value might be too thick for drawing a joint gizmo.
![Drawing a debug gizmo per each hand's joint](consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png "Drawing a debug gizmo per each hand's joint")
11. Well, now the full implementation for the <code>Draw Hand</code> function insde the <code>VRPawn</code> should look something like this:
![VRPawn Draw Hand function final implementation](consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png "VRPawn Draw Hand function final implementation")
12. Finally, go back to <code>VRPawn</code>'s event graph and the following code to the <code>Tick</code> event. Basically what we do here is call our newly implemented <code>Draw Hand</code> twice, once for each hand.
![Attempt to draw the left and right virtual hands every frame from the VR Pawn Tick event](consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png "Attempt to draw the left and right virtual hands every frame from the VR Pawn Tick event")
13. Now, go back to the <code>VRTemplateMap</code> and use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation.
![FXRMotionControllerData animated debug virtual hands](consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif "FXRMotionControllerData animated debug virtual hands")