From 31802a35f22ff1937295c94d148514cdcf85793a Mon Sep 17 00:00:00 2001 From: Mamadou Babaei Date: Sun, 21 Jul 2024 18:44:11 +0200 Subject: [PATCH] complete advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint tutorial --- .../README.md | 10 +++---- .../blueprint.md | 30 ++++++++++++++++++- ...ollerdata-animated-debug-virtual-hands.gif | 3 ++ ...rdata-blueprint-add-draw-hand-function.png | 3 ++ ...nt-add-get-motion-controller-data-node.png | 3 ++ ...eprint-break-xr-motion-controller-data.png | 3 ++ ...eprint-debug-virtual-hand-draw-options.png | 3 ++ ...a-blueprint-draw-debug-gizmo-per-joint.png | 3 ++ ...rint-draw-hand-function-implementation.png | 3 ++ ...-blueprint-get-joint-position-rotation.png | 3 ++ ...ontrollerdata-blueprint-representation.png | 3 ++ ...ioncontrollerdata-blueprint-validation.png | 3 ++ ...-vrpawn-draw-virtual-hands-every-frame.png | 3 ++ ...bling-senseglove-unrealp-engine-plugin.png | 3 ++ ...-blueprint-get-joint-position-rotation.png | 3 -- ...ontrollerdata-blueprint-representation.png | 3 -- 16 files changed, 70 insertions(+), 12 deletions(-) create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-representation.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-validation.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png create mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/enabling-senseglove-unrealp-engine-plugin.png delete mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png delete mode 100644 Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-representation.png diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/README.md b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/README.md index 17d0a8f3..aa2db720 100644 --- a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/README.md +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/README.md @@ -61,9 +61,9 @@ struct FXRMotionControllerData Which on the Blueprint side it looks like this: -![FXRMotionControllerData Blueprint representation](fxrmotioncontrollerdata-blueprint-representation.png "FXRMotionControllerData Blueprint representation") +![FXRMotionControllerData Blueprint representation](consuming-fxrmotioncontrollerdata-blueprint-representation.png "FXRMotionControllerData Blueprint representation") -But, fear not, we got you covered! +But, fear not, we've got you covered! ## FXRMotionControllerData in Unreal Engine @@ -208,7 +208,7 @@ So, getting the any joint's position or rotation is as easy as casting the enum { } - // Ensure that MotionControllerData.HandKeyPositions has the location data + // Ensure that MotionControllerData.HandKeyPositions has the position data // for 26 joints! if (!ensureAlwaysMsgf(MotionControllerData.HandKeyPositions.Num() == EHandKeypointCount, @@ -217,7 +217,7 @@ So, getting the any joint's position or rotation is as easy as casting the enum return; } - // Ensure that MotionControllerData.HandKeyPositions has the location data + // Ensure that MotionControllerData.HandKeyPositions has the rotation data // for 26 joints! if (!ensureAlwaysMsgf(MotionControllerData.HandKeyRotations.Num() == EHandKeypointCount, @@ -238,6 +238,6 @@ So, getting the any joint's position or rotation is as easy as casting the enum The equivalent Blueprint code for the above looks something like this: -![Get a joint position and rotation from FXRMotionControllerData in Blueprint](fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png "Get a joint position and rotation from FXRMotionControllerData in Blueprint") +![Get a joint position and rotation from FXRMotionControllerData in Blueprint](consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png "Get a joint position and rotation from FXRMotionControllerData in Blueprint") OK, now that we've got a glimpse of how the virtual hand's joint data could be processed we are going to draw and animate a virtual hand in both [Blueprint](blueprint.md) and [C++](cpp.md) in the upcoming sections. diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint.md b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint.md index 262d5d83..d7bd87b9 100644 --- a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint.md +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/blueprint.md @@ -6,7 +6,9 @@ Before continuing this section, please ensure you've studied the [Consuming FXRM 1. Create a new Virtual Reality project based [the Unreal VR Template](https://dev.epicgames.com/documentation/en-us/unreal-engine/vr-template-in-unreal-engine). -2. Make sure the [SenseGlove UnrealEngine is installed and enabled](/getting-started/installation.md) inside your new project. +2. Make sure the [SenseGlove UnrealEngine plugin is installed and enabled](/getting-started/installation.md) inside your new project. + +![Enabling the SenseGlove Unreal Engine Plugin](enabling-senseglove-unrealp-engine-plugin.png "Enabling the SenseGlove Unreal Engine Plugin") 3. You could use either hand-tracking or a SenseGlove device as the input data, or both of the inside the same project. Whether you would like to use hand-tracking or a SenseGlove device, please make sure the required steps are taken for each of those first. @@ -14,10 +16,36 @@ Before continuing this section, please ensure you've studied the [Consuming FXRM 5. Add a new function named Draw Hand with an input parameter of type EController Hand named Hand. +![Adding the Blueprint Draw Hand function](consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png "Adding the Blueprint Draw Hand function") + 6. Inside this function's event graph add a Get Motion Controller Data node from SenseGlove > Tracking > XR Tracker > Get Motion Controller Data. +![Adding the Get Motion Controller Data node](consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png "Adding the Get Motion Controller Data node") + 7. Then connect the functions Hand input parameter to the Get Motion Controller Data's Hand input and right-click on the OutMotionControllerData parameter and use the Break XRMotionControllerData node to break the struct to it's fields. +![Breaking the XR Motion Controller Data node](consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png "Breaking the XR Motion Controller Data node") +8. After this, we need to perform data validation by checking the return status of the Get Motion Controller Data function and FXRMotionControllerData's Valid field. Then, we check if the motion controller device is being tracked and indeed coming from a hand-tracking source. And, finally, we check whether we have the positions and rotations for exactly 26 joints or not. +![FXRMotionControllerData validation](consuming-fxrmotioncontrollerdata-blueprint-validation.png "FXRMotionControllerData validation") +9. OK, now it's time to draw the joints! If we check out the SenseGlove Debug module's draw option, we notice there are various ways to draw the debug virtual hand. Drawing a cube or a gizmo per joint, or draw the whole hand all at once by passing the retrieved FXRMotionControllerData to the DebugVirtualHand::Draw function! But, since the point of this tutorial is to learn how to consume the FXRMotionControllerData we ignore the last option. Between the debug cubes or gizmos, we are going to choose the gizmos since they better represent the rotations than the cubes. + +![Some options for drawing a debug virtual hand](consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png "Some options for drawing a debug virtual hand") + +10. In the last step inside the Draw Hand function, in order to draw a virtual hand with 26 joints, we have to first iterate through either of the Hand Key Positions or Hand Key Rotations arrays from the FXRMotionControllerData struct. Since we made sure both arrays have 26 elements before we reached this step, it's safe to just iterate over one and use the Array Index inside a For Each Loop or a For Loop to access the position and rotation of every joint. Then we use each array Get (a ref) method to access the position and rotation data inside the loop and call the Draw function from SenseGlove > Debug > Gizmo per every joint. Please note that there are two Draw functions and the only difference between the two is that one accepts an FQuat and the other a FRotator for its Rotation input parameter. In this case, we use the FQuat variant to avoid an extra conversion to FRotator. Also, please adjust the Thickness option for the Settings parameter from 1.0 to 0.2, as the default value might be too thick for drawing a joint gizmo. + +![Drawing a debug gizmo per each hand's joint](consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png "Drawing a debug gizmo per each hand's joint") + +11. Well, now the full implementation for the Draw Hand function insde the VRPawn should look something like this: + +![VRPawn Draw Hand function final implementation](consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png "VRPawn Draw Hand function final implementation") + +12. Finally, go back to VRPawn's event graph and the following code to the Tick event. Basically what we do here is call our newly implemented Draw Hand twice, once for each hand. + +![Attempt to draw the left and right virtual hands every frame from the VR Pawn Tick event](consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png "Attempt to draw the left and right virtual hands every frame from the VR Pawn Tick event") + +13. Now, go back to the VRTemplateMap and use the VR Preview button to run the game. If everything's done correctly, you should be able to see the virtual hands inside your VR simulation. + +![FXRMotionControllerData animated debug virtual hands](consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif "FXRMotionControllerData animated debug virtual hands") \ No newline at end of file diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif new file mode 100644 index 00000000..be67d688 --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-animated-debug-virtual-hands.gif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6c0f1fbdf9521bbdead363c9ba98987dd0a63a56f3f547b4f8863172fcbb8e19 +size 13726558 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png new file mode 100644 index 00000000..9aefa95e --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-draw-hand-function.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:328a336cedd8b15abbe1e9b067fd7d063b6b7d38ab042b57b39ae0cdca4fed51 +size 376083 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png new file mode 100644 index 00000000..c4b22fee --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-add-get-motion-controller-data-node.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:141c1530a7d685777ac4c199f9939f6a4077cef7f69c198779bea38c75eae54c +size 121048 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png new file mode 100644 index 00000000..28af692c --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-break-xr-motion-controller-data.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:712930fb7858e31a4d35707de545f359b05432f4d9c3b5b27ac24f3353c6c0ac +size 113995 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png new file mode 100644 index 00000000..bddb99bb --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-debug-virtual-hand-draw-options.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6be29be07f0c88334d51ab4ebe428e154bc82350e5a01209569a0e33e8e00dd +size 25411 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png new file mode 100644 index 00000000..46b09c0b --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-debug-gizmo-per-joint.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cfc41b6232d056dd5d94b96f32c59fbd7c8194d783adb5319aeca0044b1e15e +size 363903 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png new file mode 100644 index 00000000..5249e5a9 --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-draw-hand-function-implementation.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b0c01f481179dedaad17724f074fa83f6fce010affd2533600231f602a97008d +size 521212 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png new file mode 100644 index 00000000..6fc28d1e --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:741fd7256ab9f9fd45db535a58fb0248fb0621f23917fb2418333f9595bd8394 +size 372361 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-representation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-representation.png new file mode 100644 index 00000000..1cad022d --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-representation.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f028afcb4e7b021700b9dae7c480ff71ec645a5e24512b4b223dd818c1c3d3d7 +size 54379 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-validation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-validation.png new file mode 100644 index 00000000..2867b079 --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-validation.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ebd49fd6327f7f1151ee32642ab27d9c7f731717917ac7163d2ee32567a029d5 +size 299424 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png new file mode 100644 index 00000000..856f1219 --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/consuming-fxrmotioncontrollerdata-blueprint-vrpawn-draw-virtual-hands-every-frame.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5abcc48443d71288701c6a3137881e969f269e35e54f9ca3b2e9bcded090afbe +size 259642 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/enabling-senseglove-unrealp-engine-plugin.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/enabling-senseglove-unrealp-engine-plugin.png new file mode 100644 index 00000000..8bc71345 --- /dev/null +++ b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/enabling-senseglove-unrealp-engine-plugin.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4bc8ff35df13ef30212e5cd69c23556f429041bb141bce64cbce9366c27f8071 +size 48777 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png deleted file mode 100644 index c81dced6..00000000 --- a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-get-joint-position-rotation.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:8935396aca2be365d9a3d77557b2198b347911f913caa7b98fea38bd939420db -size 387644 diff --git a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-representation.png b/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-representation.png deleted file mode 100644 index d62d51ec..00000000 --- a/Handbook/src/advanced-topics/openxr/consuming-fxrmotioncontrollerdata/fxrmotioncontrollerdata-blueprint-representation.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:5b05ea7a9717644ff59c839060206e734c1a50dc8b451cc654bd40443e3f4f49 -size 136064