add the full final draft for docs section: SGPawn Events: The Puppeteer (Controller) / Puppet (Pawn) Architecture
This commit is contained in:
Mamadou Babaei
@@ -74,7 +74,7 @@ Welcom to the SenseGlove Unreal Engine Handbook!
|
||||
|
||||
- [Safe Glove Access in Blueprint](advanced-topics/safe-glove-access-blueprint/README.md)
|
||||
- [Roll Your Own Hand Manipulation System](advanced-topics/roll-your-own-hand-manipulation-system/README.md)
|
||||
- [SGPawn Events](advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events/README.md)
|
||||
- [SGPawn Events](advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events-puppeteer-puppet-architecture/README.md)
|
||||
- [SGHandTrackerComponent](advanced-topics/roll-your-own-hand-manipulation-system/sghandtrackercomponent/README.md)
|
||||
- [SGHapticsComponent](advanced-topics/roll-your-own-hand-manipulation-system/sghapticscomponent/README.md)
|
||||
- [OpenXR](advanced-topics/openxr/README.md)
|
||||
|
||||
@@ -41,7 +41,7 @@ As the SenseGlove Unreal Engine Plugin is fully OpenXR-compliant, it provides Op
|
||||
> need to figure out how to manually apply the appropriate offsets within your
|
||||
> chosen OpenXR hand interaction system.
|
||||
>
|
||||
> For example, the VRE plugin provide similar configurations in their plugin's
|
||||
> For example, the VRE plugin provides similar configurations in their plugin's
|
||||
> settings section. For more information refer to the
|
||||
> [Changing Wrist-Tracker Offsets](#changing-wrist-tracker-offsets) section
|
||||
> below.
|
||||
|
||||
@@ -1,3 +1,41 @@
|
||||
# Roll Your Own Customized Hand Manipulation and Interaction System
|
||||
|
||||
The default hand interaction system shipped with the **SenseGlove Unreal Engine Plugin** consists of various components, including `SGPawn`, `SGPlayerController`, `SGVirtualHandComponent`, `SGGrabComponent`, `SGTouchComponent`, and others. This system is very easy to get started with and is thoroughly documented throughout this handbook.
|
||||
|
||||
However, this simplicity comes at a cost: limited functionality. At SenseGlove, we prioritize usability and practicality. That said, developing a comprehensive hand interaction system that suits every possible use case is not an easy task. For example, projects such as the [VR Expansion Plugin (VRE)](https://vreue4.com/) — an Epic MegaGrants recipient — have been in development for over a decade, and development is still ongoing. Naturally, such depth also comes with a trade-off: a steep learning curve and reduced beginner-friendliness.
|
||||
|
||||
To bridge this gap and serve different groups of users, we provide the `SGPawn` system as a simple, intentionally limited, and beginner-friendly default solution that allows anyone to get up and running quickly.
|
||||
|
||||
At the same time, to support intermediate and advanced users, we have aimed for full OpenXR compatibility, opening the door to a wide range of advanced possibilities. Once enabled and loaded in Unreal Engine, the SenseGlove Unreal Engine Plugin registers itself as an `OpenXRHandTracking` provider. This makes it a fully compatible, drop-in replacement for Epic’s **OpenXRHandTracking** plugin.
|
||||
|
||||
As a result, [it can integrate seamlessly with any third-party system or plugin that consumes OpenXR hand-tracking data](../openxr/third-party-integrations/). Because the SenseGlove plugin is fully OpenXR-compliant, it provides hand-tracking data in the expected OpenXR format and becomes the active provider within Unreal. If your existing interaction system (for example, the VRE plugin) already relies on OpenXR hand-tracking, SenseGlove can function as a direct tracking source instead of a physical hand.
|
||||
|
||||
Furthermore, the SenseGlove OpenXR backend allows you to [develop and build your own hand interaction system from scratch](../openxr/third-party-tutorials/). This system can operate either via standard OpenXR hand-tracking or with a SenseGlove device interchangeably.
|
||||
|
||||
## Comparison of Available Approaches
|
||||
|
||||
The following table provides an overview and comparison of different hand interaction approaches available within the SenseGlove Unreal Engine Plugin ecosystem when it comes to hand-interaction systems:
|
||||
|
||||
| | **Built-in?** | **Works out of the box?** | **Beginner-friendly?** | **Learning Curve** | **Featureful** | **Customizable?** | **Supports Custom Gestures?** |
|
||||
|-------------------------------------|---------------|------------------------------------|--------------------------------------------------|-----------------------------|--------------------------------------------------------------------------|-----------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|
|
||||
| **SGPawn** | ✅ Yes | ✅ Yes | ✅ Most beginer-friendly | ✅ Very easy | ⚠️ Very basic | ⚠️ Very limited | ❌ Not yet, maybe added in the future |
|
||||
| **SenseGlove OpenXR** | ✅ Yes | ❌ Requires Blueprint or C++ coding | ✅ Requires a few hours of watching tutotrials | ✅ Moderate | ❌ You need to develop your own features and functionalities from scratch | ✅ Your imagination, creativity, and skill level are your limits | ✅ You have to implement your own recognition logic; example pinch detection algorithm is taught in the tutorial series |
|
||||
| **VR Expansion Plugin** | ❌ No | ⚠️ Partially – requires setup | ❌ Best suited for intermediate or advanced users | ⚠️ Steep | ✅ Diverse features and functionalities | ✅ Highly customizable | ✅ Via custom logic |
|
||||
| **Other OpenXR-compatible Plugins** | ❌ No | ❓ Check their documentation | ❓ Check their documentation | ❓ Check their documentation | ❓ Check their documentation | ❓ Check their documentation | ❓ Check their documentation |
|
||||
|
||||
## Going Beyond SGPawn
|
||||
|
||||
In the following sections, we will cover:
|
||||
|
||||
- [**The Puppeteer (Controller) / Puppet (Pawn) Architecture**](./sgpawn-events-puppeteer-puppet-architecture/): how to customize and control `SGPawn` through events.
|
||||
- [**SGHandTrackerComponent**](./sghandtrackercomponent/): how to obtain and consume SenseGlove hand-tracking data, the easy way.
|
||||
- [**SGHapticsComponent**](./sghapticscomponent/): how to add haptic feedback to your own or third-party hand interaction systems.
|
||||
|
||||
We have also covered more in-depth and advanced topics in other parts of this handbook, available in the following sections:
|
||||
|
||||
- [**OpenXR**](../advanced-topics/openxr/): an introduction to OpenXR fundamentals in Unreal Engine.
|
||||
- [**Consuming FXRHandTrackingState**](../advanced-topics/openxr/consuming-fxrhandtrackingstate/): explains the data layout of Unreal Engine’s `FXRHandTrackingState`.
|
||||
- [**Blueprint**](../advanced-topics/openxr/consuming-fxrhandtrackingstate/blueprint.md): demonstrates how to use `FXRHandTrackingState` data and render a debug hand in Blueprint.
|
||||
- [**C++**](../advanced-topics/openxr/consuming-fxrhandtrackingstate/cpp.md): demonstrates how to use `FXRHandTrackingState` data and render a debug hand in C++.
|
||||
- [**Third-Party Integrations**](../advanced-topics/openxr/third-party-integrations/): provides a sample Unreal Engine `5.4` project demonstrating how to integrate SenseGlove with the VR Expansion (VRE) Plugin.
|
||||
- [**Third-Party Tutorials**](../advanced-topics/openxr/third-party-tutorials/):— a tutorial series that guides you from beginner to advanced level in using `FXRHandTrackingState` to build your own hand interaction system by animating virtual hand meshes.
|
||||
|
||||
+593
@@ -0,0 +1,593 @@
|
||||
# SGPawn Events: The Puppeteer (Controller) / Puppet (Pawn) Architecture
|
||||
|
||||
The `SGPawn` (SenseGlove Pawn) is intentionally designed as a **data/event-driven puppet**. It detects touch, grab candidates, and hand state, but it does **not make gameplay decisions** on its own. Instead, it delegates the decisions via firing events
|
||||
|
||||
Usually these decisions are delegated to the `SGPlayerController` (or your own controller if you want to customize the behaviors), which acts as the **puppeteer** for SGPawn (the **puppet**):
|
||||
|
||||
- It registers to `SGPawn` events at the `BeginPlay` event.
|
||||
- It listens to `SGPawn` events.
|
||||
- It decides when to grab or release when certain conditions are met.
|
||||
- It applies gameplay logic.
|
||||
- It drives haptics or other responses.
|
||||
|
||||
That's [how `SGPlayerController` works under the hood](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGlove/Private/SenseGlove/GameFramework/SGPlayerController.cpp).
|
||||
|
||||
This separation ensures:
|
||||
|
||||
- Clean architecture.
|
||||
- Full and exnsible customization.
|
||||
- No hidden behavior inside `SGPawn`.
|
||||
- Deterministic control over interaction rules.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
SGPawn ---> Emits State Events ---> SGPlayerController decides what to do
|
||||
```
|
||||
|
||||
`SGPawn`:
|
||||
|
||||
- Tracks touch state.
|
||||
- Tracks grab candidates.
|
||||
- Tracks grabbed actors.
|
||||
- Emits events.
|
||||
|
||||
`SGPlayerController`:
|
||||
|
||||
- Subscribes to events.
|
||||
- Calls `Grab()` / `Release()`.
|
||||
- Applies custom interaction logic.
|
||||
- Updates haptics.
|
||||
|
||||
## Exposed Events
|
||||
|
||||
`SGPawn` provides the following event exposed to both C++ and Blueprint:
|
||||
|
||||
- `OnGrabStateUpdated`
|
||||
- `OnTouchStateUpdated`
|
||||
- `OnActorGrabbed`
|
||||
- `OnActorReleased`
|
||||
- `OnActorBeginTouch`
|
||||
- `OnActorEndTouch`
|
||||
|
||||
|
||||
|
||||
> [!IMPORTANT]
|
||||
> The current implementation of `SGPawn` relies on `3` grab colliders and `5`
|
||||
> touch colliders for grab and touch detection.
|
||||
|
||||
|
||||
|
||||
### On Grab State Updated Event
|
||||
|
||||
This is the **main decision event** for grabbing logic. The Pawn informs you:
|
||||
|
||||
> "Here is the current grab state. You decide what to do."
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_OneParam(ASGPawn, FGrabStateUpdatedEvent, const FSGGrabState& GrabState);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
This event is triggered only when the hand is visible and when any finger on the left or right hand, equipped with a grab collider, begins overlapping (colliding with) or ends overlapping (stops colliding with) an actor that owns an [`SGGrabComponent`](../../../getting-started/setup-grab-release-system/). So in summary it fires when the following conditions are met:
|
||||
|
||||
- The hand is **visible**.
|
||||
- Any finger (left or right hand) equipped with a **grab collider**:
|
||||
- **Begins overlapping** (starts colliding with), or
|
||||
- **Ends overlapping** (stops colliding with).
|
||||
- The overlapped actor owns an `SGGrabComponent`.
|
||||
|
||||
Subscribers to this event receive a snapshot of the `FSGGrabState` struct. At the moment the event is fired, the struct contains the following data:
|
||||
|
||||
- `Hand`: The `SGVirtualHandComponent` whose grab state was updated due to a finger beginning or ending an overlap with another actor.
|
||||
- `PreviousHandLocation`: `SGPawn` continuously records hand movement every engine tick. This field stores the hand’s location from the previous tick. It can be used to calculate object velocity or apply impulse forces when an object is thrown.
|
||||
- `HandVelocityHistory`: A history of previous hand locations, up to `SGPawn::MaxNumberOfHandVelocitySamples`. `MaxNumberOfHandVelocitySamples` is a `UPROPERTY` in `SGPawn` that defaults to `10` but can be adjusted as needed.
|
||||
- `ActorThumbCanGrab`: The actor currently overlapping with the thumb’s grab collider. If `null`, the thumb is not overlapping any grabbable actor (which means the actor has an `SGGrabComponent`).
|
||||
- `ActorIndexCanGrab`: The actor currently overlapping with the index finger’s grab collider. If `null`, the index finger is not overlapping any grabbable actor.
|
||||
- `ActorMiddleCanGrab`: The actor currently overlapping with the middle finger’s grab collider. If `null`, the middle finger is not overlapping any grabbable actor.
|
||||
- `GrabbedActor`: The actor currently being grabbed by this hand. If `null`, the hand is not grabbing anything at that moment.
|
||||
|
||||
Here is how the current `SGPlayerController` performs grab detection and instructs the `SGPawn` it controls to execute grab and release actions:
|
||||
|
||||
```cpp
|
||||
void ASGPlayerController::BeginPlay()
|
||||
{
|
||||
Super::BeginPlay();
|
||||
|
||||
ASGPawn* SGPawn{Cast<ASGPawn>(GetPawn())};
|
||||
if (!ensureAlwaysMsgf(IsValid(SGPawn), TEXT("%s"), TEXT("ERROR: invalid SenseGlove pawn!")))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
SGPawn->OnGrabStateUpdated().AddWeakLambda(
|
||||
this, [= SG_CAPTURE_THIS](const FSGGrabState& GrabState) -> void
|
||||
{
|
||||
if (!IsValid(SGPawn))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (!IsValid(GrabState.Hand))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
const bool bHandVisible = GrabState.Hand->IsVisible();
|
||||
if (!bHandVisible)
|
||||
{
|
||||
if (SGPawn->IsGrabbing(GrabState.Hand))
|
||||
{
|
||||
SGPawn->Release(GrabState.Hand);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (SGPawn->IsGrabbing(GrabState.Hand))
|
||||
{
|
||||
if (!IsValid(GrabState.ActorThumbCanGrab) ||
|
||||
(GrabState.ActorIndexCanGrab != GrabState.ActorThumbCanGrab
|
||||
&& GrabState.ActorMiddleCanGrab != GrabState.ActorThumbCanGrab))
|
||||
{
|
||||
SGPawn->Release(GrabState.Hand);
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
if (SGPawn->CanGrab(GrabState.Hand, GrabState.ActorThumbCanGrab))
|
||||
{
|
||||
SGPawn->Grab(GrabState.Hand, GrabState.ActorThumbCanGrab);
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
In this implementation, the `SGPlayerController` listens for grab state updates and determines whether the hand should grab or release an actor based on visibility and finger overlap conditions.
|
||||
|
||||
> [!TIP]
|
||||
> Haptic feedback is also handled and enforced through `SGPlayerController`.
|
||||
> You can
|
||||
> [review the full implementation in the plguin source code for `SGPlayerController`](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGlove/Private/SenseGlove/GameFramework/SGPlayerController.cpp).
|
||||
>
|
||||
> In general, with the current version of the plugin, you can integrate haptic
|
||||
> feedback into your own hand interaction system in several ways:
|
||||
>
|
||||
> - The [`SGHapticsComponent`](../sghapticscomponent/) high-level approach.
|
||||
> - The SenseGlove C++ API:
|
||||
> - Via the
|
||||
> [SGHandLayer API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCore/Public/SGCore/SGHandLayer.h).
|
||||
> - Via the
|
||||
> [SGHpaticGlove API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCore/Public/SGCore/SGHapticGlove.h).
|
||||
> - The SenseGlove Blueprint API:
|
||||
> - Via the
|
||||
> [SGHandLayer API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCoreKismet/Public/SGCoreKismet/SGHandLayerKismetLibrary.h).
|
||||
> - Via the
|
||||
> [SGHpaticGlove API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCoreKismet/Public/SGCoreKismet/SGHapticGloveKismetLibrary.h).
|
||||
> - Additionally, there is the
|
||||
> [`SGTouchComponent`](../../../getting-started/setup-touch-system/), which
|
||||
> provides simplified and limited functionality. On its own, it cannot trigger
|
||||
> haptics. It is designed to work in conjunction with the stock
|
||||
> `SGPlayerController` shipped with the SenseGlove Unreal Engine plugin.
|
||||
|
||||
### Touch State Updated Event
|
||||
|
||||
This is the **main decision event** for controlling the touch logic. The Pawn informs you:
|
||||
|
||||
> "Here is the current touch state. You decide what to do."
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_OneParam(ASGPawn, FTouchStateUpdatedEvent, const FSGTouchState& TouchState);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
This event is triggered only when the hand is visible and when any finger on the left or right hand, equipped with a grab collider, begins overlapping (colliding with) or ends overlapping (stops colliding with) an actor that owns an [`SGGrabComponent`](../../../getting-started/setup-grab-release-system/). So in summary it fires when the following conditions are met:
|
||||
|
||||
- The hand is **visible**.
|
||||
- Any finger (left or right hand) equipped with a **touch collider**:
|
||||
- **Begins overlapping** (starts colliding with), or
|
||||
- **Ends overlapping** (stops colliding with).
|
||||
- The overlapped actor owns an `SGTouchComponent`.
|
||||
|
||||
Subscribers to this event receive a snapshot of the `FSGTouchState` struct. At the moment the event is fired, the struct contains the following data:
|
||||
|
||||
- `Hand`: The `SGVirtualHandComponent` whose touch state was updated due to a finger beginning or ending an overlap with another actor.
|
||||
- `ActorThumbTouching`: The actor currently overlapping with the thumb’s touch collider. If `null`, the thumb is not overlapping any touchable actor (which means the actor has an `SGTouchComponent`).
|
||||
- `ActorIndexTouching`: The actor currently overlapping with the index’s touch collider. If `null`, the index is not overlapping any touchable actor.
|
||||
- `ActorMiddleTouching`: The actor currently overlapping with the middle’s touch collider. If `null`, the middle is not overlapping any touchable actor.
|
||||
- `ActorRingTouching`: The actor currently overlapping with the ring’s touch collider. If `null`, the ring is not overlapping any touchable actor.
|
||||
- `ActorPinkyTouching`: The actor currently overlapping with the pinky’s touch collider. If `null`, the pinky is not overlapping any touchable actor.
|
||||
|
||||
Here is how the current `SGPlayerController` performs touch detection and instructs the `SGPawn` it controls to apply haptics feedback:
|
||||
|
||||
```cpp
|
||||
void ASGPlayerController::BeginPlay()
|
||||
{
|
||||
Super::BeginPlay();
|
||||
|
||||
ASGPawn* SGPawn{Cast<ASGPawn>(GetPawn())};
|
||||
if (!ensureAlwaysMsgf(IsValid(SGPawn), TEXT("%s"), TEXT("ERROR: invalid SenseGlove pawn!")))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
SGPawn->OnTouchStateUpdated().AddWeakLambda(
|
||||
this, [= SG_CAPTURE_THIS](const FSGTouchState& TouchState) -> void
|
||||
{
|
||||
if (!IsValid(SGPawn))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (!IsValid(TouchState.Hand))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
const bool bHandVisible = TouchState.Hand->IsVisible();
|
||||
if (!bHandVisible)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
const bool bGloveConnected = TouchState.Hand->IsGloveConnected();
|
||||
if (!bGloveConnected)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
Pimpl->UpdateHapticsFeedback(TouchState);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
In this implementation, the `SGPlayerController` listens for touch state updates and determines whether the haptic feedbacks should be applied to the glove on that hand, or not. This decision is determined based on various conditions such as hand visibility and finger overlap conditions. Since each fingers haptic feedback application and the type of haptic feedback is decided individually, for the sake of readability the logic has been offloaded to an `SGPlayerController`'s internal function `Pimpl->UpdateHapticsFeedback()`. For example, it applies vibrotactile feedback to eligible fingers like this:
|
||||
|
||||
In this implementation, the `SGPlayerController` listens for touch state updates and determines whether haptic feedback should be applied to the glove on that hand. This decision is based on several conditions, such as hand visibility and finger overlap states. Since each finger’s haptic feedback and feedback type are evaluated individually, the detailed logic has been offloaded to the internal `SGPlayerController` function `Pimpl->UpdateHapticsFeedback()` for readability and separation of concerns.
|
||||
|
||||
For example, vibrotactile feedback is applied to eligible fingers as follows:
|
||||
|
||||
```cpp
|
||||
void ASGPlayerController::FImpl::UpdateHapticsFeedback(const FSGTouchState& TouchState)
|
||||
{
|
||||
if (!IsValid(TouchState.Hand))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
USGHapticGlove* Glove{TouchState.Hand->GetConnectedGlove()};
|
||||
if (!IsValid(Glove))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
const bool bGloveConnected = Glove->IsConnected();
|
||||
if (!bGloveConnected)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// some omitted code due to irrelevance
|
||||
....
|
||||
|
||||
// Send Vibrotactile to the thumb finger if it's touching an actor...
|
||||
if (IsValid(TouchState.ActorThumbTouching))
|
||||
{
|
||||
USGCustomWaveform* CustomWaveform(GetCustomWaveform(TouchState.ActorThumbTouching));
|
||||
Glove->SendCustomWaveform(CustomWaveform, ESGHapticLocation::ThumbTip);
|
||||
}
|
||||
|
||||
// Send Vibrotactile to the index finger if it's touching an actor...
|
||||
if (IsValid(TouchState.ActorIndexTouching))
|
||||
{
|
||||
USGCustomWaveform* CustomWaveform(GetCustomWaveform(TouchState.ActorIndexTouching));
|
||||
Glove->SendCustomWaveform(CustomWaveform, ESGHapticLocation::IndexTip);
|
||||
}
|
||||
|
||||
// Send Vibrotactile to the middle finger if it's touching an actor...
|
||||
if (IsValid(TouchState.ActorMiddleTouching))
|
||||
{
|
||||
USGCustomWaveform* CustomWaveform(GetCustomWaveform(TouchState.ActorMiddleTouching));
|
||||
Glove->SendCustomWaveform(CustomWaveform, ESGHapticLocation::MiddleTip);
|
||||
}
|
||||
|
||||
// Send Vibrotactile to the ring finger if it's touching an actor...
|
||||
if (IsValid(TouchState.ActorRingTouching))
|
||||
{
|
||||
USGCustomWaveform* CustomWaveform(GetCustomWaveform(TouchState.ActorRingTouching));
|
||||
Glove->SendCustomWaveform(CustomWaveform, ESGHapticLocation::RingTip);
|
||||
}
|
||||
|
||||
// Send Vibrotactile to the pinky finger if it's touching an actor...
|
||||
if (IsValid(TouchState.ActorPinkyTouching))
|
||||
{
|
||||
USGCustomWaveform* CustomWaveform(GetCustomWaveform(TouchState.ActorPinkyTouching));
|
||||
Glove->SendCustomWaveform(CustomWaveform, ESGHapticLocation::PinkyTip);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
As can be seen from the above code, the `SGCustomWaveform` is constructed via a separate helper function:
|
||||
|
||||
```cpp
|
||||
USGCustomWaveform* ASGPlayerController::FImpl::GetCustomWaveform(const AActor* Actor)
|
||||
{
|
||||
float Amplitude = 0.0f;
|
||||
float Duration = 0.0f;
|
||||
float Frequency = 0.0f;
|
||||
|
||||
if (IsValid(Actor))
|
||||
{
|
||||
const USGTouchComponent* TouchComponent{USGTouchComponent::GetTouchComponent(Actor)};
|
||||
if (IsValid(TouchComponent))
|
||||
{
|
||||
Amplitude = TouchComponent->GetVibrotactileAmplitude();
|
||||
Duration = TouchComponent->GetVibrotactileDuration();
|
||||
Frequency = TouchComponent->GetVibrotactileFrequency();
|
||||
}
|
||||
}
|
||||
|
||||
USGCustomWaveform* CustomWaveform{
|
||||
USGCustomWaveform::NewCustomWaveform(Owner, Amplitude, Duration, Frequency)
|
||||
};
|
||||
return CustomWaveform;
|
||||
}
|
||||
```
|
||||
|
||||
When it comes to force-feedback, the controller sends force-feedback to all fingers at once, while still constructing the force-feedback levels array via a separate function. `5` elements for `5` fingers indexed from thumb to pinky, where element `0` corresponds to the thumb, `1` to the index finger, and so on, with `4` representing the pinky; [see the `SGTouchComponent` documentation for more details](../../../getting-started/setup-touch-system/). This is how `UpdateHapticsFeedback()` sends force-feedback to the glove:
|
||||
|
||||
```cpp
|
||||
void ASGPlayerController::FImpl::UpdateHapticsFeedback(const FSGTouchState& TouchState)
|
||||
{
|
||||
if (!IsValid(TouchState.Hand))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
USGHapticGlove* Glove{TouchState.Hand->GetConnectedGlove()};
|
||||
if (!IsValid(Glove))
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
const bool bGloveConnected = Glove->IsConnected();
|
||||
if (!bGloveConnected)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// Queue the Force-Feedback command...
|
||||
TArray<float> ForceFeedbackLevels{
|
||||
GetForceFeedbackLevels(
|
||||
TouchState.ActorThumbTouching, TouchState.ActorIndexTouching, TouchState.ActorMiddleTouching,
|
||||
TouchState.ActorRingTouching, TouchState.ActorPinkyTouching)
|
||||
};
|
||||
Glove->QueueForceFeedbackLevels(MoveTemp(ForceFeedbackLevels));
|
||||
|
||||
// Send the haptics commands!
|
||||
Glove->SendHaptics();
|
||||
}
|
||||
```
|
||||
|
||||
Here is the current implementation for `GetForceFeedbackLevels()`:
|
||||
|
||||
```cpp
|
||||
TArray<float> ASGPlayerController::FImpl::GetForceFeedbackLevels(
|
||||
const AActor* ActorThumbTouching,
|
||||
const AActor* ActorIndexTouching,
|
||||
const AActor* ActorMiddleTouching,
|
||||
const AActor* ActorRingTouching,
|
||||
const AActor* ActorPinkyTouching)
|
||||
{
|
||||
const float ThumbForceFeedbackLevel = GetForceFeedbackLevel(ActorThumbTouching);
|
||||
const float IndexForceFeedbackLevel = GetForceFeedbackLevel(ActorIndexTouching);
|
||||
const float MiddleForceFeedbackLevel = GetForceFeedbackLevel(ActorMiddleTouching);
|
||||
const float RingForceFeedbackLevel = GetForceFeedbackLevel(ActorRingTouching);
|
||||
const float PinkyForceFeedbackLevel = GetForceFeedbackLevel(ActorPinkyTouching);
|
||||
|
||||
const TArray<float> ForceFeedbackLevels{
|
||||
ThumbForceFeedbackLevel,
|
||||
IndexForceFeedbackLevel,
|
||||
MiddleForceFeedbackLevel,
|
||||
RingForceFeedbackLevel,
|
||||
PinkyForceFeedbackLevel,
|
||||
};
|
||||
|
||||
return ForceFeedbackLevels;
|
||||
}
|
||||
```
|
||||
|
||||
> [!TIP]
|
||||
> You can
|
||||
> [review the full implementation in the plguin source code for `SGPlayerController`](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGlove/Private/SenseGlove/GameFramework/SGPlayerController.cpp).
|
||||
>
|
||||
> In general, with the current version of the plugin, you can integrate haptic
|
||||
> feedback into your own hand interaction system in several ways:
|
||||
>
|
||||
> - The [`SGHapticsComponent`](../sghapticscomponent/) high-level approach.
|
||||
> - The SenseGlove C++ API:
|
||||
> - Via the
|
||||
> [SGHandLayer API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCore/Public/SGCore/SGHandLayer.h).
|
||||
> - Via the
|
||||
> [SGHpaticGlove API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCore/Public/SGCore/SGHapticGlove.h).
|
||||
> - The SenseGlove Blueprint API:
|
||||
> - Via the
|
||||
> [SGHandLayer API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCoreKismet/Public/SGCoreKismet/SGHandLayerKismetLibrary.h).
|
||||
> - Via the
|
||||
> [SGHpaticGlove API](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveCoreKismet/Public/SGCoreKismet/SGHapticGloveKismetLibrary.h).
|
||||
> - Additionally, there is the
|
||||
> [`SGTouchComponent`](../../../getting-started/setup-touch-system/), which
|
||||
> provides simplified and limited functionality. On its own, it cannot trigger
|
||||
> haptics. It is designed to work in conjunction with the stock
|
||||
> `SGPlayerController` shipped with the SenseGlove Unreal Engine plugin.
|
||||
|
||||
### Actor Grabbed Event
|
||||
|
||||
This event is triggered whenever a grab is successfully performed by either the left or right hand. Subscribers to this event are notified about **which hand** performed the grab and **which grabbable actor** (an actor that owns an `SGGrabComponent`) was grabbed.
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_TwoParams(ASGPawn, FActorGrabbedEvent,
|
||||
const USGVirtualHandComponent* Hand,
|
||||
const AActor* Actor);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
### Actor Released Event
|
||||
|
||||
This event is triggered whenever a release is successfully performed by either the left or right hand. Subscribers to this event are notified about **which hand** performed the release and **which grabbable actor** (an actor that owns an `SGGrabComponent`) was released.
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_TwoParams(ASGPawn, FActorReleasedEvent,
|
||||
const USGVirtualHandComponent* Hand,
|
||||
const AActor* Actor);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
### Actor Begin Touch Event
|
||||
|
||||
This event is triggered whenever **any finger** on the left or right hand comes into contact with another actor. Subscribers to this event are notified about **which hand** initiated the overlap and **which touchable actor** (an actor that owns an `SGTouchComponent`) was touched.
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_TwoParams(ASGPawn, FActorBeginTouchEvent,
|
||||
const USGVirtualHandComponent* Hand,
|
||||
const AActor* Actor);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
### Actor End Touch Event
|
||||
|
||||
This event is triggered whenever **any finger** on the left or right hand ends contact with another actor that was previously touched by that finger. Subscribers to this event are notified about **which hand**'s finger ended the overlap and **which touchable actor** (an actor that owns an `SGTouchComponent`) is no longer being touched by that finger.
|
||||
|
||||
It is defined in C++ like this:
|
||||
|
||||
```cpp
|
||||
DECLARE_EVENT_TwoParams(ASGPawn, FActorEndTouchEvent,
|
||||
const USGVirtualHandComponent* Hand,
|
||||
const AActor* Actor);
|
||||
```
|
||||
|
||||
In Blueprint, the event appears as shown below:
|
||||
|
||||
|
||||
|
||||
## SGPawn Grab/Realase-Related Functions
|
||||
|
||||
In addition to the events described above, `SGPawn` provides a set of helper functions related to grabbing and releasing actors.
|
||||
|
||||
These functions allow you to:
|
||||
|
||||
- Check whether a specific hand can grab a given actor.
|
||||
- Determine whether a hand is currently grabbing an actor.
|
||||
- Retrieve the currently grabbed actor via an output parameter.
|
||||
- Trigger grab and release actions programmatically for either hand.
|
||||
|
||||
These helper functions are exposed to both [C++](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGlove/Public/SenseGlove/GameFramework/SGPawn.h) and [Blueprint](https://dev.azure.com/SenseGlove/_git/SenseGlove-Unreal?path=/Source/SenseGloveKismet/Public/SGKismet/SGPawnKismetLibrary.h):
|
||||
|
||||
```cpp
|
||||
public:
|
||||
FORCEINLINE bool CanLeftHandGrab(const AActor* Actor) const
|
||||
{
|
||||
return !IsLeftHandGrabbing() && ((IsValid(Actor) && Actor == LeftHandGrabState.ActorThumbCanGrab)
|
||||
&& (Actor == LeftHandGrabState.ActorIndexCanGrab || Actor == LeftHandGrabState.ActorMiddleCanGrab));
|
||||
}
|
||||
|
||||
FORCEINLINE bool IsLeftHandGrabbing(const AActor* Actor) const
|
||||
{
|
||||
return IsValid(Actor) && LeftHandGrabState.GrabbedActor == Actor;
|
||||
}
|
||||
|
||||
bool IsLeftHandGrabbing(AActor*& OutActor) const;
|
||||
|
||||
FORCEINLINE bool IsLeftHandGrabbing() const
|
||||
{
|
||||
return IsValid(LeftHandGrabState.GrabbedActor);
|
||||
}
|
||||
|
||||
FORCEINLINE bool CanRightHandGrab(const AActor* Actor) const
|
||||
{
|
||||
return !IsRightHandGrabbing() && ((IsValid(Actor) && Actor == RightHandGrabState.ActorThumbCanGrab)
|
||||
&& (Actor == RightHandGrabState.ActorIndexCanGrab || Actor == RightHandGrabState.ActorMiddleCanGrab));
|
||||
}
|
||||
|
||||
FORCEINLINE bool IsRightHandGrabbing(const AActor* Actor) const
|
||||
{
|
||||
return IsValid(Actor) && RightHandGrabState.GrabbedActor == Actor;
|
||||
}
|
||||
|
||||
FORCEINLINE bool IsRightHandGrabbing() const
|
||||
{
|
||||
return IsValid(RightHandGrabState.GrabbedActor);
|
||||
}
|
||||
|
||||
bool IsRightHandGrabbing(AActor*& OutActor) const;
|
||||
|
||||
FORCEINLINE bool CanGrab(const USGVirtualHandComponent* Hand, const AActor* Actor) const
|
||||
{
|
||||
return Hand == HandRight ? CanRightHandGrab(Actor) : CanLeftHandGrab(Actor);
|
||||
}
|
||||
|
||||
FORCEINLINE bool IsGrabbing(const USGVirtualHandComponent* Hand, const AActor* Actor) const
|
||||
{
|
||||
return Hand == HandRight ? IsRightHandGrabbing(Actor) : IsLeftHandGrabbing(Actor);
|
||||
}
|
||||
|
||||
bool IsGrabbing(const USGVirtualHandComponent* Hand, AActor*& OutActor) const;
|
||||
|
||||
FORCEINLINE bool IsGrabbing(const USGVirtualHandComponent* Hand) const
|
||||
{
|
||||
return Hand == HandRight ? IsRightHandGrabbing() : IsLeftHandGrabbing();
|
||||
}
|
||||
|
||||
public:
|
||||
FORCEINLINE void GrabLeft(AActor* Actor)
|
||||
{
|
||||
Grab(HandLeft, Actor);
|
||||
}
|
||||
|
||||
FORCEINLINE void GrabRight(AActor* Actor)
|
||||
{
|
||||
Grab(HandRight, Actor);
|
||||
}
|
||||
|
||||
void Grab(USGVirtualHandComponent* Hand, AActor* Actor);
|
||||
|
||||
void ReleaseLeft();
|
||||
|
||||
void ReleaseRight();
|
||||
|
||||
FORCEINLINE void Release(const USGVirtualHandComponent* Hand)
|
||||
{
|
||||
return Hand == HandRight ? ReleaseRight() : ReleaseLeft();
|
||||
}
|
||||
```
|
||||
|
||||
The same functions can be searched within the Blueprint Editor or accessed under the `SenseGlove > Game Framework > Pawn` category:
|
||||
|
||||
|
||||
|
||||
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
LFS
Executable
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
BIN
Binary file not shown.
-3
@@ -1,3 +0,0 @@
|
||||
# SGPawn Events
|
||||
|
||||
|
||||
@@ -38,7 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
- Added a [v2.7.x to v2.8.x migration guide](../misc/upgrade-guide/#upgrading-from-v27x-to-v28x) for transitioning from `FXRMotionControllerData` to `FXRHandTrackingState` for projects that directly consume SenseGlove OpenXR data.
|
||||
- Added documentation section [Roll Your Own Hand Manipulation System](../advanced-topics/roll-your-own-hand-manipulation-system/).
|
||||
- Added documentation section [SGPawn Events](../advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events/).
|
||||
- Added documentation section [SGPawn Events](../advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events-puppeteer-puppet-architecture/).
|
||||
- Added documentation section [SGHandTrackerComponent](../advanced-topics/roll-your-own-hand-manipulation-system/sghandtrackercomponent/).
|
||||
- Added documentation section [SGHapticsComponent](../advanced-topics/roll-your-own-hand-manipulation-system/sghapticscomponent/).
|
||||
- Updated [Enabling XR_EXT_hand_tracking on VR Headsets](../getting-started/enabling-xr-ext-hand-tracking-vr-headsets/) documentation, replacing `FXRMotionControllerData` usage with `FXRHandTrackingState`.
|
||||
|
||||
@@ -76,7 +76,7 @@ For users familiar with the basics, this section explores advanced features of t
|
||||
|
||||
- [Safe Glove Access in Blueprint](../advanced-topics/safe-glove-access-blueprint/)
|
||||
- [Roll Your Own Hand Manipulation System](../advanced-topics/roll-your-own-hand-manipulation-system/)
|
||||
- [SGPawn Events](../advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events/)
|
||||
- [SGPawn Events](../advanced-topics/roll-your-own-hand-manipulation-system/sgpawn-events-puppeteer-puppet-architecture/)
|
||||
- [SGHandTrackerComponent](../advanced-topics/roll-your-own-hand-manipulation-system/sghandtrackercomponent/)
|
||||
- [SGHapticsComponent](../advanced-topics/roll-your-own-hand-manipulation-system/sghapticscomponent/)
|
||||
- [OpenXR](../advanced-topics/openxr/)
|
||||
|
||||
Reference in New Issue
Block a user