Skip to content

Commit

Permalink
react-three/handles docs
Browse files Browse the repository at this point in the history
  • Loading branch information
bbohlender committed Jan 20, 2025
1 parent 46c2ab0 commit 8550621
Show file tree
Hide file tree
Showing 18 changed files with 921 additions and 21 deletions.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ export function App() {
## Tutorials

- 💾 [Store](https://docs.pmnd.rs/xr/tutorials/store)
- 👌 [Interactions](https://docs.pmnd.rs/xr/tutorials/interactions)
- 👆 [Interactions](https://docs.pmnd.rs/xr/tutorials/interactions)
- 👌 [Handles](https://docs.pmnd.rs/xr/tutorials/handles)
- 🔧 [Options](https://docs.pmnd.rs/xr/tutorials/options)
- 🧊 [Object Detection](https://docs.pmnd.rs/xr/tutorials/object-detection)
-[Origin](https://docs.pmnd.rs/xr/tutorials/origin)
Expand All @@ -80,6 +81,10 @@ export function App() {
- 🎯 [Hit Test](https://docs.pmnd.rs/xr/tutorials/hit-test)
-[Guards](https://docs.pmnd.rs/xr/tutorials/guards)

## External Tutorials

- 🥇 [**WebXR First Steps React** by Meta Quest](https://github.com/meta-quest/webxr-first-steps-react)

## Roadmap

- 🤳 XR Gestures
Expand Down
32 changes: 17 additions & 15 deletions docs/getting-started/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,32 +16,35 @@ useFrame((state) => console.log(state.camera.getWorldPosition(new Vector3())))

In contrast to non-immersive 3D applications, the camera transformation in MR/VR/AR applications should never be directly controlled by the developer since the user's head movement must control the camera's transformation. Therefore, pmndrs/xr provides the XROrigin component, which allows to control where the session's origin is placed inside the 3D scene. The session origin is at the users' feet once they recenter their session. This allows to implicitly control the camera position but prevents the user from getting motion sick when their movement is not reflected in the camera's movement.

##

## Having problems accessing the camera position or rotation.

Check if you have OrbitControls, CameraControls, or other controls in your scene and make sure to place an `<IfInSessionMode deny={['immersive-ar', 'immersive-vr']}>` guard around them when in XR. This prevents overwriting the camera transformation which is controlled through WebXR when inside an immersive session and allows to access the correct transformation.
Check if you have OrbitControls, CameraControls from `@react-three/drei`, or other controls in your scene and make sure to place an `<IfInSessionMode deny={['immersive-ar', 'immersive-vr']}>` guard around them when in XR or replace them with `OrbitHandles` or `MapHandles` from `@react-three/handle`. This prevents overwriting the camera transformation which is controlled through WebXR when inside an immersive session and allows to access the correct transformation.

```tsx
const OrbitControlsWrapper = () => {
return (
<IfInSessionMode deny={['immersive-ar', 'immersive-vr']} >
<OrbitControls />
</IfInSessionMode>
)
}
import { OrbitHandles } from '@react-three/handles'
import { noEvents, PointerEvents } from '@react-three/xr'

<Canvas events={noEvents}>
<PointerEvents />
<OrbitHandles />
</Canvas>
```

## I cannot enter the XR session!

1. **Missing Https**
If you are trying to enter the AR or VR modus and nothing is happening, make sure that you are accessing the website using `https://`.
In case you are using vite, we recommend using the `@vitejs/plugin-basic-ssl` to try out your vite application on your device while developing.
If you are trying to enter the AR or VR modus and nothing is happening, make sure that you are accessing the website using `https://`.
In case you are using vite, we recommend using the `@vitejs/plugin-basic-ssl` to try out your vite application on your device while developing.

2. **Missing XR component**
If you made sure that the website is accessed using `https://` and still nothing happens when executing `enterAR` or `enterVR`, it is likely that the `<XR>` component is missing. Be sure to add the `<XR>` component directly into the `<Canvas>` and make sure both the `<Canvas>` and the `<XR>` component are present when the button is pressed.
If you made sure that the website is accessed using `https://` and still nothing happens when executing `enterAR` or `enterVR`, it is likely that the `<XR>` component is missing. Be sure to add the `<XR>` component directly into the `<Canvas>` and make sure both the `<Canvas>` and the `<XR>` component are present when the button is pressed.

3. **Entering while loading content**
If you cannot enter the VR or AR experience, there might be assets in your scene that are loading.
Make sure to place a suspense boundary around your scene. With this setup, the `<XR>` component stays mounted while your scene loads.
If you cannot enter the VR or AR experience, there might be assets in your scene that are loading.
Make sure to place a suspense boundary around your scene. With this setup, the `<XR>` component stays mounted while your scene loads.

```tsx
<Canvas>
<XR>
Expand All @@ -68,7 +71,7 @@ For non-handheld VR and AR experiences, you can use [react-three/uikit](https://

## Does it work on iOS?

WebXR for VR experiences is supported on Safari for Apple Vision Pro.
WebXR for VR experiences is supported on Safari for Apple Vision Pro.
WebXR is not supported on iOS Safari yet. The alternative is to use products such as [Variant Launch](https://launch.variant3d.com/), which allow to build WebXR experiences for iOS.

## XRSpace
Expand All @@ -78,4 +81,3 @@ If you are placing `<XRSpace>` components outside of the `<XROrigin>` while chan
## `onClick` does not play video or allow file uploading (in certain browsers)

As a performance optimization the react-three/xr event system batches html user events per frame. This only applies if you are using `PointerEvents`, `forwardHtmlEvents`, or `forwardObjectEvents`. This can cause issue when executing functions that require a user action. For instance, uploading a file through a input element in a safari can only be triggered manually when immediately caused by a user input. For these use cases, please disable the event batching performance optimization through the options by setting `batchEvents` to `false`.

Binary file added docs/handles/annotated-door.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/handles/door-handle.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/handles/editor.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
95 changes: 95 additions & 0 deletions docs/handles/handle-component.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: Handle Component
description: The Handle and HandleTarget components and their properties
nav: 24
---

The `Handle` component is the core component of the `@react-three/handle` library, which is built on the `HandleStore`. This store provides developers with more control over the current state and user interactions.

## Handle Store

The handle store manages the handle's state and translates user interactions into state modifications.

### Exposed Functions

**getState**
Allows retrieval of the state of the current interaction. If no interaction is currently happening, `getState` returns `undefined`.

**capture**
Normally, a pointer is captured when the user interacts with an object. The `capture` method allows programmatic initiation of an interaction without requiring user input.

**save**
Allows saving the current state so that modifications to the handle options do not affect modifications from previous interactions.

**cancel**
Normally, an interaction is canceled when the user releases the button that started the interaction. The `cancel` function allows programmatic cancellation of the interaction.

## Handle Component

**Example**
```tsx
<Handle translate="as-scale" scale={{ uniform: true }}>
<mesh>
<boxGeometry />
</mesh>
</Handle>
```
*Allows scaling of the cube by dragging it outward from the cube's center.*

### Properties

**handleRef**
Allows overriding to pass a custom handle object.

**useTargetFromContext**
Required to use the target provided by a surrounding `HandleTarget` component.

**getHandleOptions**
Allows passing a function that dynamically generates options to override the current handle options.

**bind**
Allows disabling automatic binding of the event listeners to the provided handle, which can be necessary when capturing pointers manually.

**apply**
The `apply` function is used to apply a state modification that originates from a user interaction to the state. This property allows overriding the default apply function, giving the developer complete control over how modifications affect the state. For instance, instead of applying the modification directly, the developer can apply it to their own state management solution. The state management solution can then apply the modification to the handle target.

**alwaysUpdate**
In situations where the handle target is placed inside a constantly changing group, the `alwaysUpdate` flag ensures that the handle target's transformation is updated every frame to reflect the current state of the handle.

**multitouch**
By default, handles can be interacted with using multiple input devices. By setting `multitouch` to `false`, only the first input device will be used.

**stopPropagation**
By default, events that occur on handles are not propagated upwards and therefore do not reach their ancestors. Setting `stopPropagation` to `false` will re-enable event propagation for events that occur on the handle.

**rotate**
The `rotate` property allows configuring if and how the user can rotate the target. Setting `rotate` to `false` disables rotation. Setting `rotate` to `x` restricts rotation to the x-axis. Setting `rotate` to `{ x: false, y: [0, Math.PI] }` disables rotation on the x-axis and restricts rotation on the y-axis to be between 0 and 180°, while rotation on the z-axis is enabled.

**scale**
The `scale` property allows configuring if and how the user can scale the target. Setting `scale` to `false` disables scaling. Setting `scale` to `x` restricts scaling to the x-axis. Setting `scale` to `{ x: false, y: [1, 2] }` disables scaling on the x-axis and restricts the scaling factor on the y-axis to be between 1 and 2, while scaling on the z-axis is enabled.

**translate**
The `translate` property allows configuring if and how the user can translate the target. Setting `translate` to `false` disables translation. Setting `translate` to `x` restricts translation to the x-axis. Setting `translate` to `{ x: false, y: [-1, 1] }` disables translation on the x-axis and restricts translation on the y-axis to be between -1 and 1, while translation on the z-axis is enabled. Furthermore, the `translate` property can be configured to transform translations into rotations and/or scalings using `translate="as-scale"`, allowing the user to scale the target by grabbing and moving the handle.

**ref**
Allows retrieval of a reference to the internal handle store (`<Handle ref={handleStoreRef}>`).

## Handle Target Component

The `HandleTarget` component allows declaratively specifying a handle target that is hierarchically above the `Handle` component. To prevent accidentally providing a different target to a handle, using the target from the context requires setting the `useTargetFromContext` on the `Handle` component.

**Example**
```tsx
<HandleTarget>
<group>
<mesh>
<boxGeometry />
<Handle useTargetFromContext>
<mesh position-x={2}>
<boxGeometry />
</mesh>
</Handle>
</mesh>
</group>
</HandleTarget>
```
78 changes: 78 additions & 0 deletions docs/handles/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
title: Handles
description: Easily build 3D interactions using the concept of handles
nav: 23
---

Handles are everywhere, from scrollbar thumbs to window bars to door handles.

![](./react-three-handle-gif-1.gif)

Handles make it possible to move, rotate, and scale 2D or 3D objects using intuitive gestures. `@react-three/handle` makes it super simple to build these intuitive 3D handles for XR and non-XR devices with only a few lines of code. Let's showcase that while building a 3D door.

![](./annotated-door.jpg)
_Door model from [witnessk](https://sketchfab.com/witnessk)_

To build our door, we have two components: the door handle and the door body, which the handle should rotate when grabbed. Therefore, the door body is the handle target, on which the transformations from the handle should be applied.

We can build this door by downloading the GLTF model from Sketchfab and transforming the model into its individual parts using [gltfjsx](https://gltf.pmnd.rs/). This separates the door frame, the door body, and the door handle, allowing us to wrap the door handle with the `<Handle>` component and the door body with the `<HandleTarget>` component.

```tsx {6,10,14-15}
export function Door() {
const { nodes, materials } = useGLTF('/door.glb')
return (
<group rotation={[-Math.PI / 2, 0, 0]} scale={100}>
<group position={[-0.435, -0.101, 0.249]}>
<HandleTarget>
<mesh geometry={nodes.Plane001_Glossy_0.geometry} material={materials.Glossy} />
<mesh geometry={nodes.Plane001_Door_0.geometry} material={materials.Door} />
<mesh geometry={nodes.Plane003_Door_0.geometry} material={materials.Door} position={[0.852, 0.017, 0.782]} />
<Handle useTargetFromContext translate="as-rotate" rotate={{ x: false, y: false, z: [-Math.PI, 0] }}>
<group position={[0.81, 0.043, 0.803]}>
<mesh geometry={nodes.Circle002_Glossy_0.geometry} material={materials.Glossy} />
</group>
</Handle>
</HandleTarget>
</group>
<mesh geometry={nodes.Plane002_Glossy_0.geometry} material={materials.Glossy} />
<mesh geometry={nodes.Plane002_Door_0.geometry} material={materials.Door} />
</group>
)
}
```

Next, we need to configure the handle to rotate the door when grabbed. We instruct it to use the target from the context using `useTargetFromContext`, making sure the transformations are applied to the door body. Additionally, we ensure that moving the handle is translated into a rotation using the `translate="as-rotate"` property. Lastly, we disable rotations on all axes except for the `z` axis and limit the rotation between -180° and 0°.

*Learn more about all the available properties for the handle component [here](./handle-component.md).*

The final result looks like this (I added an additional handle around the door handle that allows it to rotate on its own y-axis).

![](./door-handle.gif)

### Editor Example

Handles are made for all kinds of use cases, from games to professional applications, which we emphasized by building the following editor demo that uses over 40 handles for moving the elements on the screen, resizing the virtual screen, and moving the virtual camera. The nice part is that it works on all devices, ranging from mouse-driven PCs to eye-driven mixed-reality headsets (Apple Vision Pro).

![](./editor.gif)

You can check it out [here](https://pmndrs.github.io/xr/examples/editor/) and read the source code (700 LOC) [here](https://github.com/pmndrs/xr/tree/main/examples/editor/app.tsx).

### Screen Handles

A specific type of handle is the screen handle, where not an individual object is the handle, but the whole screen. Therefore, we build screen handles, which allow panning, zooming, and rotating the camera using swipe, drag, and scroll interactions. These are built on the ideas of `OrbitControls` and `MapControls` from Three.js, but respect the event system (e.g., you don't need to disable them when you drag an object) and are automatically forwarded to virtual screens, as shown in the editor demo.

Learn more about the available screen handles [here](./screen-handle-components.md).

### Prebuilt Handles

For many use cases, such as 3D editors, handles often come in specific forms, like the `TransformControls` available in Three.js. These opinionated pre-built handles have proven to be very useful, which is why `@react-three/handle` ships with implementations for `TransformHandles` and `PivotHandles`.

![](./prebuild-handles.gif)

Learn more about the available prebuilt handles [here](./screen-handle-components.md).

## Sponsors

This project is supported by a few companies and individuals building cutting-edge 3D Web & XR experiences. Check them out!

![Sponsors Overview](https://bbohlender.github.io/sponsors/screenshot.png)
Binary file added docs/handles/pivot.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/handles/prebuild-handles.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 8550621

Please sign in to comment.