How to integrate react-three-fiber with 8thWall?

[nishantnaagnihotri]

Hi Team

I am trying to create a poc using 8thWall.

This platform allows to render ThreeJS objects around image targets on camera feed.
Under the hood, it manipulated the canvas dom element at the time of initialisation.

I was feeling quite excited to integrate react-three-fiber in this project, but I am unable to accomplish it.
One reason, I think, is above manipulation of canvas dom, about which React is not aware.

I need your advice, if there is a way to integrate it.

[nishantnaagnihotri]

I was able to avoid dom manipulation of canvas element.

But the issue that I am facing now is that, I am not able to pass camera, scene and renderer as parameters to the Canvas component.

Any help is much appreciated.

[maxfelker]

What exactly are you trying to accomplish? If you can post some example of what you're attempting and what is happening, that might be a good way for us to start helping.

Thanks!

[nishantnaagnihotri]

I will have to prepare an example.

Just to add to the explanation for now, 8thWall uses ThreeJS to render objects over camera feed.
Here I create a canvas tag and pass its ref to 8thWall.
8thWall provides the camera, scene and render objects.
And I am able to just plain ThreeJS to draw objects.

But now I was curious to use react-three-fibre in this project.

Hence I used Canvas component.
From here I was able to pass ref of canvas element to 8thWall.
So that is good.

However, 8thWall has its own set of camera, scene and renderer objects, which enables me to draw graphics on camera feed.
But I am not able to reuse this set of Objects with Canvas component.
I have observed that it creates it's own set of objects.
Hence any declarative ThreeJS object that I create under Canvas component, it doesn't render on the camera feed.

I hope I am able to explain :)

Please let me know your suggestions.

[drcmda]

you can use portals to render declarative stuff into any existing three object, but that comes with some drawbacks.

[ThomasRutzer]

Is there any working example for the 8thWall integration?

[Splact]

I did some tries following this article by @nishantnaagnihotri, I passed the r3f canvas to 8th wall, set camera and added to 8th wall scene a <group> wrapping all my objects components. But after this I still have some issues:

• repeated warning performance warning: READ-usage buffer was read back without waiting on a fence. This caused a graphics pipeline stall.
• when the target is on camera, the mesh is placed correctly over it just for 1 frame, then the mesh is misplaced (keep track of the target)

[drcmda]

@Splact @ThomasRutzer @nishantnaagnihotri

never tried 8thwall, but from what you have written you could probably do it like this:

import { createPortal, useThree, useFrame } from "react-three-fiber"

<Canvas>
  <EightwallBridge>
    <mesh>
      <boxBufferGeometry />
      <meshBasicMaterial color="hotpink" />
    </mesh>
  </EightwallBridge>
</Canvas>

function EightwallBridge({ children }) {
  const { setDefaultCamera } = useThree()
  useEffect(() => void setDefaultCamera(EIGTHWALLCAMERA), [])
  useFrame(() => null, 1) // switch off rendering, let 8thwall take care of it
  return createPortal(children, EIGHTWALLSCENE)
}

that should prevent r3f from rendering (on the gpu), but it will render the declarative scene into 8thwall. also makes 8thwalls cam the default, so raytracing should work. we're abusing both systems at this point, but if it works it works.

[ifawcett4]

Were you ever able to find a successful way to integrate R3F into 8th Wall? I am attempting the same and having issues getting even a basic setup working.

[vettorazi]

Hey, I just pushed a boilerplate for React Three.js + drei for 8thwall. here's the repo!
https://github.com/vettorazi/react-three-8thwall

[vettorazi]

Hey, I just pushed a boilerplate for React Three.js + drei for 8thwall. here's the repo!
https://github.com/vettorazi/react-three-8thwall

[fl-martin]

Hi! Thank you for sharing this useful boilerplate. I'd like to know the reason for creating a second canvas element (canvasRef) instead of using the one created by R3F and passing it to XR8.run(), like here.

[vettorazi]

When I started the project, I had some elements on both canvases, and I wanted to leave it open in case someone wants to write a three.js in an imperative way :)

[fl-martin]

Good to know! Probably I'll start a conversation on your repo soon^^