MeshStandardMaterial normalMap reflection has offset & strange behavior, texture issue or bug?

[ArsenicBismuth]

I found similar issue related to this, but in the drei repo: pmndrs/drei#1558

But in my case, it also happens on the MeshStandardMaterial with normalMap. I can't really be sure if it's related to the textures or not, but I tested different PBR materials online from different creators, and they exhibit similar results.

Look the images below. Everything is default, and yet the shine shown on the floor is not where the actual PointLight is. And what's worse, is that the offset is inconsistent. On certain camera angle you could see worse deviation.

CodeSanbox: https://codesandbox.io/p/sandbox/volumetric-spotlight-forked-tq5t9v

Also, if I move the point light close to the surface, it somehow emits in a specific direction on the surface:

Which disappears if I remove the normalMap:

[ArsenicBismuth]

Turns out the fix is very simple, I just need to change the Color Space of the normal map like so:

normalMap-colorSpace={THREE.LinearSRGBColorSpace}

Final results are at the bottom.

But this is where the problem lies. After a bit of testing, it seems the default Three.js (code below) would initialize colored textures (both map and normalMap) as THREE.NoColorSpace, thus the colors of your textures would be wrong but the normals are perfect.

const mat = new THREE.MeshStandardMaterial();
mat.map = map;
mat.normalMap = normalMap
map.colorSpace = THREE.SRGBColorSpace; // To correct the base map

Meanwhile, R3F would apply all colored textures as THREE.SRGBColorSpace. So, as a result, you will get correctly colored base texture, but wrong normal.

<mesh receiveShadow position={[0, 0, 0]} rotation={[-Math.PI / 2, 0, 0]}>
    <planeGeometry args={[10, 10]} />
    <meshStandardMaterial
      map={map}
      normalMap={normalMap}

      // To correct the normal map
      normalMap-colorSpace={THREE.LinearSRGBColorSpace}
    />
  </mesh>

Is this an intentional behavior? If we're setting default values different from the official Three.js, why not automatically set normalMap color space to THREE.LinearSRGBColorSpace altogether? Since we don't expect normals to be defined as non-linear color space anyway.

---

Final result after correcting the normal map's color space:

[CodyJasonBennett]

Not exactly intended behavior, but technically a breaking change to correct. It was once intended to set an sRGB color space for textures, assuming they contained color, but some textures contain data like normals/roughness, which makes tough to debug bugs. In the future, more color spaces for input textures will be supported in three as well. We'll not touch textures' color spaces except for known built-in material properties with our next major v9. https://github.com/pmndrs/react-three-fiber/blob/v9/docs/tutorials/v9-migration-guide.mdx#color-management-of-textures

[ArsenicBismuth]

@CodyJasonBennett Ahhh okay good to know it's being acknowledged! The new change is definitely a welcome one,

Will keep this issue open until then, just as a note since it can be very tough to debug for people (as you said).