InstancedMesh pointer events not working after updating the underlying matrix

[callumbooth]

Hi all,

I'm not sure if this is a R3F issue or a ThreeJS issue.

I've got a repro here. https://codesandbox.io/p/sandbox/instanced-vertex-colors-forked-vz36hj and the video below.

Basically, I have taken the Instanced vertex colours example and slimmed it down. I've added a couple of buttons which just adds an offset to the x position of each instance. On the first load, if you hover over the left hand side the pointer events work. However if you click the move left button a couple of times. The left hand side instances no longer work with the pointer events. If you click more times, more of the instancedMesh instances will not work with the pointer events. Almost like the raycaster isn't updated correctly.

Any ideas, if I'm doing something wrong?

Screen.Recording.2025-10-20.at.11.11.21.mov