Failure case of TRACE in PoseTrack video

[hongsukchoi]

Hi @Arthur151

Thank you for your continuous great works! I have a question about TRACE.

How can I visualize the global camera trajectories (sequential 6D camera poses)?

I have world_cams output from the trace model's function. However, I found two problems. 1) It doesn't have rotation of cameras 2) More critically, multiple camera poses are defined per frame. It violates TRACE's contribution that TRACE assumes a single world frame for one scene.

Overall, I get this kind of weird result with multiple camera trajectories. The result also shows huge misalignment between people. They all look good in the 2D rendering output, but it's bad in 3D visualization of Viser. Could you clarify if I am doing something wrong, or the result in 3D visualization is something likely to happen? I obtained camera rotations from cam_rot_motion_maps[pred_batch_ids,:,pred_czyxs[:,1],pred_czyxs[:,2]].

Caution: These images are not exactly time synchronized.
Full videos:
https://youtu.be/1ER6bYZfDyY
https://youtu.be/YdAeIbQHC2c

This is my visualization code:
https://github.com/hongsukchoi/TRACE_vis/tree/hongsuk/simple_romp/trace2
https://github.com/hongsukchoi/TRACE_vis/blob/hongsuk/simple_romp/mp_global_viser_vis.py