Information about tracing models

[edufschmidt]

Hi @ShunChengWu, I'm trying to trace the models you provide in this repo to use them on ONNX in SceneGraphFusion, but am struggling a bit due to several factors. It seems that a few of the modules the code relies on don't have the necessary trace function implemented. Also, for the models built on pytorch-geometric, it seems that tracing is not supported due to the utilization of pytorch-scatter, which in turn uses functions that don't play well with ONNX. I'm wondering if you could confirm that it is possible and you have traced the models for utilization with SceneGraphFusion. If so, where can I find instructions on how to reproduce that? If not, what do you suggest one should do in order to be able to integrate the provided models with SceneGraphFusion? Thanks in advance!

[edufschmidt]

Ok, I was able to trace at least one of the models by:

• upgraded pretty much every module to its latest version;
• changed the onnx opset (>18) used when exporting;
• modified the 3DSSG code a bit;
• monkeypatched pytorch’s min and max operators;

I am now struggling with using the traced model with SceneGraphFusion. Specifically, I couldn't find instructions to generate the args.json SceneGraphFusion seems to rely on. Any help would be much appreciated.

[ShunChengWu]

The newer version of this repo is not supported. You will have to use the cvpr21 branch to be compatible with the SceneGraphFusion repo

[edufschmidt]

Thanks @ShunChengWu for your reply. Does that mean that the only model compatible with SceneGraphFusion is the one available in this link found in the cvpr21 branch? Could you please also confirm that the other models whose weights are available for download here in the main branch are not compatible and have not been tested with SceneGraphFusion? Thanks in advance!

[ShunChengWu]

The models trained with the cvpr21 branch are compatible with the SceneGraphFusion repo. The models from the current main branch are not compatible

[edufschmidt]

Thanks again for your reply. I understand. Are any weights available for the models in the cvpr21 branch or do I have to do the training myself?

[ShunChengWu]

The link you mentioned before is the trained and traced model weights. As far as I remember, we didn't provide the typical checkpoint files.