ONNX export failed: Couldn't export Python operator _DeformConv

[XYudong]

Hi,

• in docker container from pytorch/pytorch:1.11.0-cuda11.3-cudnn8-devel
• torch 1.11.0
• torchvision 0.12.0
• latest detectron2 main branch
• python3 export.py failed while exporting model sparse_inst_r50vd_dcn_giam_aug.yaml

Error log:

Traceback (most recent call last):
  File "export.py", line 285, in <module>
    torch.onnx.export(
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/__init__.py", line 305, in export
    return utils.export(model, args, f, export_params, verbose, training,
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 118, in export
    _export(model, args, f, export_params, verbose, training, input_names, output_names,
  File "/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py", line 738, in _export
    proto, export_map, val_use_external_data_format = graph._export_onnx(
RuntimeError: ONNX export failed: Couldn't export Python operator _DeformConv


Defined at:
/yolov7_d2/detectron2/detectron2/layers/deform_conv.py(394): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/yolov7_d2/yolov7/modeling/backbone/resnetvd.py(105): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/container.py(141): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/yolov7_d2/yolov7/modeling/backbone/resnetvd.py(509): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/yolov7_d2/yolov7/modeling/meta_arch/sparseinst.py(147): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1098): _slow_forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py(118): wrapper
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py(127): forward
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(1110): _call_impl
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py(1166): _get_trace_graph
/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py(391): _trace_and_get_graph_from_model
/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py(440): _create_jit_graph
/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py(499): _model_to_graph
/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py(719): _export
/opt/conda/lib/python3.8/site-packages/torch/onnx/utils.py(118): export
/opt/conda/lib/python3.8/site-packages/torch/onnx/__init__.py(305): export
export.py(285): <module>

[leandro-svg]

You can check this git issue where @jinfagang asked about DCN aswell :
onnx/onnx#2395

[lucasjinreal]

DCN onnx export is not supported by yolov7-d2

[XYudong]

DCN onnx export is not supported by yolov7-d2

OK, thanks.

So, if I want to use DCN in TensorRT, I have to convert it directly from Pytorch to .trt?