Why should we use this repo?

[kadirnar]

Hello @zhiqwang ,

How are they different from the tensorRT code in the main repo? Fps and performance differences?

[zhiqwang]

Hi @kadirnar , Thank you for opening this ticket.

How are they different from the tensorRT code in the main repo?

You can check out https://zhiqwang.com/yolov5-rt-stack/notebooks/onnx-graphsurgeon-inference-tensorrt.html for some numerical comparison. Actually we only embedded the EfficientNMS to the YOLOv5 architecture for TensorRT deployment, and the other modules are almost identical or equivalent to the official YOLOv5.

The EfficientNMS plugin was introduced by TensorRT when they deploy the EfficientDet model, we first applied it to YOLOv5 architecture, and you can also use it in official YOLOv5 repo with some modification.

Why should we use this repo?

YOLOv5 is great, but I personally don't like the official YOLOv5's yaml and parsing approach to build models, I think the yaml-parsing mechanism is not flexible to extend with other frameworks, so I refactor the model building mechanism according to TorchVision's philosophy.

The abstractions of the modules have converged as the object detection models have evolved, and we believe that the pre-processing and post-processing will converge to a coarse-grained operator. The post-processing is something like the EfficientNMS, which can boost the efficiency of deployment if it is embedded within the model.

At the same time I am personally encouraging people to apply this approach to frameworks such as YOLOX, YOLOv6, YOLOv7 and others, in fact you can see that these frameworks have applied it in a variety of different ways.

[YoungjaeDev]

@zhiqwang
Why do source need to use at least tensorrt 8.2 version? Is it because of EfficientNMS plugin?

[zhiqwang]

Yep, TensorRT introduce EfficientNMS at 8.2 version

[YoungjaeDev]

@zhiqwang
I'm sorry, but if I ask you one more question, does tensorrt export include preprocessing (letterbox)? Has the floatcast issue mentioned in docs disappeared for the current letterbox function?
Also, what I'm curious about is that traceable or scriptable can be said to be faster compare to original? There is no advantage for speed mentioned this repo, so I wonder if the simple yolov5-rt has a purpose to write easily only

[zhiqwang]

Does tensorrt export include preprocessing (letterbox)?

Nope, YOLOv5's native preprocessing include some dynamic shape operations, I guess TensorRT is not particularly good at handling this at the moment.

Has the floatcast issue mentioned in docs disappeared for the current letterbox function?

Nope, This is due to the limitations of PyTorch, Fortunately I have seen PyTorch working on this task at pytorch/pytorch#90771 recently, and I am looking forward to this work being merged sooner.

Also, what I'm curious about is that traceable or scriptable can be said to be faster compare to original? There is no advantage for speed mentioned this repo, so I wonder if the simple yolov5-rt has a purpose to write easily only

That's a sad story, PyTorch 2.0 will be released next year, and seems they encourage users to use the torch.compile feature, maybe torch.jit.trace or torch.jit.script will be deprecated at some time in the future.

[kadirnar]

Thank you for the explanation.