Skip to content

[ECCV 2024] 3D Small Object Detection with Dynamic Spatial Pruning

License

Notifications You must be signed in to change notification settings

xuxw98/DSPDet3D

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

56 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

3D Small Object Detection with Dynamic Spatial Pruning

Introduction

This repo contains PyTorch implementation for paper 3D Small Object Detection with Dynamic Spatial Pruning based on MMDetection3D. Look here for 中文解读.

3D Small Object Detection with Dynamic Spatial Pruning
Xiuwei Xu*, Zhihao Sun*, Ziwei Wang, Hongmin Liu, Jie Zhou, Jiwen Lu

teaser

News

  • [2024/7/01]: DSPDet3D is accepted to ECCV 2024!
  • [2023/6/04]: We transfer DSPDet3D to extremely large scenes and show great performance! Demo will be released in our project page.
  • [2023/5/01]: Code release.

Method

Overall pipeline of DSPDet3D:

overview

Getting Started

For data preparation and environment setup:

For training and evaluation:

Demo

This is a tutorial on how to use class-agnostic DSPDet3D on custom data and visualize the results. Please download checkpoint from HERE and move it to demo folder. We provide two demo scenes from ScanNet and Matterport3D. You can download (ScanNet, Matterport3D) and also put them into demo folder. Then run the following command for detection and visualization.

Dataset Scannet Matterport3D
Command bash demo/demo.sh demo/scannet.ply demo/config_room.py bash demo/demo.sh demo/mp3d.ply demo/config_building.py
Result vis vis2

You can also try DSPDet3D on your own data in ply format. Run

bash demo/demo.sh /path/to/your/ply demo/config_{}.py

We use different hyperparamters of 3D NMS for different scales of scenes. For room-size scenes, use config_room.py. For building-level scenes, use config_building.py. You can also adjust the prune_threshold in the config file to tradeoff between accuracy and efficiency.

Main Results

We provide the checkpoints for quick reproduction of the results reported in the paper. The pruning threshold can be adjusted freely to tradeoff between accuracy and efficiency without any finetuning.

Benchmark [email protected] [email protected] Downloads
ScanNet-md40 65.04 54.35 model
TO-SCENE-down 66.12 58.55 model

Comparison with state-of-the-art methods on TO-SCENE dataset:

drawing

Visualization results on ScanNet:

vis

Visualization results on Matterport3D:

vis2

Acknowledgement

We thank a lot for the flexible codebase of FCAF3D and valuable datasets provided by ScanNet and TO-SCENE.

Bibtex

If this work is helpful for your research, please consider citing the following BibTeX entry.

@article{xu2023dsp, 
      title={3D Small Object Detection with Dynamic Spatial Pruning}, 
      author={Xiuwei Xu and Zhihao Sun and Ziwei Wang and Hongmin Liu and Jie Zhou and Jiwen Lu},
      journal={arXiv preprint arXiv:2305.03716},
      year={2023}
}