CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradations

This is the official repository of the paper: CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradations for segmentation tasks.

🛠️ Installation

• Step0. Download and install Miniconda from the official website.
• Step1. Create a conda environment and activate it.

conda create --name mmseg python=3.8 -y
conda activate mmseg

• Step2.Install PyTorch following official instructions, e.g.

conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch

• Step3. Install MMEngine and MMCV using MIM.

pip install -U openmim
mim install mmengine
mim install "mmcv>=2.0.0"

• Step4. Install other related packages

cd CPA_Seg
pip install -r ./cpa/requirements.txt

📁 Data Preparation

You can download our reorganized ACDC datase (in cityscapes format) from this link. (pwd: tuag) Remember to modify the data_root in configs/__base__/datasets/cityscapes.py.

🎯 Usage

• Recompile the code.

pip install -v -e .

• Training

# CPA-Enhancer + deeplabv3plus
python tools/train.py configs/deeplabv3plus/acdc_deeplabv3plus_config.py

# CPA-Enhancer + segformer
python tools/train.py  configs/segformer/acdc_segformer_config.py

• Testing

# CPA-Enhancer + deeplabv3plus
python tools/test.py configs/deeplabv3plus/acdc_deeplabv3plus_config.py cpa/pretrained_models/deeplabv3plus.epoch

# CPA-Enhancer + segformer
python tools/test.py  configs/segformer/acdc_segformer_config.py cpa/pretrained_models/segformer.epoch

• Inference

python demo/image_demo.py \
	--img path/to/testimg.png  # path to your input image
	--config path/to/configfile # Eg. ..configs/segformer/acdc_segformer_config.py 
	--weights path/to/pretrained_models/xx.pth 

You can download our pretrained models from this link. (pwd: m1r1)

📊 Results

We cascade our proposed CPA-Enhancer with two basic segmentation models, DeepLabv3+ and Segformer, and labeled them as Ours(D) and Ours(S) respectively.

Quantitative results

Quantitative comparisons on the ACDC test set.

Visual Results

Qualitative comparisons of semantic segmentation on the ACDC validation set. Zoom in on the colored annotation boxes to better observe the differences.

💐 Acknowledgments

Special thanks to the creators of mmsegmentation upon which this code is built, for their valuable work in advancing object detection research.

🔗 Citation

If you use this codebase, or CPA-Enhancer inspires your work, we would greatly appreciate it if you could star the repository and cite it using the following BibTeX entry.

@misc{zhang2024cpaenhancer,
      title={CPA-Enhancer: Chain-of-Thought Prompted Adaptive Enhancer for Object Detection under Unknown Degradations}, 
      author={Yuwei Zhang and Yan Wu and Yanming Liu and Xinyue Peng},
      year={2024},
      eprint={2403.11220},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}