Official PyTorch implementation of "Learning Not to Reconstruct Anomalies"

This is the implementation of the paper "Learning Not to Reconstruct Anomalies" (BMVC 2021). Paper || arxiv || Presentation Video

Dependencies

• Python 3.6
• PyTorch = 1.7.0
• Numpy
• Sklearn

Datasets

• USCD Ped2 [dataset]
• CUHK Avenue [dataset]
• ShanghaiTech [dataset]
• CIFAR-100 (for patch based pseudo anomalies)
• ImageNet (for patch based pseudo anomalies)

Download the datasets into dataset folder, like ./dataset/ped2/, ./dataset/avenue/, ./dataset/shanghai/, ./dataset/cifar100/, ./dataset/imagenet/

Training

git clone https://github.com/aseuteurideu/LearningNotToReconstructAnomalies

• Training baseline

python train.py --dataset_type ped2

• Training patch based model

python train.py --dataset_type ped2 --pseudo_anomaly_cifar_inpainting_smoothborder 0.2 --max_size 0.5 --max_move 10

• Training skip frame based model

python train.py --dataset_type ped2 --pseudo_anomaly_jump_inpainting 0.2 --jump 2 3 4 5

Select --dataset_type from ped2, avenue, or shanghai.

For more details, check train.py

Pre-trained models

• Model in Table 1

Model	Dataset	AUC	Weight
Baseline	Ped2	92.49%	[ drive ]
Baseline	Avenue	81.47%	[ drive ]
Baseline	ShanghaiTech	71.28%	[ drive ]
Patch based	Ped2	94.77%	[ drive ]
Patch based	Avenue	84.91%	[ drive ]
Patch based	ShanghaiTech	72.46%	[ drive ]
Skip frame based	Ped2	96.50%	[ drive ]
Skip frame based	Avenue	84.67%	[ drive ]
Skip frame based	ShanghaiTech	75.97%	[ drive ]

• Various patch based models on Ped2 (Fig. 5(c))

Intruder Dataset	Patching Technique	AUC	Weight
CIFAR-100	SmoothMixS	94.77%	[ drive ]
ImageNet	SmoothMixS	93.34%	[ drive ]
ShanghaiTech	SmoothMixS	94.74%	[ drive ]
Ped2	SmoothMixS	94.15%	[ drive ]
CIFAR-100	SmoothMixC	94.22%	[ drive ]
CIFAR-100	CutMix	93.54%	[ drive ]
CIFAR-100	MixUp-patch	94.52%	[ drive ]

Evaluation

• Test the model

python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth

• Test the model and save result image

python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth --img_dir folder_path_to_save_image_results

• Test the model and generate demonstration video frames

python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth --vid_dir folder_path_to_save_video_results

Then compile the frames into video. For example, to compile the first video in ubuntu:

ffmpeg -framerate 10 -i frame_00_%04d.png -c:v libx264 -profile:v high -crf 20 -pix_fmt yuv420p video_00.mp4

Bibtex

@inproceedings{astrid2021learning,
  title={Learning Not to Reconstruct Anomalies},
  author={Astrid, Marcella and Zaheer, Muhammad Zaigham and Lee, Jae-Yeong and Lee, Seung-Ik},
  booktitle={BMVC},
  year={2021}
}

Acknowledgement

The code is built on top of code provided by Park et al. [ github ] and Gong et al. [ github ]