Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time adaptation (NeurIPS 2024)

by Mingjia Li, Shuang Li, Tongrui Su, Longhui Yuan, Jian Liang, and Wei Li

        

Note

DUSA is a method that:

• Leverages diffusion models for robust performance.
• Draws inspiration from the structure of score functions.
• Incorporates semantic priors to enhance adaptability.
• Operates efficiently with only a single timestep.

It is highly competitive in fully, continual, and potentially other test-time adaptation tasks.

TODO:

• Release code for test-time segmentation

Overview

Capitalizing on the complementary advantages of generative and discriminative models has always been a compelling vision in machine learning, backed by a growing body of research. This work discloses the hidden semantic structure within score-based generative models, unveiling their potential as effective discriminative priors. Inspired by our theoretical findings, we propose DUSA to exploit the structured semantic priors underlying diffusion score to facilitate the test-time adaptation of image classifiers or dense predictors. Notably, DUSA extracts knowledge from a single timestep of denoising diffusion, lifting the curse of Monte Carlo-based likelihood estimation over timesteps.

For more details please refer to our project page or paper.

Reproduce

To replicate the results for classification tasks, please refer to instructions here.

License

This project is released under the Apache 2.0 License.

Portions of the code are based on MMPreTrain, which is also licensed under the Apache 2.0 License.

Portions of the code are from DiT, which is licensed under CC BY-NC 4.0 License.

Please ensure to review the licenses coming with the official implementations of baselines as they may differ.

Citation

@inproceedings{li2024exploring,
  title={Exploring Structured Semantic Priors Underlying Diffusion Score for Test-time Adaptation},
  author={Mingjia Li and Shuang Li and Tongrui Su and Longhui Yuan and Jian Liang and Wei Li},
  booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
  year={2024},
  url={https://openreview.net/forum?id=c7m1HahBNf}
}