Improving Out-of-Distribution Detection Performance using Synthetic Outlier Exposure Generated by Visual Foundation Models
Gitaek Kwon (VUNO Inc.), Jaeyoung Kim (VUNO Inc.),* Hong-Jun Choi (VUNO Inc.), Byung-Moo Yoon (Gachon University), Sungchul Choi (Pukyong National University), Kyu-Hwan Jung (Sungkyunkwan University)
The 34th British Machine Vision Conference
Abstract
Real-world deep learning applications often encounter out-of-distribution (OOD) samples that do not belong to the label spaces of the training dataset. Therefore, neural networks should detect OOD samples and refrain from making predictions on the detected ones to help users be less confused about models’ decisions. A rejection network that has learned representations of OOD can be used to detect distribution shifts, but most existing methods require an additional data collection procedure to train the rejection network. In this paper, we propose the Synthetic Harmless outlier Images generator From Training samples (SHIFT), a realistic OOD generator that converts a training image into synthetic OOD samples by using vision foundation models in a zero-shot manner. Specifically, to construct the surrogate OOD image, the SHIFT uses CLIP to erase the regions of the in-distribution (ID) object, and the latent diffusion model replaces the key regions with realistic features considering the marginal background. Therefore, our method can eliminate the need to collect external outlier samples to train a rejection network. We demonstrate the competitiveness of the proposed method on several benchmarks (i.e., CIFAR-10/100 and STL-10), and the code is publicly available at https://github.com/Anears/SHIFT.Video
Citation
@inproceedings{Kwon_2023_BMVC,
author = {Gitaek Kwon and Jaeyoung Kim and Hong-Jun Choi and Byung-Moo Yoon and Sungchul Choi and Kyu-Hwan Jung},
title = {Improving Out-of-Distribution Detection Performance using Synthetic Outlier Exposure Generated by Visual Foundation Models},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0010.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by