Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization
Zhao Wang (The Chinese University of Hong Kong),* Aoxue Li (Noah's Ark Lab), Fengwei Zhou (Huawei Noah's Ark Lab), Zhenguo Li (Huawei Noah's Ark Lab), DOU QI (The Chinese University of Hong Kong)
The 34th British Machine Vision Conference
Abstract
Classical object detectors are incapable of detecting novel class objects that are not encountered before. Regarding this issue, Open-Vocabulary Object Detection (OVOD) is proposed, which aims to detect the objects in the candidate class list. However, current OVOD models are suffering from overfitting on the base classes, heavily relying on the large-scale extra data, and complex training process. To overcome these issues, we propose a novel framework with Meta prompt and Instance Contrastive learning (MIC) schemes. Firstly, we simulate a novel-class-emerging scenario to help the prompt learner that learns class and background prompts generalize to novel classes. Secondly, we design an instance-level contrastive strategy to promote intra-class compactness and inter-class separation, which benefits generalization of the detector to novel class objects. Without using knowledge distillation, ensemble model or extra training data during detector training, our proposed MIC outperforms previous SOTA methods trained with these complex techniques on LVIS. Most importantly, MIC shows great generalization ability on novel classes, e.g., with +4.3% and +1.9% AP improvement compared with previous SOTA on COCO and Objects365, respectively.Video
Citation
@inproceedings{Wang_2023_BMVC,
author = {Zhao Wang and Aoxue Li and Fengwei Zhou and Zhenguo Li and DOU QI},
title = {Open-Vocabulary Object Detection with Meta Prompt Representation and Instance Contrastive Optimization},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0093.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by