Open-world Text-specifed Object Counting
Niki Amini-Naieni (University of Oxford),* Kiana Amini-Naieni (University of California, Davis), Tengda Han (University of Oxford), Andrew Zisserman (University of Oxford)
The 34th British Machine Vision Conference
Abstract
Our objective is open-world object counting in images, where the target object class is specified by a text description. To this end, we propose CounTX, a class-agnostic, single-stage model using a transformer decoder counting head on top of pre-trained joint text-image representations. CounTX is able to count the number of instances of any class given only an image and a text description of the target object class, and can be trained end-to-end. In addition to this model, we make the following contributions: (i) we compare the performance of CounTX to prior work on open-world object counting, and show that our approach exceeds the state of the art on all measures on the FSC-147 benchmark for methods that use text to specify the task; (ii) we present and release FSC-147-D, an enhanced version of FSC-147 with text descriptions, so that object classes can be described with more detailed language than their simple class names. FSC-147-D and the code are available at https://www.robots.ox.ac.uk/~vgg/research/countx.Video
Citation
@inproceedings{Amini-Naieni_2023_BMVC,
author = {Niki Amini-Naieni and Kiana Amini-Naieni and Tengda Han and Andrew Zisserman},
title = {Open-world Text-specifed Object Counting},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0510.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by