Enhancing Interpretable Object Abstraction via Clustering-based Slot Initialization
Ning Gao (Bosch Center for Artificial Intelligence (BCAI)),* Bernard Hohmann (Karlsruhe Institute of Technology), Gerhard Neumann (Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany)
The 34th British Machine Vision Conference
Abstract
Object-centric representations using slots have shown the advances towards efficient, flexible and interpretable abstraction from low-level perceptual features in a compositional scene. Current approaches randomize the initial state of slots followed by an iterative refinement. As we show in this paper, the random slot initialization significantly affects the accuracy of the final slot prediction. Moreover, current approaches require a predetermined number of slots from prior knowledge of the data, which limits the applicability in the real world. In our work, we initialize the slot representations with clustering algorithms conditioned on the perceptual input features. This requires an additional layer in the architecture to initialize the slots given the identified clusters. We design permutation invariant and permutation equivariant versions of this layer to enable the exchangeable slot representations after clustering. Additionally, we employ mean-shift clustering to automatically identify the number of slots for a given scene. We evaluate our method on object discovery and novel view synthesis tasks with various datasets. The results show that our method outperforms prior works consistently, especially for complex scenes.Video
Citation
@inproceedings{Gao_2023_BMVC,
author = {Ning Gao and Bernard Hohmann and Gerhard Neumann},
title = {Enhancing Interpretable Object Abstraction via Clustering-based Slot Initialization},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0471.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by