Weakly-supervised Spatially Grounded Concept Learner for Few-Shot Learning
Gaurav Bhatt (The University of British Columbia),* Deepayan Das (IIT-H), Leonid Sigal (University of British Columbia), Vineeth N Balasubramanian (Indian Institute of Technology, Hyderabad)
The 34th British Machine Vision Conference
Abstract
One of the fundamental properties of an intelligent learning system is its ability to decompose a complex problem into smaller reusable concepts and use those concepts to adapt to new tasks. This core construct has inspired several concept-based few-shot learning approaches. However, most existing methods lack explicit semantics or require strong supervision to impose semantic structure over their concept representations. In this work, we propose a weakly-supervised and visually grounded concept learner (VGCoL), which enforces semantic structure over the learned spatial representations. The core of VGCoL is its reusable block that learns semantic concept prototypes and grounds them in an image by associating the cell features (obtained as the output of the convolution over the image) with these concept prototypes using an attention mechanism. To ensure the learned prototypes are semantic and disentangled, we introduce a regularization that aligns these prototypes with weights of the image-level concept/attribute classifiers and induces orthogonality. We illustrate that this hierarchical and semantic representation results in state-of-the-art few-shot classification performance on multiple datasets, resulting in improvements of 3--4\% on CUB, SUN, and AWA2 datasets. Further, we illustrate that we can learn meaningful, interpretable, spatially coherent, and grounded concept representations despite weak class-level concept supervision.Video
Citation
@inproceedings{Bhatt_2023_BMVC,
author = {Gaurav Bhatt and Deepayan Das and Leonid Sigal and Vineeth N Balasubramanian},
title = {Weakly-supervised Spatially Grounded Concept Learner for Few-Shot Learning},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0858.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by