Weakly-supervised Spatially Grounded Concept Learner for Few-Shot Learning

Gaurav Bhatt (The University of British Columbia),* Deepayan Das (IIT-H), Leonid Sigal (University of British Columbia), Vineeth N Balasubramanian (Indian Institute of Technology, Hyderabad)
The 34th British Machine Vision Conference


One of the fundamental properties of an intelligent learning system is its ability to decompose a complex problem into smaller reusable concepts and use those concepts to adapt to new tasks. This core construct has inspired several concept-based few-shot learning approaches. However, most existing methods lack explicit semantics or require strong supervision to impose semantic structure over their concept representations. In this work, we propose a weakly-supervised and visually grounded concept learner (VGCoL), which enforces semantic structure over the learned spatial representations. The core of VGCoL is its reusable block that learns semantic concept prototypes and grounds them in an image by associating the cell features (obtained as the output of the convolution over the image) with these concept prototypes using an attention mechanism. To ensure the learned prototypes are semantic and disentangled, we introduce a regularization that aligns these prototypes with weights of the image-level concept/attribute classifiers and induces orthogonality. We illustrate that this hierarchical and semantic representation results in state-of-the-art few-shot classification performance on multiple datasets, resulting in improvements of 3--4\% on CUB, SUN, and AWA2 datasets. Further, we illustrate that we can learn meaningful, interpretable, spatially coherent, and grounded concept representations despite weak class-level concept supervision.



author    = {Gaurav Bhatt and Deepayan Das and Leonid Sigal and Vineeth N Balasubramanian},
title     = {Weakly-supervised Spatially Grounded Concept Learner for Few-Shot Learning},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {https://papers.bmvc2023.org/0858.pdf}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection