Object-Centric Multi-Task Learning for Human Instances

Hyeongseok Son (Samsung Advanced Institute of Technology),* Sangil Jung (Samsung), Solae Lee (Samsung Advanced Institute of Technology), Seongeun Kim (Samsung), Seung-In Park (SAIT), ByungIn Yoo (Samsung Advanced Institute of Technology)
The 34th British Machine Vision Conference


Human is one of the most essential classes in visual recognition tasks such as detection, segmentation, and pose estimation. Despite considerable efforts in addressing these tasks individually, their integration within a multi-task learning framework has been relatively unexplored. In this paper, we explore a compact multi-task network architecture that maximally shares the parameters of the multiple tasks via object-centric learning. To this end, we introduce a novel human-centric query (HCQ) that effectively encodes human instance information, including explicit structural information such as keypoints. Besides, we utilize HCQ in prediction heads of the target tasks directly and also interweave HCQ with the deformable attention in Transformer decoders to exploit a well-learned object-centric representation. Experimental results show that the proposed multi-task network achieves comparable accuracy to state-of-the-art task-specific models in human detection, segmentation, and pose estimation tasks, while it consumes less computational costs. The project page is available at https://hyeongseokson1.github.io/HCQNet



author    = {Hyeongseok Son and Sangil Jung and Solae Lee and Seongeun Kim and Seung-In Park and ByungIn Yoo},
title     = {Object-Centric Multi-Task Learning for Human Instances},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {https://papers.bmvc2023.org/0082.pdf}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection