X-PDNet: Accurate Joint Plane Instance Segmentation and Monocular Depth Estimation with Cross-Task Distillation and Boundary Correction
Duc Cao Dinh (Computer Vision Lab, Hanyang University),* Jongwoo Lim (Seoul National University)
The 34th British Machine Vision Conference
Abstract
Segmentation of planar regions from a single RGB image is a particularly important task in the perception of complex scenes. To utilize both visual and geometric properties in images, recent approaches often formulate the problem as a joint estimation of plane instances and dense depth through feature fusion mechanisms and geometric constraint losses. Despite promising results, these methods do not consider cross-task feature distillation and perform poorly at boundary regions. To overcome these limitations, we propose X-PDNet, a framework for the multi-task learning of plane instance segmentation and depth estimation with improvements in the following two aspects. Firstly, we construct the cross-task attention design which promotes early information sharing between multiple tasks for specific task improvements. Secondly, we highlight the current limitations of using the ground truth boundary to develop boundary regression loss, and propose a method that exploits depth information to support precise boundary region segmentation. Finally, we manually annotate more than 3000 images from Stanford 2D-3D-Semantics dataset and make available for evaluation of plane instance segmentation. Through the experiments, our proposed method proves the advantages, outperforming the baseline with large improvement margins in the quantitative results on the ScanNet and the Stanford 2D-3D-S dataset, demonstrating the effectiveness of our proposals.Video
Citation
@inproceedings{Dinh_2023_BMVC,
author = {Duc Cao Dinh and Jongwoo Lim},
title = {X-PDNet: Accurate Joint Plane Instance Segmentation and Monocular Depth Estimation with Cross-Task Distillation and Boundary Correction},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0326.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by