Label-guided Real-time Fusion Network forRGB-T Semantic Segmentation
Zengrong Lin (Sun Yat-sen University), Baihong Lin (University of Electronic Science and Technology of China),* Yulan Guo (Sun Yat-sen University)
The 34th British Machine Vision Conference
Abstract
RGB-T semantic segmentation has emerged as a promising solution to handle hard scenes with poor lighting conditions by fusing a pair of RGB and thermal images. Although various deep-learning-based fusion networks have been proposed with excellent performance, most of them are not suitable for real-time applications due to high computational overhead and latency. To realize high-accuracy RGB-T real-time semantic segmentation, this paper proposes a novel Label-guided Real-time Fusion Network which fuses multi-level features of RGB and thermal images extracted from double two-pathway lightweight backbones based on the proposed Label-guided Fusion Module (LFM). The proposed LFM realizes efficient multi-modal feature fusion by spatial weighted summation, in which a spatial attention map is generated with the guidance of semantic label in the training phase to accurately indicate the contributions of different modalities. Exhaustive experiments on the MFNet and PST900 datasets demonstrate that the proposed method simultaneously achieves higher speed and accuracy compared with other state-of-the-art methods.Video
Citation
@inproceedings{Lin_2023_BMVC,
author = {Zengrong Lin and Baihong Lin and Yulan Guo},
title = {Label-guided Real-time Fusion Network for RGB-T Semantic Segmentation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0767.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by