Cross-domain Semantic Decoupling for Weakly-Supervised Semantic Segmentation

Zaiquan Yang (Beihang University),* Zhanghan Ke (City University of Hong Kong), Gerhard P. Hancke (City University of Hong Kong), Rynson W.H. Lau (City University of Hong Kong)
The 34th British Machine Vision Conference


Weakly-supervised semantic segmentation (WSSS) aims to obtain pixel-wise pseudo labels from image-level labels for segmentation supervision. However, due to the co-occurrence of multiple categories in an image, it is difficult to obtain accurate pseudo labels for supervision, leading to the unsatisfactory performances of current methods. In this paper, we observe that accurate pseudo labels are easier to obtain from images with only a single semantic object (i.e., single-label images) compared to those with multiple semantic objects (i.e., multi-label images). This inspires us to treat the localization maps from single-label images (referred to as the source domain) as good prior knowledge and transfer to multi-label images (referred to as the target domain). Specifically, we present a cross-domain semantic decoupling (CSD) method that first splits image data into source and target domains, and then utilizes the co-occurrence oriented copy-and-paste scheme to enforce pixel-wise consistency and regularize the network responses to the same objects in the two domains. Such a design reduces semantic ambiguity and generates more accurate class boundaries for the pseudo labels. Our method can be seamlessly incorporated into existing WSSS models. Extensive experiments on PASCAL-VOC 2012 demonstrate that the proposed CSD can significantly improve the quality of pseudo labels and final segmentation results.



author    = {Zaiquan Yang and Zhanghan Ke and Gerhard P. Hancke and Rynson W.H. Lau},
title     = {Cross-domain Semantic Decoupling for Weakly-Supervised Semantic Segmentation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection