PanoMixSwap – Panorama Mixing via Structural Swapping for Indoor Scene Understanding
Yu-Cheng Hsieh (National Tsing Hua University),* Cheng Sun (National Tsing Hua University), Suraj Dengale (National Tsing Hua University), Min Sun (NTHU)
The 34th British Machine Vision Conference
Abstract
The volume and diversity of training data are critical for modern deep learning-based methods. Compared to the massive amount of labeled perspective images, 360◦ panoramic images fall short in both volume and diversity. In this paper, we propose PanoMixSwap, a novel data augmentation technique specifically designed for indoor panoramic images. PanoMixSwap explicitly mixes various background styles, foreground furniture, and room layouts from the existing indoor panorama datasets and generates a diverse set of new panoramic images to enrich the datasets. We first decompose each panoramic image into its constituent parts: background style, foreground furniture, and room layout. Then, we generate an augmented image by mixing these three parts from three different images, such as the foreground furniture from one image, the background style from another image, and the room structure from the third image. Our method yields high diversity since there is a cubical increase in image combinations. We also evaluate the effectiveness of PanoMixSwap on two indoor scene understanding tasks: semantic segmentation and layout estimation. Our experiments demonstrate that state-of-the-art methods trained with PanoMixSwap outperform their original setting on both tasks consistentlyVideo
Citation
@inproceedings{Hsieh_2023_BMVC,
author = {Yu-Cheng Hsieh and Cheng Sun and Suraj Dengale and Min Sun},
title = {PanoMixSwap – Panorama Mixing via Structural Swapping for Indoor Scene Understanding},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0226.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by