Propose-and-Complete: Auto-regressive Semantic Group Generation for Personalized Scene Synthesis
Shoulong Zhang (Beihang University), Shuai Li (BeihangUniversity), Xinwei Huang (Beihang University), Wenchong Xu (Beihang University), Aimin Hao (BeihangUniversity), HONG QIN (Stony Brook University)*
The 34th British Machine Vision Conference
Abstract
Our research goal is to build a novel scene synthesis framework enabling the flexible generation of individualized indoor virtual environments. Current deep methods only learn the layout patterns from training scene samples, affording only partial co-occurrence possibilities while ignoring any user intent. In contrast, this paper devises a novel framework by flexibly combining and generating function-oriented semantic object groups while accommodating strong user intent. Conforming to this group-centric design paradigm, we consider different strategies for proposing group-level locations and completing semantic clusters with intra-group relationships. The entire framework hinges upon two technical innovations. First, we design a conditional normalizing flow-based ProposeNet to learn the exact distribution of semantic groups, by which we sample potentially plausible group-level locations constrained by user-desirable room functionalities. Second, we design a conditional graph variational auto-encoder, CompleteNet, to instantiate each semantic group with the user-specific complexity (e.g., graph size). With the complete groups readily available, we then recursively select the most plausible proposals and optimize the final layout subject to a collision-free, accessible room space and an arbitrary floor plan. Comprehensive experiments have confirmed that our new framework can produce personalized and versatile unseen 3D scenes from a more expansive design space than conventional domains delimited by training data.Video
Citation
@inproceedings{Zhang_2023_BMVC,
author = {Shoulong Zhang and Shuai Li and Xinwei Huang and Wenchong Xu and Aimin Hao and HONG QIN},
title = {Propose-and-Complete: Auto-regressive Semantic Group Generation for Personalized Scene Synthesis},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0231.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by