3D Structure-guided Network for Tooth Alignment in 2D Photograph
Yulong Dou (Shanghaitech),* Lanzhuju Mei (ShanghaiTech University), Dinggang Shen (United Imaging Intelligence), Zhiming Cui (HKU)
The 34th British Machine Vision Conference
Abstract
Orthodontics focuses on rectifying misaligned teeth (i.e., malocclusions), affecting both masticatory function and aesthetics. However, orthodontic treatment often involves complex, lengthy procedures. As such, generating a 2D photograph depicting aligned teeth prior to orthodontic treatment is crucial for effective dentist-patient communication and, more importantly, for encouraging patients to accept orthodontic intervention. In this paper, we propose a 3D structure-guided tooth alignment network that takes 2D photographs as input (e.g., photos captured by smartphones) and aligns the teeth within the 2D image space to generate an orthodontic comparison photograph featuring aesthetically pleasing, aligned teeth. Notably, while the process operates within a 2D image space, our method employs 3D intra-oral scanning models collected in clinics to learn about orthodontic treatment, i.e., projecting the pre- and post-orthodontic 3D tooth structures onto 2D tooth contours, followed by a diffusion model to learn the mapping relationship. Ultimately, the aligned tooth contours are leveraged to guide the generation of a 2D photograph with aesthetically pleasing, aligned teeth and realistic textures. We evaluate our network on various facial photographs, demonstrating its exceptional performance and strong applicability within the orthodontic industry.Video
Citation
@inproceedings{Dou_2023_BMVC,
author = {Yulong Dou and Lanzhuju Mei and Dinggang Shen and Zhiming Cui},
title = {3D Structure-guided Network for Tooth Alignment in 2D Photograph},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0322.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by