Laughing Matters: Introducing Audio-Driven Laughing-Face Generation with Diffusion Models
Antoni Bigata Casademunt (Imperial College London),* Rodrigo Mira (Imperial College London), Nikita Drobyshev (Imperial College London), Konstantinos Vougioukas (Imperial College London), Stavros Petridis (Imperial College London), Maja Pantic (Facebook / Imperial College London )
The 34th British Machine Vision Conference
Abstract
Speech-driven animation has gained significant traction in recent years, with current methods achieving near-photorealistic results. However, the field remains underexplored regarding non-verbal communication despite evidence demonstrating its importance in human interaction. In particular, generating laughter sequences presents a unique challenge due to the intricacy and nuances of this behaviour. This paper aims to bridge this gap by proposing a novel model capable of generating realistic laughter sequences, given a still portrait and an audio clip containing laughter. We highlight the failure cases of traditional facial animation methods and leverage recent advances in diffusion models to produce convincing laughter videos. We train our model on a diverse set of laughter datasets and introduce an evaluation metric specifically designed for laughter. When compared with previous speech-driven approaches, our model achieves state-of-the-art performance across all metrics, even when these are re-trained for laughter generation.Video
Citation
@inproceedings{Casademunt_2023_BMVC,
author = {Antoni Bigata Casademunt and Rodrigo Mira and Nikita Drobyshev and Konstantinos Vougioukas and Stavros Petridis and Maja Pantic},
title = {Laughing Matters: Introducing Audio-Driven Laughing-Face Generation with Diffusion Models},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0761.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by