Spatio-Temporal Graph Diffusion for Text-Driven Human Motion Generation
Chang Liu (University of Trento),* Mengyi Zhao (Beihang University), Bin Ren (University of Trento), Mengyuan Liu (Peking University, Shenzhen Graduate School), Nicu Sebe (University of Trento)
The 34th British Machine Vision Conference
Abstract
Text-based human motion generation is challenging due to the complexity and context-dependency of natural human motions. In recent years, an increasing number of studies have focused on using transformer-based diffusion models to tackle this issue. However, an over-reliance on transformers has resulted in a lack of adequate detail in the generated motions. This study proposes a novel graph network-based diffusion model to address this challenging problem. Specifically, we use spatio-temporal graphs to capture local details for each node and an auxiliary transformer to aggregate the information across all nodes. In addition, the transformer is also used to process conditional global information that is difficult to handle with graph networks. Our model achieves competitive results on currently the largest dataset HumanML3D and outperforms existing diffusion models in terms of FID and diversity, demonstrating the advantages of graph neural networks in modeling human motion data. Supplementary materials can be found https://stg-md.github.io/.Video
Citation
@inproceedings{Liu_2023_BMVC,
author = {Chang Liu and Mengyi Zhao and Bin Ren and Mengyuan Liu and Nicu Sebe},
title = {Spatio-Temporal Graph Diffusion for Text-Driven Human Motion Generation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0722.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by