Spatio-Temporal Graph Diffusion for Text-Driven Human Motion Generation

Chang Liu (University of Trento),* Mengyi Zhao (Beihang University), Bin Ren (University of Trento), Mengyuan Liu (Peking University, Shenzhen Graduate School), Nicu Sebe (University of Trento)
The 34th British Machine Vision Conference


Text-based human motion generation is challenging due to the complexity and context-dependency of natural human motions. In recent years, an increasing number of studies have focused on using transformer-based diffusion models to tackle this issue. However, an over-reliance on transformers has resulted in a lack of adequate detail in the generated motions. This study proposes a novel graph network-based diffusion model to address this challenging problem. Specifically, we use spatio-temporal graphs to capture local details for each node and an auxiliary transformer to aggregate the information across all nodes. In addition, the transformer is also used to process conditional global information that is difficult to handle with graph networks. Our model achieves competitive results on currently the largest dataset HumanML3D and outperforms existing diffusion models in terms of FID and diversity, demonstrating the advantages of graph neural networks in modeling human motion data. Supplementary materials can be found



author    = {Chang Liu and Mengyi Zhao and Bin Ren and Mengyuan Liu and Nicu Sebe},
title     = {Spatio-Temporal Graph Diffusion for Text-Driven Human Motion Generation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year      = {2023},
url       = {}

Copyright © 2023 The British Machine Vision Association and Society for Pattern Recognition
The British Machine Vision Conference is organised by The British Machine Vision Association and Society for Pattern Recognition. The Association is a Company limited by guarantee, No.2543446, and a non-profit-making body, registered in England and Wales as Charity No.1002307 (Registered Office: Dept. of Computer Science, Durham University, South Road, Durham, DH1 3LE, UK).

Imprint | Data Protection