mmPoint: Dense Human Point Cloud Generation from mmWave
Qian Xie (University of Oxford),* Qianyi Deng (University of Oxford), Ta-Ying Cheng (University of Oxford), Peijun Zhao (Massachusetts Institute of Technology), Amir Patel (University of Cape Town), Niki Trigoni (University of Oxford), Andrew Markham (University of Oxford)
The 34th British Machine Vision Conference
Abstract
Millimeter-wave (mmWave) radars have emerged as a promising technology for sensing humans in diverse environments, owing to their ability to easily obtain 3D information in the form of point clouds. However, mmWave point clouds are typically characterized by sparsity and irregularity, which may limit their potential for certain applications. To address this issue, we propose mmPoint, the first model capable of generating dense human point clouds from mmWave radar signals. Specifically, mmPoint takes a single radar frame of a human as input and generates a dense point cloud that accurately reflects the shape of the detected human as output. The proposed model consists of a novel Encoder-Decoder architecture that utilizes a Multi-Modal Encoder (MME) to extract features from both the radar signal and a point cloud template. A Multi-Resolution Decoder (MRD) is then utilized to gradually infer a dense point cloud in a three-step fashion, with a Lift-and-Deform Module (LDM) employed at each step to increase the number of points and deform the point cloud based on the radar feature. Experimental results demonstrate that mmPoint achieves excellent performance on dense point cloud generation from mmWave radar signals, thus offering promising potential for high-precision human-centered intelligent applications.Video
Citation
@inproceedings{Xie_2023_BMVC,
author = {Qian Xie and Qianyi Deng and Ta-Ying Cheng and Peijun Zhao and Amir Patel and Niki Trigoni and Andrew Markham},
title = {mmPoint: Dense Human Point Cloud Generation from mmWave},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0194.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by