Structured Knowledge Distillation Towards Efficient Multi-View 3D Object Detection
Linfeng Zhang (Tsinghua University ),* Yukang Shi (Xi’an Jiaotong University), Ke Wang (UNC Chapel Hill), Zhipeng Zhang (DiDi), Hung-Shuo Tai (Didi Autonomous Drive), Yuan He (KargoBot), Kaisheng Ma (Tsinghua University )
The 34th British Machine Vision Conference
Abstract
Detecting 3D objects from multi-view images is a fundamental problem in 3D computer vision. Recently, significant breakthrough has been made in multi-view 3D detection tasks. However, the unprecedented detection performance of these vision BEV (bird's-eye-view) detection models is accompanied with enormous parameters and computation, which make them unaffordable on edge devices. To address this problem, in this paper, we propose a structured knowledge distillation framework, aiming to improve the efficiency of modern vision-only BEV detection models. The proposed framework mainly includes: (a) spatial-temporal distillation which distills teacher knowledge of information fusion from different timestamps and views, (b) BEV response distillation which distills teacher response to different pillars, and (c) weight-inheriting which solves the problem of inconsistent inputs between students and teacher in modern transformer architectures. Experimental results show that our method leads to an average improvement of 2.16 mAP and 2.27 NDS on the nuScenes benchmark, outperforming multiple baselines by a large margin.Video
Citation
@inproceedings{Zhang_2023_BMVC,
author = {Linfeng Zhang and Yukang Shi and Ke Wang and Zhipeng Zhang and Hung-Shuo Tai and Yuan He and Kaisheng Ma},
title = {Structured Knowledge Distillation Towards Efficient Multi-View 3D Object Detection},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0339.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by