Robust and Efficient Edge-guided Pose Estimation with Resolution-conditioned NeRF
Liesbeth Claessens (ETH Zurich),* Fabian Manhardt (Google), Ricardo Martin-Brualla (Google), Roland Siegwart (ETH Zürich, Autonomous Systems Lab), Cesar Cadena Lerma (ETH Zurich), Federico Tombari (Google)
The 34th British Machine Vision Conference
Abstract
Monocular 6D pose estimation attempts to obtain the 3D location and rotation of an object from a single input image. Deep Learning based methods have lead to large improvements in this area, but still require highly realistic 3D CAD models or manual data labeling. Neural Radiance Fields (NeRF) have made significant progress in solving the inverse problem of realistically rendering an object from a novel pose without these requirements. Nevertheless, current methods that try to harness them to extract the object’s pose from a given image using an analysis-by-synthesis approach lack robustness and speed, and require hours of pretraining on the object of interest. We propose a novel pose estimation pipeline that makes NeRF-based analysis-by-synthesis reliable and fast. Our proposal improves the quality of the optimization by 1) changing the representation of the pose to a decoupled and continuous parameterization, 2) increasing the model’s robustness to changes in scale by means of conditioning it on the used resolution, and 3) developing an edge-based sampling strategy that focuses on shooting rays near image regions with a strong learning signal. These improvements, along with our backbone choice, allow us to estimate the pose with more than 5% higher recall and more than 4 times faster than prior work, while reducing the pretraining time from hours to minutes.Video
Citation
@inproceedings{Claessens_2023_BMVC,
author = {Liesbeth Claessens and Fabian Manhardt and Ricardo Martin-Brualla and Roland Siegwart and Cesar Cadena Lerma and Federico Tombari},
title = {Robust and Efficient Edge-guided Pose Estimation with Resolution-conditioned NeRF},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0543.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by