ManifoldNeRF: View-dependent Image Feature Supervision for Few-shot Neural Radiance Fields
Daiju Kanaoka (Kyushu Institute of Technology),* Motoharu Sonogashira (RIKEN), Hakaru Tamukoh (Kyushu Institute of Technology), Yasutomo Kawanishi (RIKEN)
The 34th British Machine Vision Conference
Abstract
Novel view synthesis has recently made significant progress with the advent of Neural Radiance Fields (NeRF). DietNeRF is an extension of NeRF that aims to achieve this task from only a few images by introducing a new loss function for unknown viewpoints with no input images. The loss function is based on the assumption that the feature vectors obtained from a feature extractor of a pre-trained classification model should be the same even at different viewpoints. However, while that assumption is ideal, in reality, it is known that as viewpoints continuously change, also feature vectors continuously change. Thus, the assumption can harm training. To avoid this harmful training, we propose ManifoldNeRF, a method for supervising feature vectors at unknown viewpoints using interpolated features from neighboring known viewpoints. Since the method provides appropriate supervision for each unknown viewpoint by the interpolated features, the volume representation is learned better than DietNeRF. Experimental results show that the proposed method performs better than other methods in a complex scene. We also experimented with several subsets of viewpoints from a set of viewpoints and identified an effective set of viewpoints for real environments. This provided a basic policy of viewpoint patterns for real-world application.Video
Citation
@inproceedings{Kanaoka_2023_BMVC,
author = {Daiju Kanaoka and Motoharu Sonogashira and Hakaru Tamukoh and Yasutomo Kawanishi},
title = {ManifoldNeRF: View-dependent Image Feature Supervision for Few-shot Neural Radiance Fields},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0682.pdf}
}
Copyright © 2023
The British Machine Vision Conference is organised by