Cascaded and Generalizable Neural Radiance Fields for Fast View Synthesis

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F24%3A00371779" target="_blank" >RIV/68407700:21230/24:00371779 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1109/TPAMI.2023.3335311" target="_blank" >https://doi.org/10.1109/TPAMI.2023.3335311</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TPAMI.2023.3335311" target="_blank" >10.1109/TPAMI.2023.3335311</a>

Result language

angličtina

Original language name

Cascaded and Generalizable Neural Radiance Fields for Fast View Synthesis

Original language description

We present CG-NeRF, a cascade and generalizable neural radiance fields method for view synthesis. Recent generalizing view synthesis methods can render high-quality novel views using a set of nearby input views. However, the rendering speed is still slow due to the nature of uniformly-point sampling of neural radiance fields. Existing scene-specific methods can train and render novel views efficiently but can not generalize to unseen data. Our approach addresses the problems of fast and generalizing view synthesis by proposing two novel modules: a coarse radiance fields predictor and a convolutional-based neural renderer. This architecture infers consistent scene geometry based on the implicit neural fields and renders new views efficiently using a single GPU. We first train CG-NeRF on multiple 3D scenes of the DTU dataset, and the network can produce high-quality and accurate novel views on unseen real and synthetic data using only photometric losses. Moreover, our method can leverage a denser set of reference images of a single scene to produce accurate novel views without relying on additional explicit representations and still maintains the high-speed rendering of the pre-trained model. Experimental results show that CG-NeRF outperforms state-of-the-art generalizable neural rendering methods on various synthetic and real datasets.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

IEEE Transactions on Pattern Analysis and Machine Intelligence

ISSN

0162-8828

e-ISSN

1939-3539

Volume of the periodical

46

Issue of the periodical within the volume

5

Country of publishing house

US - UNITED STATES

Number of pages

12

Pages from-to

2758-2769

UT code for WoS article

001196751500047

EID of the result in the Scopus database

2-s2.0-85178037115