RGBD-Net: Predicting Color and Depth Images for Novel Views Synthesis

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F21%3A00357172" target="_blank" >RIV/68407700:21230/21:00357172 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/3DV53792.2021.00117" target="_blank" >https://doi.org/10.1109/3DV53792.2021.00117</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/3DV53792.2021.00117" target="_blank" >10.1109/3DV53792.2021.00117</a>

Jazyk výsledku

angličtina

Název v původním jazyce

RGBD-Net: Predicting Color and Depth Images for Novel Views Synthesis

Popis výsledku v původním jazyce

We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network. The former one predicts depth maps of the target views by using adaptive depth scaling, while the latter one leverages the predicted depths and renders spatially and temporally consistent target images. In the experimental evaluation on standard datasets, RGBD-Net not only outperforms the state-of-the-art by a clear margin, but it also generalizes well to new scenes without per-scene optimization. Moreover, we show that RGBD-Net can be optionally trained without depth supervision while still retaining high-quality rendering. Thanks to the depth regression network, RGBD-Net can be also used for creating dense 3D point clouds that are more accurate than those produced by some state-of-the-art multi-view stereo methods.

Název v anglickém jazyce

RGBD-Net: Predicting Color and Depth Images for Novel Views Synthesis

Popis výsledku anglicky

We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network. The former one predicts depth maps of the target views by using adaptive depth scaling, while the latter one leverages the predicted depths and renders spatially and temporally consistent target images. In the experimental evaluation on standard datasets, RGBD-Net not only outperforms the state-of-the-art by a clear margin, but it also generalizes well to new scenes without per-scene optimization. Moreover, we show that RGBD-Net can be optionally trained without depth supervision while still retaining high-quality rendering. Thanks to the depth regression network, RGBD-Net can be also used for creating dense 3D point clouds that are more accurate than those produced by some state-of-the-art multi-view stereo methods.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF16_019%2F0000765" target="_blank" >EF16_019/0000765: Výzkumné centrum informatiky</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2021

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

3DV 2021: Proceedings of the International Conference on 3D Vision

ISBN

978-1-6654-2688-6

ISSN

2378-3826

e-ISSN

2475-7888

Počet stran výsledku

11

Strana od-do

1095-1105

Název nakladatele

IEEE Computer Soc.

Místo vydání

Los Alamitos, CA

Místo konání akce

Virtual

Datum konání akce

1. 12. 2021

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—