Learned Semantic Multi-Sensor Depth Map Fusion

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F19%3A00338337" target="_blank" >RIV/68407700:21230/19:00338337 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/ICCVW.2019.00264" target="_blank" >https://doi.org/10.1109/ICCVW.2019.00264</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICCVW.2019.00264" target="_blank" >10.1109/ICCVW.2019.00264</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Learned Semantic Multi-Sensor Depth Map Fusion

Popis výsledku v původním jazyce

Volumetric depth map fusion based on truncated signed distance functions has become a standard method and is used in many 3D reconstruction pipelines. In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor: Depth information can originate from different sensors or algorithms with very different noise and outlier statistics which are considered during data fusion. 3) Scene denoising and completion: Sensors can fail to recover depth for certain materials and light conditions, or data is missing due to occlusions. Our method denoises the geometry, closes holes and computes a watertight surface for every semantic class. 4) Learning: We propose a neural network reconstruction method that unifies all these properties within a single powerful framework. Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo algorithms. Our approach is the first to unify all these properties. Experimental evaluations on both synthetic and real data sets demonstrate clear improvements.

Název v anglickém jazyce

Learned Semantic Multi-Sensor Depth Map Fusion

Popis výsledku anglicky

Volumetric depth map fusion based on truncated signed distance functions has become a standard method and is used in many 3D reconstruction pipelines. In this paper, we are generalizing this classic method in multiple ways: 1) Semantics: Semantic information enriches the scene representation and is incorporated into the fusion process. 2) Multi-Sensor: Depth information can originate from different sensors or algorithms with very different noise and outlier statistics which are considered during data fusion. 3) Scene denoising and completion: Sensors can fail to recover depth for certain materials and light conditions, or data is missing due to occlusions. Our method denoises the geometry, closes holes and computes a watertight surface for every semantic class. 4) Learning: We propose a neural network reconstruction method that unifies all these properties within a single powerful framework. Our method learns sensor or algorithm properties jointly with semantic depth fusion and scene completion and can also be used as an expert system, e.g. to unify the strengths of various photometric stereo algorithms. Our approach is the first to unify all these properties. Experimental evaluations on both synthetic and real data sets demonstrate clear improvements.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA18-05360S" target="_blank" >GA18-05360S: Řešení inverzních problémů vznikajících při analýze rychle se pohybujících objektů</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2019 IEEE International Conference on Computer Vision Workshops (ICCVW 2019)

ISBN

—

ISSN

2473-9944

e-ISSN

2473-9944

Počet stran výsledku

11

Strana od-do

2089-2099

Název nakladatele

IEEE Computer Society

Místo vydání

Los Alamitos

Místo konání akce

Seoul

Datum konání akce

27. 10. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—