Camera Orientation Estimation in Natural Scenes Using Semantic Cues

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F18%3APU130783" target="_blank" >RIV/00216305:26230/18:PU130783 - isvavai.cz</a>

Výsledek na webu

<a href="http://www.fit.vutbr.cz/research/pubs/all.php?id=11829" target="_blank" >http://www.fit.vutbr.cz/research/pubs/all.php?id=11829</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/3DV.2018.00033" target="_blank" >10.1109/3DV.2018.00033</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Camera Orientation Estimation in Natural Scenes Using Semantic Cues

Popis výsledku v původním jazyce

Camera orientation estimation in natural scenes has recently been approached by several methods, which rely mainly on matching a single modality - edges or horizon lines with 3D digital elevation models. In contrast to previous works, our new image to model matching scheme is based on a fusion of multiple modalities and is designed to be naturally extensible with different cues. In this paper, we use semantic segments and edges. To our knowledge, we are the first to consider using semantic segments jointly with edges for alignment with digital elevation model. We show that high-level features, such as semantic segments, complement the low-level edge information and together help to estimate the camera orientation more robustly compared to methods relying solely on edges or horizon lines. In a series of experiments, we show that segment boundaries tend to be imprecise and important information for matching is encoded in the segment area and a coarse shape. Intuitively, semantic segments encode low frequency information as opposed to edges, which encode high frequencies. Our experiments exhibit that semantic segments and edges are complementary, improving camera orientation estimation reliability when used together. We demonstrate that our method combining semantic and edge features is able to reach state-of-the-art performance on three datasets.

Název v anglickém jazyce

Camera Orientation Estimation in Natural Scenes Using Semantic Cues

Popis výsledku anglicky

Camera orientation estimation in natural scenes has recently been approached by several methods, which rely mainly on matching a single modality - edges or horizon lines with 3D digital elevation models. In contrast to previous works, our new image to model matching scheme is based on a fusion of multiple modalities and is designed to be naturally extensible with different cues. In this paper, we use semantic segments and edges. To our knowledge, we are the first to consider using semantic segments jointly with edges for alignment with digital elevation model. We show that high-level features, such as semantic segments, complement the low-level edge information and together help to estimate the camera orientation more robustly compared to methods relying solely on edges or horizon lines. In a series of experiments, we show that segment boundaries tend to be imprecise and important information for matching is encoded in the segment area and a coarse shape. Intuitively, semantic segments encode low frequency information as opposed to edges, which encode high frequencies. Our experiments exhibit that semantic segments and edges are complementary, improving camera orientation estimation reliability when used together. We demonstrate that our method combining semantic and edge features is able to reach state-of-the-art performance on three datasets.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/TE01020415" target="_blank" >TE01020415: Centrum kompetence ve zpracování vizuálních informací (V3C - Visual Computing Competence Center)</a><br>

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2018

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2018 International Conference on 3D Vision

ISBN

978-1-5386-2610-8

ISSN

—

e-ISSN

—

Počet stran výsledku

10

Strana od-do

208-217

Název nakladatele

IEEE Computer Society

Místo vydání

Verona

Místo konání akce

Verona

Datum konání akce

5. 9. 2018

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000449774200022