Visual Localization using Imperfect 3D Models from the Internet

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F23%3A00367543" target="_blank" >RIV/68407700:21230/23:00367543 - isvavai.cz</a>

Alternative codes found

RIV/68407700:21730/23:00367543

Result on the web

<a href="https://doi.org/10.1109/CVPR52729.2023.01266" target="_blank" >https://doi.org/10.1109/CVPR52729.2023.01266</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPR52729.2023.01266" target="_blank" >10.1109/CVPR52729.2023.01266</a>

Result language

angličtina

Original language name

Visual Localization using Imperfect 3D Models from the Internet

Original language description

Visual localization is a core component in many applications, including augmented reality (AR). Localization algorithms compute the camera pose of a query image w.r.t. a scene representation, which is typically built from images. This often requires capturing and storing large amounts of data, followed by running Structure-from-Motion (SfM) algorithms. An interesting, and underexplored, source of data for building scene representations are 3D models that are readily available on the Internet, e.g., hand-drawn CAD models, 3D models generated from building footprints, or from aerial images. These models allow to perform visual localization right away without the time-consuming scene capturing and model building steps. Yet, it also comes with challenges as the available 3D models are often imperfect reflections of reality. E.g., the models might only have generic or no textures at all, might only provide a simple approximation of the scene geometry, or might be stretched. This paper studies how the imperfections of these models affect localization accuracy. We create a new benchmark for this task and provide a detailed experimental evaluation based on multiple 3D models per scene. We show that 3D models from the Internet show promise as an easy-to-obtain scene representation. At the same time, there is significant room for improvement for visual localization pipelines. To foster research on this interesting and challenging task, we release our benchmark.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

Result was created during the realization of more than one project. More information in the Projects tab.

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

ISBN

979-8-3503-0129-8

ISSN

1063-6919

e-ISSN

2575-7075

Number of pages

12

Pages from-to

13175-13186

Publisher name

IEEE Computer Society

Place of publication

USA

Event location

Vancouver

Event date

Jun 18, 2023

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001062522105047