Exploring logical consistency and viewport sensitivity in compositional VQA models

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F19%3A00336936" target="_blank" >RIV/68407700:21230/19:00336936 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/68407700:21730/19:00336936

Výsledek na webu

<a href="https://ieeexplore.ieee.org/abstract/document/8967758" target="_blank" >https://ieeexplore.ieee.org/abstract/document/8967758</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IROS40897.2019.8967758" target="_blank" >10.1109/IROS40897.2019.8967758</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Exploring logical consistency and viewport sensitivity in compositional VQA models

Popis výsledku v původním jazyce

The most recent architectures for Visual Question Answering (VQA), such as TbD or DDRprog, have already outperformed human-level accuracy on benchmark datasets (e.g. CLEVR). We administered an advanced analysis of their performance based on novel metrics called consistency (sum of all object feature instances in the scene (e.g. shapes) equals the total number of the objects in the scene) and revealed only 56% consistency for the most accurate architecture (TbD). In respect to this finding, we propose a new method of the VQA training, which reaches 98% consistency. Furthermore, testing of the VQA model in real world brings out a problem with precise mimicking of the camera position from the original dataset. We therefore created a virtual environment along with its real-world counterpart with variable camera positions to test the accuracy and consistency from different viewports. Based on these errors, we were able to estimate optimal position of the camera. The proposed method thus allows us to find the optimal camera viewport in the real environment without knowing the geometry and the exact position of the camera in the synthetic training environment.

Název v anglickém jazyce

Exploring logical consistency and viewport sensitivity in compositional VQA models

Popis výsledku anglicky

The most recent architectures for Visual Question Answering (VQA), such as TbD or DDRprog, have already outperformed human-level accuracy on benchmark datasets (e.g. CLEVR). We administered an advanced analysis of their performance based on novel metrics called consistency (sum of all object feature instances in the scene (e.g. shapes) equals the total number of the objects in the scene) and revealed only 56% consistency for the most accurate architecture (TbD). In respect to this finding, we propose a new method of the VQA training, which reaches 98% consistency. Furthermore, testing of the VQA model in real world brings out a problem with precise mimicking of the camera position from the original dataset. We therefore created a virtual environment along with its real-world counterpart with variable camera positions to test the accuracy and consistency from different viewports. Based on these errors, we were able to estimate optimal position of the camera. The proposed method thus allows us to find the optimal camera viewport in the real environment without knowing the geometry and the exact position of the camera in the synthetic training environment.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

ISBN

978-1-7281-4004-9

ISSN

2153-0858

e-ISSN

2153-0866

Počet stran výsledku

6

Strana od-do

2108-2113

Název nakladatele

IEEE

Místo vydání

Piscataway, NJ

Místo konání akce

Macau

Datum konání akce

4. 11. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000544658401111