Exploring logical consistency and viewport sensitivity in compositional VQA models
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F19%3A00336936" target="_blank" >RIV/68407700:21230/19:00336936 - isvavai.cz</a>
Alternative codes found
RIV/68407700:21730/19:00336936
Result on the web
<a href="https://ieeexplore.ieee.org/abstract/document/8967758" target="_blank" >https://ieeexplore.ieee.org/abstract/document/8967758</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1109/IROS40897.2019.8967758" target="_blank" >10.1109/IROS40897.2019.8967758</a>
Alternative languages
Result language
angličtina
Original language name
Exploring logical consistency and viewport sensitivity in compositional VQA models
Original language description
The most recent architectures for Visual Question Answering (VQA), such as TbD or DDRprog, have already outperformed human-level accuracy on benchmark datasets (e.g. CLEVR). We administered an advanced analysis of their performance based on novel metrics called consistency (sum of all object feature instances in the scene (e.g. shapes) equals the total number of the objects in the scene) and revealed only 56% consistency for the most accurate architecture (TbD). In respect to this finding, we propose a new method of the VQA training, which reaches 98% consistency. Furthermore, testing of the VQA model in real world brings out a problem with precise mimicking of the camera position from the original dataset. We therefore created a virtual environment along with its real-world counterpart with variable camera positions to test the accuracy and consistency from different viewports. Based on these errors, we were able to estimate optimal position of the camera. The proposed method thus allows us to find the optimal camera viewport in the real environment without knowing the geometry and the exact position of the camera in the synthetic training environment.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach
Others
Publication year
2019
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
ISBN
978-1-7281-4004-9
ISSN
2153-0858
e-ISSN
2153-0866
Number of pages
6
Pages from-to
2108-2113
Publisher name
IEEE
Place of publication
Piscataway, NJ
Event location
Macau
Event date
Nov 4, 2019
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000544658401111