Across Images and Graphs for Question Answering

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AGCD72W98" target="_blank" >RIV/00216208:11320/25:GCD72W98 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85200447291&doi=10.1109%2fICDE60146.2024.00112&partnerID=40&md5=2c3d78ee352cdb18e861e0fb7c79f868" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85200447291&doi=10.1109%2fICDE60146.2024.00112&partnerID=40&md5=2c3d78ee352cdb18e861e0fb7c79f868</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICDE60146.2024.00112" target="_blank" >10.1109/ICDE60146.2024.00112</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Across Images and Graphs for Question Answering

Popis výsledku v původním jazyce

Cross-source query serves as a proxy for scene understanding to support many web applications such as rec-ommendation systems, e-commerce, and e-learning applications. In this paper, we propose SVQA that semantically combines the knowledge from available images and graphs to answer the complex question. To this end, we design a graph-based method to unify various data sources into one representation. We then develop a complex question parse method that utilizes the structure of languages to transform the query into a query graph. A graph query engine that performs the query graph over the unified data source while optimizing the query process. To evaluate the proposed system, we build a vanilla dataset called MVQA and show that the state-of-the-art (SOTA) VQA models fail to perform our task. The comprehensive evaluations show that the proposed SVQA is able to reason implicit relationships over multiple images and external knowledge to correctly answer a complex query. We hope that our first attempt provides researchers with a fresh taste of multimodal data analysis. © 2024 IEEE.

Název v anglickém jazyce

Across Images and Graphs for Question Answering

Popis výsledku anglicky

Cross-source query serves as a proxy for scene understanding to support many web applications such as rec-ommendation systems, e-commerce, and e-learning applications. In this paper, we propose SVQA that semantically combines the knowledge from available images and graphs to answer the complex question. To this end, we design a graph-based method to unify various data sources into one representation. We then develop a complex question parse method that utilizes the structure of languages to transform the query into a query graph. A graph query engine that performs the query graph over the unified data source while optimizing the query process. To evaluate the proposed system, we build a vanilla dataset called MVQA and show that the state-of-the-art (SOTA) VQA models fail to perform our task. The comprehensive evaluations show that the proposed SVQA is able to reason implicit relationships over multiple images and external knowledge to correctly answer a complex query. We hope that our first attempt provides researchers with a fresh taste of multimodal data analysis. © 2024 IEEE.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proc Int Conf Data Eng

ISBN

979-835031715-2

ISSN

1084-4627

e-ISSN

—

Počet stran výsledku

14

Strana od-do

1366-1379

Název nakladatele

IEEE Computer Society

Místo vydání

—

Místo konání akce

Utrecht

Datum konání akce

1. 1. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—