II-20: Intelligent and pragmatic analytic categorization of image collections

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F21%3A00344308" target="_blank" >RIV/68407700:21730/21:00344308 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1109/TVCG.2020.3030383" target="_blank" >https://doi.org/10.1109/TVCG.2020.3030383</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TVCG.2020.3030383" target="_blank" >10.1109/TVCG.2020.3030383</a>

Result language

angličtina

Original language name

II-20: Intelligent and pragmatic analytic categorization of image collections

Original language description

In this paper, we introduce II-20 (Image Insight 2020), a multimedia analytics approach for analytic categorization of image collections. Advanced visualizations for image collections exist, but they need tight integration with a machine model to support the task of analytic categorization. Directly employing computer vision and interactive learning techniques gravitates towards search. Analytic categorization, however, is not machine classification (the difference between the two is called the pragmatic gap): a human adds/redefines/deletes categories of relevance on the fly to build insight, whereas the machine classifier is rigid and non-adaptive. Analytic categorization that truly brings the user to insight requires a flexible machine model that allows dynamic sliding on the exploration-search axis, as well as semantic interactions: a human thinks about image data mostly in semantic terms. II-20 brings three major contributions to multimedia analytics on image collections and towards closing the pragmatic gap. Firstly, a new machine model that closely follows the user's interactions and dynamically models her categories of relevance. II-20's machine model, in addition to matching and exceeding the state of the art's ability to produce relevant suggestions, allows the user to dynamically slide on the exploration-search axis without any additional input from her side. Secondly, the dynamic, 1-image-at-a-time Tetris metaphor that synergizes with the model. It allows a well-trained model to analyze the collection by itself with minimal interaction from the user and complements the classic grid metaphor. Thirdly, the fast-forward interaction, allowing the user to harness the model to quickly expand ("fast-forward") the categories of relevance, expands the multimedia analytics semantic interaction dictionary. Automated experiments show that II-20's machine model outperforms the existing state of the art and also demonstrate the Tetris metaphor's analytic quality.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/EF15_003%2F0000470" target="_blank" >EF15_003/0000470: Robotics 4 Industry 4.0</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

C - Předmět řešení projektu podléhá obchodnímu tajemství (§ 504 Občanského zákoníku), ale název projektu, cíle projektu a u ukončeného nebo zastaveného projektu zhodnocení výsledku řešení projektu (údaje P03, P04, P15, P19, P29, PN8) dodané do CEP, jsou upraveny tak, aby byly zveřejnitelné.

Name of the periodical

IEEE Transactions on Visualization and Computer Graphics

ISSN

1077-2626

e-ISSN

1941-0506

Volume of the periodical

27

Issue of the periodical within the volume

2

Country of publishing house

US - UNITED STATES

Number of pages

10

Pages from-to

422-431

UT code for WoS article

000706330100030

EID of the result in the Scopus database

2-s2.0-85100415947