Dynamic Sub-region Search In Homogeneous Collections Using CLIP

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10504477" target="_blank" >RIV/00216208:11320/25:10504477 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1007/978-3-032-06069-3_9" target="_blank" >http://dx.doi.org/10.1007/978-3-032-06069-3_9</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-032-06069-3_9" target="_blank" >10.1007/978-3-032-06069-3_9</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Dynamic Sub-region Search In Homogeneous Collections Using CLIP

Popis výsledku v původním jazyce

Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions.

Název v anglickém jazyce

Dynamic Sub-region Search In Homogeneous Collections Using CLIP

Popis výsledku anglicky

Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA25-16785S" target="_blank" >GA25-16785S: Využití Large Language Modelů v Multi-Objective Doporučovacích Systémech</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

18th International Conference, SISAP 2025

ISBN

978-3-032-06069-3

ISSN

—

e-ISSN

—

Počet stran výsledku

14

Strana od-do

105-118

Název nakladatele

Springer Nature Switzerland AG

Místo vydání

Cham

Místo konání akce

Reykjavik, Iceland

Datum konání akce

1. 10. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—