Dynamic Sub-region Search In Homogeneous Collections Using CLIP

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10504477" target="_blank" >RIV/00216208:11320/25:10504477 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-032-06069-3_9" target="_blank" >http://dx.doi.org/10.1007/978-3-032-06069-3_9</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-032-06069-3_9" target="_blank" >10.1007/978-3-032-06069-3_9</a>

Result language
angličtina
Original language name
Dynamic Sub-region Search In Homogeneous Collections Using CLIP
Original language description
Querying with text-image-based search engines in highly homogeneous domain-specific image collections is challenging for users, as they often struggle to provide descriptive text queries. For example, in an underwater domain, users can usually characterize entities only with abstract labels, such as corals and fish, which leads to low recall rates. Our work investigates whether recall can be improved by supplementing text queries with position information. Specifically, we explore dynamic image partitioning approaches that divide candidates into semantically meaningful regions of interest. Instead of querying entire images, users can specify regions they recognize. This enables the use of position constraints while preserving the semantic capabilities of multimodal models. We introduce and evaluate strategies for integrating position constraints into semantic search models and compare them against static partitioning approaches. Our evaluation highlights both the potential and the limitations of sub-region-based search methods using dynamic partitioning. Dynamic search models achieve up to double the retrieval performance compared to static partitioning approaches but are highly sensitive to perturbations in the specified query positions.
Czech name
—
Czech description
—

Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project
<a href="/en/project/GA25-16785S" target="_blank" >GA25-16785S: Empowering Multi-Objective Recommender Systems with Large Language Models</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)