Improving Personalized Search with Regularized Low-Rank Parameter Updates
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F25%3A00388538" target="_blank" >RIV/68407700:21730/25:00388538 - isvavai.cz</a>
Result on the web
<a href="https://doi.org/10.1109/CVPR52734.2025.01839" target="_blank" >https://doi.org/10.1109/CVPR52734.2025.01839</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1109/CVPR52734.2025.01839" target="_blank" >10.1109/CVPR52734.2025.01839</a>
Alternative languages
Result language
angličtina
Original language name
Improving Personalized Search with Regularized Low-Rank Parameter Updates
Original language description
Personalized vision-language retrieval seeks to recognize new concepts (e.g., “my dog Fido”) from only a few examples. This task is challenging because it requires not only learning a new concept from a few images, but also integrating the personal and general knowledge together to recognize the concept in different contexts. In this paper, we show how to effectively adapt the internal representation of a vision-language dual encoder model for personalized vision-language retrieval. We find that regularized low-rank adaption of a small set of parameters in the language encoder’s final layer serves as a highly effective alternative to textual inversion for recognizing the personal concept while preserving general knowledge. Additionally, we explore strategies for combining parameters of multiple learned personal concepts, finding that parameter addition is effective. To evaluate how well general knowledge is preserved in a finetuned representation, we introduce a metric that measures image retrieval accuracy based on captions generated by a vision language model (VLM). Our approach achieves state-of-the-art accuracy on two benchmarks for personalized image retrieval with natural language queries – DeepFashion2 and ConCon-Chi – outperforming the prior art by 4% - 22% on personal retrievals.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
N - Vyzkumna aktivita podporovana z neverejnych zdroju
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
ISBN
979-8-3315-4364-8
ISSN
1063-6919
e-ISSN
2575-7075
Number of pages
10
Pages from-to
19748-19757
Publisher name
IEEE Computer Society
Place of publication
Los Alamitos
Event location
Nashville
Event date
Jun 11, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
001601158200161