LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00385335" target="_blank" >RIV/68407700:21230/25:00385335 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/CVPR52734.2025.00915" target="_blank" >https://doi.org/10.1109/CVPR52734.2025.00915</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPR52734.2025.00915" target="_blank" >10.1109/CVPR52734.2025.00915</a>

Jazyk výsledku

angličtina

Název v původním jazyce

LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation

Popis výsledku v původním jazyce

We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs). Our approach enhances the initial per-patch predictions of VLMs through label propagation, which jointly optimizes predictions by incorporating patch-to-patch relationships. Since VLMs are primarily optimized for cross-modal alignment and not for intra-modal similarity, we use a Vision Model (VM) that is observed to better capture these relationships. We address resolution limitations inherent to patch-based encoders by applying label propagation at the pixel level as a refinement step, significantly improving segmentation accuracy near class boundaries. Our method, called LPOSS+, performs inference over the entire image, avoiding window-based processing and thereby capturing contextual interactions across the full image. LPOSS+ achieves state-of-the-art performance among training-free methods, across a diverse set of datasets. Code: https://github.com/vladan-stojnic/LPOSS

Název v anglickém jazyce

LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation

Popis výsledku anglicky

We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs). Our approach enhances the initial per-patch predictions of VLMs through label propagation, which jointly optimizes predictions by incorporating patch-to-patch relationships. Since VLMs are primarily optimized for cross-modal alignment and not for intra-modal similarity, we use a Vision Model (VM) that is observed to better capture these relationships. We address resolution limitations inherent to patch-based encoders by applying label propagation at the pixel level as a refinement step, significantly improving segmentation accuracy near class boundaries. Our method, called LPOSS+, performs inference over the entire image, avoiding window-based processing and thereby capturing contextual interactions across the full image. LPOSS+ achieves state-of-the-art performance among training-free methods, across a diverse set of datasets. Code: https://github.com/vladan-stojnic/LPOSS

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GM21-28830M" target="_blank" >GM21-28830M: Učení Univerzální Vizuální Reprezentace s Omezenou Supervizí</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

ISBN

979-8-3315-4364-8

ISSN

1063-6919

e-ISSN

2575-7075

Počet stran výsledku

10

Strana od-do

9794-9803

Název nakladatele

IEEE Computer Society

Místo vydání

Los Alamitos

Místo konání akce

Nashville

Datum konání akce

11. 6. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001601106700332