Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch Classification

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00379804" target="_blank" >RIV/68407700:21230/25:00379804 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1007/978-981-96-0972-7_20" target="_blank" >https://doi.org/10.1007/978-981-96-0972-7_20</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-981-96-0972-7_20" target="_blank" >10.1007/978-981-96-0972-7_20</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch Classification

Popis výsledku v původním jazyce

This work proposes a novel method for object co-segmentation, i.e. pixel-level localization of a common object in a set of images, that uses no pixel-level supervision for training. Two pre-trained Vision Transformer (ViT) models are exploited: ImageNet classification-trained ViT, whose features are used to estimate rough object localization through intra-class token relevance, and a self-supervised DINO-ViT for intra-image token relevance. On recent challenging benchmarks, the method achieves state-of-the-art performance among methods trained with the same level of supervision (image labels) while being competitive with methods trained with pixel-level supervision (binary masks). The benefits of the proposed co-segmentation method are further demonstrated in the task of large-scale sketch recognition, that is, the classification of sketches into a wide range of categories. The limited amount of hand-drawn sketch training data is leveraged by exploiting readily available image-level-annotated datasets of natural images containing a large number of classes. To bridge the domain gap, the classifier is trained on a sketch-like proxy domain derived from edges detected on natural images. We show that sketch recognition significantly benefits when the classifier is trained on sketch-like structures extracted from the co-segmented area rather than from the full image.

Název v anglickém jazyce

Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch Classification

Popis výsledku anglicky

This work proposes a novel method for object co-segmentation, i.e. pixel-level localization of a common object in a set of images, that uses no pixel-level supervision for training. Two pre-trained Vision Transformer (ViT) models are exploited: ImageNet classification-trained ViT, whose features are used to estimate rough object localization through intra-class token relevance, and a self-supervised DINO-ViT for intra-image token relevance. On recent challenging benchmarks, the method achieves state-of-the-art performance among methods trained with the same level of supervision (image labels) while being competitive with methods trained with pixel-level supervision (binary masks). The benefits of the proposed co-segmentation method are further demonstrated in the task of large-scale sketch recognition, that is, the classification of sketches into a wide range of categories. The limited amount of hand-drawn sketch training data is leveraged by exploiting readily available image-level-annotated datasets of natural images containing a large number of classes. To bridge the domain gap, the classifier is trained on a sketch-like proxy domain derived from edges detected on natural images. We show that sketch recognition significantly benefits when the classifier is trained on sketch-like structures extracted from the co-segmented area rather than from the full image.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Computer Vision – ACCV 2024, Part X

ISBN

978-981-96-0971-0

ISSN

0302-9743

e-ISSN

1611-3349

Počet stran výsledku

17

Strana od-do

342-358

Název nakladatele

Springer Nature Singapore Pte Ltd.

Místo vydání

—

Místo konání akce

Hanoi

Datum konání akce

8. 12. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001542340100020