Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch Classification

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00379804" target="_blank" >RIV/68407700:21230/25:00379804 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1007/978-981-96-0972-7_20" target="_blank" >https://doi.org/10.1007/978-981-96-0972-7_20</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-981-96-0972-7_20" target="_blank" >10.1007/978-981-96-0972-7_20</a>

Result language

angličtina

Original language name

Co-segmentation Without any Pixel-Level Supervision with Application to Large-Scale Sketch Classification

Original language description

This work proposes a novel method for object co-segmentation, i.e. pixel-level localization of a common object in a set of images, that uses no pixel-level supervision for training. Two pre-trained Vision Transformer (ViT) models are exploited: ImageNet classification-trained ViT, whose features are used to estimate rough object localization through intra-class token relevance, and a self-supervised DINO-ViT for intra-image token relevance. On recent challenging benchmarks, the method achieves state-of-the-art performance among methods trained with the same level of supervision (image labels) while being competitive with methods trained with pixel-level supervision (binary masks). The benefits of the proposed co-segmentation method are further demonstrated in the task of large-scale sketch recognition, that is, the classification of sketches into a wide range of categories. The limited amount of hand-drawn sketch training data is leveraged by exploiting readily available image-level-annotated datasets of natural images containing a large number of classes. To bridge the domain gap, the classifier is trained on a sketch-like proxy domain derived from edges detected on natural images. We show that sketch recognition significantly benefits when the classifier is trained on sketch-like structures extracted from the co-segmented area rather than from the full image.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Computer Vision – ACCV 2024, Part X

ISBN

978-981-96-0971-0

ISSN

0302-9743

e-ISSN

1611-3349

Number of pages

17

Pages from-to

342-358

Publisher name

Springer Nature Singapore Pte Ltd.

Place of publication

—

Event location

Hanoi

Event date

Dec 8, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001542340100020