Self-Supervised Video Similarity Learning

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F23%3A00370573" target="_blank" >RIV/68407700:21230/23:00370573 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/CVPRW59228.2023.00504" target="_blank" >https://doi.org/10.1109/CVPRW59228.2023.00504</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPRW59228.2023.00504" target="_blank" >10.1109/CVPRW59228.2023.00504</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Self-Supervised Video Similarity Learning

Popis výsledku v původním jazyce

We introduce S2VS, a video similarity learning approach with self-supervision. Self-Supervised Learning (SSL) is typically used to train deep models on a proxy task so as to have strong transferability on target tasks after fine-tuning. Here, in contrast to prior work, SSL is used to perform video similarity learning and address multiple retrieval and detection tasks at once with no use of labeled data. This is achieved by learning via instance-discrimination with task-tailored augmentations and the widely used InfoNCE loss together with an additional loss operating jointly on self-similarity and hard-negative similarity. We benchmark our method on tasks where video relevance is defined with varying granularity, ranging from video copies to videos depicting the same incident or event. We learn a single universal model that achieves state-of-the-art performance on all tasks, surpassing previously proposed methods that use labeled data. The code and pretrained models are publicly available at: https://github.com/gkordo/s2vs

Název v anglickém jazyce

Self-Supervised Video Similarity Learning

Popis výsledku anglicky

We introduce S2VS, a video similarity learning approach with self-supervision. Self-Supervised Learning (SSL) is typically used to train deep models on a proxy task so as to have strong transferability on target tasks after fine-tuning. Here, in contrast to prior work, SSL is used to perform video similarity learning and address multiple retrieval and detection tasks at once with no use of labeled data. This is achieved by learning via instance-discrimination with task-tailored augmentations and the widely used InfoNCE loss together with an additional loss operating jointly on self-similarity and hard-negative similarity. We benchmark our method on tasks where video relevance is defined with varying granularity, ranging from video copies to videos depicting the same incident or event. We learn a single universal model that achieves state-of-the-art performance on all tasks, surpassing previously proposed methods that use labeled data. The code and pretrained models are publicly available at: https://github.com/gkordo/s2vs

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GM21-28830M" target="_blank" >GM21-28830M: Učení Univerzální Vizuální Reprezentace s Omezenou Supervizí</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Whorkshops (CVPRW)

ISBN

979-8-3503-0249-3

ISSN

2160-7508

e-ISSN

2160-7516

Počet stran výsledku

11

Strana od-do

4756-4766

Název nakladatele

IEEE Computer Society

Místo vydání

USA

Místo konání akce

Vancouver

Datum konání akce

18. 6. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—