End-to-End Learning of Visual Representations from Uncurated Instructional Videos

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F20%3A00343640" target="_blank" >RIV/68407700:21730/20:00343640 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/CVPR42600.2020.00990" target="_blank" >https://doi.org/10.1109/CVPR42600.2020.00990</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPR42600.2020.00990" target="_blank" >10.1109/CVPR42600.2020.00990</a>

Jazyk výsledku

angličtina

Název v původním jazyce

End-to-End Learning of Visual Representations from Uncurated Instructional Videos

Popis výsledku v původním jazyce

Annotating videos is cumbersome, expensive and not scalable. Yet, many strong video models still rely on manually annotated data. With the recent introduction of the HowTo100M dataset, narrated videos now offer the possibility of learning video representations without manual supervision. In this work we propose a new learning approach, MIL-NCE, capable of addressing mis- alignments inherent in narrated videos. With this approach we are able to learn strong video representations from scratch, without the need for any manual annotation. We evaluate our representations on a wide range of four downstream tasks over eight datasets: action recognition (HMDB-51, UCF-101, Kinetics-700), text-to- video retrieval (YouCook2, MSR-VTT), action localization (YouTube-8M Segments, CrossTask) and action segmentation (COIN). Our method outperforms all published self-supervised approaches for these tasks as well as several fully supervised baselines.

Název v anglickém jazyce

End-to-End Learning of Visual Representations from Uncurated Instructional Videos

Popis výsledku anglicky

Annotating videos is cumbersome, expensive and not scalable. Yet, many strong video models still rely on manually annotated data. With the recent introduction of the HowTo100M dataset, narrated videos now offer the possibility of learning video representations without manual supervision. In this work we propose a new learning approach, MIL-NCE, capable of addressing mis- alignments inherent in narrated videos. With this approach we are able to learn strong video representations from scratch, without the need for any manual annotation. We evaluate our representations on a wide range of four downstream tasks over eight datasets: action recognition (HMDB-51, UCF-101, Kinetics-700), text-to- video retrieval (YouCook2, MSR-VTT), action localization (YouTube-8M Segments, CrossTask) and action segmentation (COIN). Our method outperforms all published self-supervised approaches for these tasks as well as several fully supervised baselines.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF15_003%2F0000468" target="_blank" >EF15_003/0000468: Inteligentní strojové vnímání</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition

ISBN

978-1-7281-7169-2

ISSN

1063-6919

e-ISSN

2575-7075

Počet stran výsledku

11

Strana od-do

9876-9886

Název nakladatele

IEEE Computer Society

Místo vydání

USA

Místo konání akce

Seattle

Datum konání akce

13. 6. 2020

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—