Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F21%3A00356152" target="_blank" >RIV/68407700:21730/21:00356152 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/ICCV48922.2021.00186" target="_blank" >https://doi.org/10.1109/ICCV48922.2021.00186</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICCV48922.2021.00186" target="_blank" >10.1109/ICCV48922.2021.00186</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions

Popis výsledku v původním jazyce

We introduce the task of weakly supervised learning for detecting human and object interactions in videos. Our task poses unique challenges as a system does not know what types of human-object interactions are present in a video or the actual spatiotemporal location of the human and the object. To address these challenges, we introduce a contrastive weakly supervised training loss that aims to jointly associate spatiotemporal regions in a video with an action and object vocabulary and encourage temporal continuity of the visual appearance of moving objects as a form of self-supervision. To train our model, we introduce a dataset comprising over 6.5k videos with human-object interaction annotations that have been semi-automatically curated from sentence captions associated with the videos. We demonstrate improved performance over weakly supervised baselines adapted to our task on our video dataset.

Název v anglickém jazyce

Weakly Supervised Human-Object Interaction Detection in Video via Contrastive Spatiotemporal Regions

Popis výsledku anglicky

We introduce the task of weakly supervised learning for detecting human and object interactions in videos. Our task poses unique challenges as a system does not know what types of human-object interactions are present in a video or the actual spatiotemporal location of the human and the object. To address these challenges, we introduce a contrastive weakly supervised training loss that aims to jointly associate spatiotemporal regions in a video with an action and object vocabulary and encourage temporal continuity of the visual appearance of moving objects as a form of self-supervision. To train our model, we introduce a dataset comprising over 6.5k videos with human-object interaction annotations that have been semi-automatically curated from sentence captions associated with the videos. We demonstrate improved performance over weakly supervised baselines adapted to our task on our video dataset.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF15_003%2F0000468" target="_blank" >EF15_003/0000468: Inteligentní strojové vnímání</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2021

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

ICCV2021: Proceedings of the International Conference on Computer Vision

ISBN

978-1-6654-2812-5

ISSN

1550-5499

e-ISSN

2380-7504

Počet stran výsledku

11

Strana od-do

1825-1835

Název nakladatele

IEEE

Místo vydání

Piscataway

Místo konání akce

Montreal

Datum konání akce

11. 10. 2021

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000797698902003