Cross-Task Weakly Supervised Learning From Instructional Videos

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F19%3A00337182" target="_blank" >RIV/68407700:21730/19:00337182 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1109/CVPR.2019.00365" target="_blank" >https://doi.org/10.1109/CVPR.2019.00365</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/CVPR.2019.00365" target="_blank" >10.1109/CVPR.2019.00365</a>

Result language

angličtina

Original language name

Cross-Task Weakly Supervised Learning From Instructional Videos

Original language description

In this paper, we investigate learning visual models for the steps of ordinary tasks using weak supervision via instructional narrations and an ordered list of steps instead of strong supervision via temporal annotations. At the heart of our approach is the observation that weakly supervised learning may be easier if a model shares components while learning different steps: `pour egg' should be trained jointly with other tasks involving `pour' and `egg'. We formalize this in a component model for recognizing steps and a weakly supervised learning framework that can learn this model under temporal constraints from narration and the list of steps. Past data does not permit systematic studying of sharing and so we also gather a new dataset, CrossTask, aimed at assessing cross-task sharing. Our experiments demonstrate that sharing across tasks improves performance, especially when done at the component level and that our component model can parse previously unseen tasks by virtue of its compositionality.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/EF15_003%2F0000468" target="_blank" >EF15_003/0000468: Intelligent Machine Perception</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2019

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

CVPR 2019: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition

ISBN

978-1-7281-3294-5

ISSN

1063-6919

e-ISSN

2575-7075

Number of pages

9

Pages from-to

3532-3540

Publisher name

IEEE

Place of publication

—

Event location

Long Beach

Event date

Jun 15, 2019

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

000529484003070