Multimodal Abstractive Summarization for How2 Videos

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F19%3A10405592" target="_blank" >RIV/00216208:11320/19:10405592 - isvavai.cz</a>

Result on the web

—

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Multimodal Abstractive Summarization for How2 Videos

Original language description

In this paper, we study abstractive summarization for open-domain videos. Unlike the traditional text news summarization, the goal is less to &quot;compress&quot; text information but rather to provide a fluent textual summary of information that has been collected and fused from different source modalities, in our case video and audio transcripts (or text). We show how a multi-source sequence-to-sequence model with hierarchical attention can integrate information from different modalities into a coherent output, compare various models trained with different modalities and present pilot experiments on the How2 corpus of instructional videos. We also propose a new evaluation metric (Content F1) for abstractive summarization task that measures semantic adequacy rather than fluency of the summaries, which is covered by metrics like ROUGE and BLEU.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2019

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics

ISBN

978-1-950737-48-2

ISSN

—

e-ISSN

—

Number of pages

10

Pages from-to

6587-6596

Publisher name

Association for Computational Linguistics

Place of publication

Stroudsburg, PA, USA

Event location

Firenze, Italy

Event date

Jul 28, 2019

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—