MLASK: Multimodal Summarization of Video-based News Articles
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A10476790" target="_blank" >RIV/00216208:11320/23:10476790 - isvavai.cz</a>
Result on the web
<a href="http://hdl.handle.net/11234/1-5135" target="_blank" >http://hdl.handle.net/11234/1-5135</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
MLASK: Multimodal Summarization of Video-based News Articles
Original language description
In recent years, the pattern of news consumption has been changing. The most popular multimedia news formats are now multimodal - the reader is often presented not only with a textual article but also with a short, vivid video. To draw the attention of the reader, such video-based articles are usually presented as a short textual summary paired with an image thumbnail.In this paper, we introduce MLASK (MultimodaL Article Summarization Kit) - a new dataset of video-based news articles paired with a textual summary and a cover picture, all obtained by automatically crawling several news websites. We demonstrate how the proposed dataset can be used to model the task of multimodal summarization by training a Transformer-based neural model. We also examine the effects of pre-training when the usage of generative pre-trained language models helps to improve the model performance, but (additional) pre-training on the simpler task of text summarization yields even better results. Our experiments suggest that the benefits of pre-training and using additional modalities in the input are not orthogonal.
Czech name
—
Czech description
—
Classification
Type
R - Software
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Internal product ID
https://github.com/ufal/MLASK
Technical parameters
Výsledek je volně dostupný na adrese: http://hdl.handle.net/11234/1-5135
Economical parameters
50.000 Kč
Owner IČO
00216208
Owner name
Univerzita Karlova v Praze