Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F19%3A10405618" target="_blank" >RIV/00216208:11320/19:10405618 - isvavai.cz</a>
Result on the web
<a href="https://www.aclweb.org/anthology/W19-8630" target="_blank" >https://www.aclweb.org/anthology/W19-8630</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.18653/v1/W19-8630" target="_blank" >10.18653/v1/W19-8630</a>
Alternative languages
Result language
angličtina
Original language name
Efficiency Metrics for Data-Driven Models: A Text Summarization Case Study
Original language description
Using data-driven models for solving text summarization or similar tasks has become very common in the last years. Yet most of the studies report basic accuracy scores only, and nothing is known about the ability of the proposed models to improve when trained on more data. In this paper, we define and propose three data efficiency metrics: data score efficiency, data time deficiency and overall data efficiency. We also propose a simple scheme that uses those metrics and apply it for a more comprehensive evaluation of popular methods on text summarization and title generation tasks. For the latter task, we process and release a huge collection of 35 million abstract-title pairs from scientific articles. Our results reveal that among the tested models, the Transformer is the most efficient on both tasks.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2019
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 12th International Conference on Natural Language Generation (INLG 2019)
ISBN
978-1-950737-94-9
ISSN
—
e-ISSN
—
Number of pages
11
Pages from-to
229-239
Publisher name
Association for Computational Linguistics
Place of publication
Stroudsubrgh, PA, USA
Event location
Tokyo, Japan
Event date
Oct 29, 2019
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—