GUMsley: Evaluating Entity Salience in Summarization for 12 English Genres

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A3PWYAZBC" target="_blank" >RIV/00216208:11320/25:3PWYAZBC - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189928757&partnerID=40&md5=09c98e0dbcecc09b86d0918f176ebd49" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85189928757&partnerID=40&md5=09c98e0dbcecc09b86d0918f176ebd49</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

GUMsley: Evaluating Entity Salience in Summarization for 12 English Genres

Original language description

As NLP models become increasingly capable of understanding documents in terms of coherent entities rather than strings, obtaining the most salient entities for each document is not only an important end task in itself but also vital for Information Retrieval (IR) and other downstream applications such as controllable summarization. In this paper, we present and evaluate GUMsley, the first entity salience dataset covering all named and non-named salient entities for 12 genres of English text, aligned with entity types, Wikification links and full coreference resolution annotations. We promote a strict definition of salience using human summaries and demonstrate high inter-annotator agreement for salience based on whether a source entity is mentioned in the summary. Our evaluation shows poor performance by pre-trained SOTA summarization models and zero-shot LLM prompting in capturing salient entities in generated summaries. We also show that predicting or providing salient entities to several model architectures enhances performance and helps derive higher-quality summaries by alleviating the entity hallucination problem in existing abstractive summarization. © 2024 Association for Computational Linguistics.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

EACL - Conf. European Chapter Assoc. Comput. Linguist., Proc. Conf.

ISBN

979-889176088-2

ISSN

—

e-ISSN

—

Number of pages

14

Pages from-to

2575-2588

Publisher name

Association for Computational Linguistics (ACL)

Place of publication

—

Event location

St. Julian's

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—