DISRPT: A Multilingual, Multi-domain, Cross-framework Benchmark for Discourse Processing

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AQ2RTBSVE" target="_blank" >RIV/00216208:11320/25:Q2RTBSVE - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195913456&partnerID=40&md5=0d36bd2ee59debe512fb17e9b4954ea0" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85195913456&partnerID=40&md5=0d36bd2ee59debe512fb17e9b4954ea0</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

DISRPT: A Multilingual, Multi-domain, Cross-framework Benchmark for Discourse Processing

Original language description

This paper presents DISRPT, a multilingual, multi-domain, and cross-framework benchmark dataset for discourse processing, covering the tasks of discourse unit segmentation, connective identification, and relation classification. DISRPT includes 13 languages, with data from 24 corpora covering about 4 millions tokens and around 250, 000 discourse relation instances from 4 discourse frameworks: RST, SDRT, PDTB, and Discourse Dependencies. We present an overview of the data, its development across three NLP shared tasks on discourse processing carried out in the past five years, and the latest modifications and added extensions. We also carry out an evaluation of state-of-the-art multilingual systems trained on the data for each task, showing plateau performance on segmentation, but important room for improvement for connective identification and relation classification. The DISRPT benchmark employs a unified format that we make available on GitHub and HuggingFace in order to encourage future work on discourse processing across languages, domains, and frameworks. © 2024 ELRA Language Resource Association: CC BY-NC 4.0.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Jt. Int. Conf. Comput. Linguist., Lang. Resour. Eval., LREC-COLING - Main Conf. Proc.

ISBN

978-249381410-4

ISSN

—

e-ISSN

—

Number of pages

16

Pages from-to

4990-5005

Publisher name

European Language Resources Association (ELRA)

Place of publication

—

Event location

Torino, Italia

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—