Czech news dataset for semantic textual similarity

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11230%2F25%3A10491328" target="_blank" >RIV/00216208:11230/25:10491328 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/49777513:23520/25:43973860

Výsledek na webu

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=X2vLiEMTgf" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=X2vLiEMTgf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s10579-024-09795-z" target="_blank" >10.1007/s10579-024-09795-z</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Czech news dataset for semantic textual similarity

Popis výsledku v původním jazyce

This paper describes a novel dataset consisting of sentences with two different semantic similarity annotations; with and without surrounding context. The data originate from the journalistic domain in the Czech language. The final dataset contains 138,556 human annotations divided into train and test sets. In total, 485 journalism students participated in the creation process. To increase the reliability of the test set, we compute the final annotations as an average of 9 individual annotation scores. We evaluate the dataset quality by measuring inter and intra-annotator agreements. Besides agreement numbers, we provide detailed statistics of the collected dataset. We conclude our paper with a baseline experiment of building a system for predicting the semantic similarity of sentences. Due to the massive number of training annotations (116,956), the model significantly outperforms an average annotator (0.92 versus 0.86 of Pearson&apos;s correlation coefficient).

Název v anglickém jazyce

Czech news dataset for semantic textual similarity

Popis výsledku anglicky

This paper describes a novel dataset consisting of sentences with two different semantic similarity annotations; with and without surrounding context. The data originate from the journalistic domain in the Czech language. The final dataset contains 138,556 human annotations divided into train and test sets. In total, 485 journalism students participated in the creation process. To increase the reliability of the test set, we compute the final annotations as an average of 9 individual annotation scores. We evaluate the dataset quality by measuring inter and intra-annotator agreements. Besides agreement numbers, we provide detailed statistics of the collected dataset. We conclude our paper with a baseline experiment of building a system for predicting the semantic similarity of sentences. Due to the massive number of training annotations (116,956), the model significantly outperforms an average annotator (0.92 versus 0.86 of Pearson&apos;s correlation coefficient).

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/TL02000288" target="_blank" >TL02000288: Proměna etických aspektů s nástupem žurnalistiky umělé inteligence</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Language Resources and Evaluation

ISSN

1574-020X

e-ISSN

1574-0218

Svazek periodika

59

Číslo periodika v rámci svazku

3

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

18

Strana od-do

1989-2006

Kód UT WoS článku

001371498800001

EID výsledku v databázi Scopus

2-s2.0-85211814751