Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10511598" target="_blank" >RIV/00216208:11320/25:10511598 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007/978-3-032-04354-2_14#citeas" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-032-04354-2_14#citeas</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-032-04354-2_14" target="_blank" >10.1007/978-3-032-04354-2_14</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality

Popis výsledku v původním jazyce

ELOQUENT is a CLEF lab for evaluating generative language model quality with a focus on such aspects of quality that do not come to the fore with current standard test suites and test collections and to develop and promote new test regimes and methods that fit a multilingual application scenario for generative artificial intelligence. This year is the second year of ELOQUENT. This year&apos;s experiment tracks have evolved from the first year: this year we continue challenging the capability of classifiers to distinguish machine-generated from human-authored text; we explore how consistent language models are in responding to value-oriented questions across languages and system settings; we test how accurately language models are able to predict human preferences between variants of generated material; and we investigate how well language models are able provide sensible topical quizzes to fit given target texts.

Název v anglickém jazyce

Overview of ELOQUENT 2025: Shared Tasks for Evaluating Generative Language Model Quality

Popis výsledku anglicky

ELOQUENT is a CLEF lab for evaluating generative language model quality with a focus on such aspects of quality that do not come to the fore with current standard test suites and test collections and to develop and promote new test regimes and methods that fit a multilingual application scenario for generative artificial intelligence. This year is the second year of ELOQUENT. This year&apos;s experiment tracks have evolved from the first year: this year we continue challenging the capability of classifiers to distinguish machine-generated from human-authored text; we explore how consistent language models are in responding to value-oriented questions across languages and system settings; we test how accurately language models are able to predict human preferences between variants of generated material; and we investigate how well language models are able provide sensible topical quizzes to fit given target texts.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EH23_020%2F0008518" target="_blank" >EH23_020/0008518: Jazykověda, umělá inteligence a jazykové a řečové technologie: od výzkumu k aplikacím</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Experimental IR Meets Multilinguality, Multimodality, and Interaction. Proceedings of the Sixteenth International Conference of the CLEF Association (CLEF 2025)

ISBN

978-3-032-04354-2

ISSN

1611-3349

e-ISSN

—

Počet stran výsledku

18

Strana od-do

224-241

Název nakladatele

Springer

Místo vydání

Cham, Switzerland

Místo konání akce

Madrid, Spain

Datum konání akce

9. 9. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—