Overview of the CLEF-2023 LongEval Lab on Longitudinal Evaluation of Model Performance

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A10492534" target="_blank" >RIV/00216208:11320/23:10492534 - isvavai.cz</a>

Result on the web

<a href="https://link.springer.com/chapter/10.1007/978-3-031-42448-9_28" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-031-42448-9_28</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-42448-9_28" target="_blank" >10.1007/978-3-031-42448-9_28</a>

Result language

angličtina

Original language name

Overview of the CLEF-2023 LongEval Lab on Longitudinal Evaluation of Model Performance

Original language description

We describe the first edition of the LongEval CLEF 2023 shared task. This lab evaluates the temporal persistence of Information Retrieval (IR) systems and Text Classifiers. Task 1 requires IR systems to run on corpora acquired at several timestamps, and evaluates the drop in system quality (NDCG) along these timestamps. Task 2 tackles binary sentiment classification at different points in time, and evaluates the performance drop for different temporal gaps. Overall, 37 teams registered for Task 1 and 25 for Task 2. Ultimately, 14 and 4 teams participated in Task 1 and Task 2, respectively.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/LM2018101" target="_blank" >LM2018101: Digital Research Infrastructure for the Language Technologies, Arts and Humanities</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Experimental IR Meets Multilinguality, Multimodality, and Interaction 14th International Conference of the CLEF Association (CLEF 2023)

ISBN

978-3-031-42447-2

ISSN

0302-9743

e-ISSN

—

Number of pages

19

Pages from-to

440-458

Publisher name

Springer

Place of publication

Berlin, Germany

Event location

Thessaloniki, Greece

Event date

Sep 18, 2023

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—