skLEP: A Slovak General Language Understanding Benchmark
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3AUSPT99V2" target="_blank" >RIV/00216208:11320/26:USPT99V2 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2025.findings-acl.1371/" target="_blank" >https://aclanthology.org/2025.findings-acl.1371/</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.48550/arXiv.2506.21508" target="_blank" >10.48550/arXiv.2506.21508</a>
Alternative languages
Result language
angličtina
Original language name
skLEP: A Slovak General Language Understanding Benchmark
Original language description
In this work, we introduce skLEP, the first comprehensive benchmark specifically designed for evaluating Slovak natural language understanding (NLU) models. We have compiled skLEP to encompass nine diverse tasks that span token-level, sentence-pair, and document-level challenges, thereby offering a thorough assessment of model capabilities. To create this benchmark, we curated new, original datasets tailored for Slovak and meticulously translated established English NLU resources. Within this paper, we also present the first systematic and extensive evaluation of a wide array of Slovak-specific, multilingual, and English pre-trained language models using the skLEP tasks. Finally, we also release the complete benchmark data, an open-source toolkit facilitating both fine-tuning and evaluation of models, and a public leaderboard at https://github.com/slovak-nlp/sklep in the hopes of fostering reproducibility and drive future research in Slovak NLU.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Findings of the Association for Computational Linguistics
ISBN
979-8-89176-256-5
ISSN
—
e-ISSN
—
Number of pages
28
Pages from-to
26716-26743
Publisher name
—
Place of publication
—
Event location
Vienna, Austria
Event date
Jan 1, 2026
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—