Tuning of language models in Eastern European languages on Twitter/X

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61988987%3A17610%2F25%3AA2603DAC" target="_blank" >RIV/61988987:17610/25:A2603DAC - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/47813059:19240/25:A0001541

Výsledek na webu

<a href="https://ceur-ws.org/Vol-4092/" target="_blank" >https://ceur-ws.org/Vol-4092/</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.5281/ZENODO.17609598" target="_blank" >10.5281/ZENODO.17609598</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Tuning of language models in Eastern European languages on Twitter/X

Popis výsledku v původním jazyce

We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.

Název v anglickém jazyce

Tuning of language models in Eastern European languages on Twitter/X

Popis výsledku anglicky

We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EH23_025%2F0008724" target="_blank" >EH23_025/0008724: Biografie dezinformace s přívlastkem AI: Rizikový fenomén prizmatem moderních věd o člověku</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025)

ISBN

—

ISSN

1613-0073

e-ISSN

—

Počet stran výsledku

11

Strana od-do

242-252

Název nakladatele

Technical University & CreateSpace Independent Publishing

Místo vydání

Aachen

Místo konání akce

Telgárt

Datum konání akce

26. 9. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—