Tuning of language models in Eastern European languages on Twitter/X

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61988987%3A17610%2F25%3AA2603DAC" target="_blank" >RIV/61988987:17610/25:A2603DAC - isvavai.cz</a>

Alternative codes found

RIV/47813059:19240/25:A0001541

Result on the web

<a href="https://ceur-ws.org/Vol-4092/" target="_blank" >https://ceur-ws.org/Vol-4092/</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.5281/ZENODO.17609598" target="_blank" >10.5281/ZENODO.17609598</a>

Result language

angličtina

Original language name

Tuning of language models in Eastern European languages on Twitter/X

Original language description

We address the problem of fine-tuning large language models (LLMs) for sentiment analysis on Twitter/X in underrepresented Eastern European languages (Czech, Slovak, Polish, and Hungarian). We study the influence of a number of experimental settings on the efficiency of fine-tuning in two groups of LLMs: transfer-learning models (BERT, BERTweet or XLM-T, the latter two pre-trained on a Twitter corpus) and popular mid-sized universal models (Llama, Mistral). We show that adapter fine-tuning with as few as ≈ 600 tweets improved scores of our universal models to the level previously reported by Twitter/X-specialised models on popular datasets, while our transfer-learning models performed worse. We also show that, despite previous successful experiments with multilingual models, translating from underrepresented languages into English still improves the results of all models tested. Several other factors that influence the success of fine-tuning are also included in the study.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/EH23_025%2F0008724" target="_blank" >EH23_025/0008724: Biography of Fake News with a Touch of AI: Dangerous Phenomenon through the Prism of Modern Human Sciences</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 25th Conference Information Technologies – Applications and Theory (ITAT 2025)

ISBN

—

ISSN

1613-0073

e-ISSN

—

Number of pages

11

Pages from-to

242-252

Publisher name

Technical University & CreateSpace Independent Publishing

Place of publication

Aachen

Event location

Telgárt

Event date

Sep 26, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—