Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics?
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10511632" target="_blank" >RIV/00216208:11320/25:10511632 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2025.naacl-srw.41/" target="_blank" >https://aclanthology.org/2025.naacl-srw.41/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Evaluating Text Style Transfer Evaluation: Are There Any Reliable Metrics?
Original language description
Text style transfer (TST) is the task of transforming a text to reflect a particular style while preserving its original content. Evaluating TSToutputs is a multidimensional challenge, requiring the assessment of style transfer accuracy, content preservation, and naturalness. Us-ing human evaluation is ideal but costly, as is common in other natural language processing (NLP) tasks; however, automatic metrics forTST have not received as much attention as metrics for, e.g., machine translation or summarization. In this paper, we examine both set ofexisting and novel metrics from broader NLP tasks for TST evaluation, focusing on two popular subtasks—sentiment transfer and detoxification—in a multilingual context comprising English, Hindi, and Bengali. By conducting meta-evaluation through correlation with hu-man judgments, we demonstrate the effectiveness of these metrics when used individually and in ensembles. Additionally, we investigatethe potential of large language models (LLMs) as tools for TST ev
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop)
ISBN
979-8-89176-192-6
ISSN
—
e-ISSN
—
Number of pages
17
Pages from-to
418-434
Publisher name
Association for Computational Linguistics
Place of publication
Kerrville, TX, USA
Event location
Albuquerque, NM, USA
Event date
Apr 30, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—