Are Large Language Models Actually Good at Text Style Transfer?
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F24%3A10492887" target="_blank" >RIV/00216208:11320/24:10492887 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2024.inlg-main.42" target="_blank" >https://aclanthology.org/2024.inlg-main.42</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Are Large Language Models Actually Good at Text Style Transfer?
Original language description
We analyze the performance of large language models (LLMs) on Text Style Transfer (TST), specifically focusing on sentiment transfer and text detoxification across three languages: English, Hindi, and Bengali. Text Style Transfer involves modifying the linguistic style of a text while preserving its core content. We evaluate the capabilities of pre-trained LLMs using zero-shot and few-shot prompting as well as parameter-efficient finetuning on publicly available datasets. Our evaluation using automatic metrics, GPT-4 and human evaluations reveals that while some prompted LLMs perform well in English, their performance in on other languages (Hindi, Bengali) remains average. However, finetuning significantly improves results compared to zero-shot and few-shot prompting, making them comparable to previous state-of-the-art. This underscores the necessity of dedicated datasets and specialized models for effective TST.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
R - Projekt Ramcoveho programu EK
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 17th International Natural Language Generation Conference
ISBN
979-8-89176-122-3
ISSN
—
e-ISSN
—
Number of pages
17
Pages from-to
523-539
Publisher name
Association for Computational Linguistics
Place of publication
Kerrville, TX, USA
Event location
Tokyo, Japan
Event date
Sep 23, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—