Large Language Models as a Normalizer for Transliteration and Dialectal Translation

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3ACJZ24EBH" target="_blank" >RIV/00216208:11320/26:CJZ24EBH - isvavai.cz</a>

Výsledek na webu

—

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Large Language Models as a Normalizer for Transliteration and Dialectal Translation

Popis výsledku v původním jazyce

NLP models trained on standardized language data often struggle with non-standard variations. We assess various Large Language Models (LLMs) for transliteration and dialectal normalization. Tuning open-source LLMs with as little as 10,000 parallel examples using LoRA can achieve results comparable to or better than closed-source LLMs. We perform dialectal normalization experiments for twelve South Asian languages and dialectal translation experiments for six language continua worldwide. The dialectal normalization task can also be a preliminary step for the downstream dialectal translation task. Among the six languages used in dialectal translation, our approach enables Italian and Swiss German to surpass the baseline model by 21.5 and 25.8 BLEU points, respectively. © 2025 Association for Computational Linguistics.

Název v anglickém jazyce

Large Language Models as a Normalizer for Transliteration and Dialectal Translation

Popis výsledku anglicky

NLP models trained on standardized language data often struggle with non-standard variations. We assess various Large Language Models (LLMs) for transliteration and dialectal normalization. Tuning open-source LLMs with as little as 10,000 parallel examples using LoRA can achieve results comparable to or better than closed-source LLMs. We perform dialectal normalization experiments for twelve South Asian languages and dialectal translation experiments for six language continua worldwide. The dialectal normalization task can also be a preliminary step for the downstream dialectal translation task. Among the six languages used in dialectal translation, our approach enables Italian and Swiss German to surpass the baseline model by 21.5 and 25.8 BLEU points, respectively. © 2025 Association for Computational Linguistics.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

VarDial - Workshop NLP Similar Lang., Var. Dialects, Proc. Workshop

ISBN

979-8-89176-208-4

ISSN

—

e-ISSN

—

Počet stran výsledku

29

Strana od-do

39-67

Název nakladatele

Association for Computational Linguistics (ACL)

Místo vydání

—

Místo konání akce

Abu Dhabi

Datum konání akce

1. 1. 2026

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—