GAATME: A Genetic Algorithm for Adversarial Translation Metrics Evaluation
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F24%3A10492867" target="_blank" >RIV/00216208:11320/24:10492867 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2024.lrec-main.668/" target="_blank" >https://aclanthology.org/2024.lrec-main.668/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
GAATME: A Genetic Algorithm for Adversarial Translation Metrics Evaluation
Original language description
Building on a recent method for decoding translation candidates from a Machine Translation (MT) model via a genetic algorithm, we modify it to generate adversarial translations to test and challenge MT evaluation metrics. The produced translations score very well in an arbitrary MT evaluation metric selected beforehand, despite containing serious, deliberately introduced errors. The method can be used to create adversarial test sets to analyze the biases and shortcomings of the metrics. We publish various such test sets for the Czech to English language pair, as well as the code to convert any parallel data into a similar adversarial test set.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
ISBN
978-2-493-81410-4
ISSN
2522-2686
e-ISSN
—
Number of pages
8
Pages from-to
7562-7569
Publisher name
European Language Resources Association
Place of publication
Torino, Italy
Event location
Torino, Italy
Event date
May 22, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—