Evaluating Machine Translation Quality Using Short Segments Annotations

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F15%3A10335393" target="_blank" >RIV/00216208:11320/15:10335393 - isvavai.cz</a>
Result on the web
<a href="http://ufal.mff.cuni.cz/pbml/103/art-machacek-bojar.pdf" target="_blank" >http://ufal.mff.cuni.cz/pbml/103/art-machacek-bojar.pdf</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1515/pralin-2015-0005" target="_blank" >10.1515/pralin-2015-0005</a>

Result language
angličtina
Original language name
Evaluating Machine Translation Quality Using Short Segments Annotations
Original language description
We propose a manual evaluation method for machine translation (MT), in which annotators rank only translations of short segments instead of whole sentences. This results in an easier and more efficient annotation. We have conducted an annotation experiment and evaluated a set of MT systems using this method. The obtained results are very close to the official WMT14 evaluation results. We also use the collected database of annotations to automatically evaluate new, unseen systems and to tune parameters of a statistical machine translation system. The evaluation of unseen systems, however, does not work and we analyze the reasons.
Czech name
—
Czech description
—

Type
J<sub>x</sub> - Unclassified - Peer-reviewed scientific article (Jimp, Jsc and Jost)
CEP classification
AI - Linguistics
OECD FORD branch
—

Project
—
Continuities
S - Specificky vyzkum na vysokych skolach<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year
2015
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)