Identification of Relevant and Redundant Automatic Metrics for MT Evaluation
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216275%3A25410%2F16%3A39902477" target="_blank" >RIV/00216275:25410/16:39902477 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-319-49397-8_12" target="_blank" >http://dx.doi.org/10.1007/978-3-319-49397-8_12</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-319-49397-8_12" target="_blank" >10.1007/978-3-319-49397-8_12</a>
Alternative languages
Result language
angličtina
Original language name
Identification of Relevant and Redundant Automatic Metrics for MT Evaluation
Original language description
The paper is aimed at automatic metrics for translation quality assessment (TQA), specifically at machine translation (MT) output and the metrics for the evaluation of MT output (Precision, Recall, F-measure, BLEU, PER, WER and CDER). We examine their reliability and we determine the metrics which show decreasing reliability of the automatic evaluation of MT output. Besides the traditional measures (Cronbach's alpha and standardized alpha) we use entropy for assessing the reliability of the automatic metrics of MT output. The results were obtained on a dataset covering translation from a low resource language (SK) into English (EN). The main contribution consists of the identification of the redundant automatic MT evaluation metrics.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
IN - Informatics
OECD FORD branch
—
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2016
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Lecture Notes in Computer Science
ISBN
978-3-319-49396-1
ISSN
0302-9743
e-ISSN
—
Number of pages
12
Pages from-to
141-152
Publisher name
Springer
Place of publication
Heidelberg
Event location
Chiang Mai
Event date
Dec 7, 2016
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000389332100012