Towards the use of entropy as a measure for the reliability of automatic MT evaluation metrics
Identifikátory výsledku
Kód výsledku v IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216275%3A25410%2F18%3A39912365" target="_blank" >RIV/00216275:25410/18:39912365 - isvavai.cz</a>
Výsledek na webu
<a href="http://dx.doi.org/10.3233/JIFS-169505" target="_blank" >http://dx.doi.org/10.3233/JIFS-169505</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.3233/JIFS-169505" target="_blank" >10.3233/JIFS-169505</a>
Alternativní jazyky
Jazyk výsledku
angličtina
Název v původním jazyce
Towards the use of entropy as a measure for the reliability of automatic MT evaluation metrics
Popis výsledku v původním jazyce
The study describes an experiment with different estimations of reliability. Reliability reflects the technical quality of the measurement procedure such as an automatic evaluation of Machine Translation (MT). Reliability is an indicator of accuracy, the reliability of measuring, in our case, measuring the accuracy and error rate of MT output based on automatic metrics (precision, recall, f-measure, Bleu-n, WER, PER, and CDER). The experiment showed metrics (Bleu-4 and WER) that reduce the overall reliability of the automatic evaluation of accuracy and error rate using entropy. Based on the results we can say, that the use of entropy for the estimation of reliability brings more accurate results than conventional estimations of reliability (Cronbach's alpha and correlation). MT evaluation, based on n-grams or edit distance, using entropy could offer a new view on lexicon-based metrics in comparison to commonly used ones.
Název v anglickém jazyce
Towards the use of entropy as a measure for the reliability of automatic MT evaluation metrics
Popis výsledku anglicky
The study describes an experiment with different estimations of reliability. Reliability reflects the technical quality of the measurement procedure such as an automatic evaluation of Machine Translation (MT). Reliability is an indicator of accuracy, the reliability of measuring, in our case, measuring the accuracy and error rate of MT output based on automatic metrics (precision, recall, f-measure, Bleu-n, WER, PER, and CDER). The experiment showed metrics (Bleu-4 and WER) that reduce the overall reliability of the automatic evaluation of accuracy and error rate using entropy. Based on the results we can say, that the use of entropy for the estimation of reliability brings more accurate results than conventional estimations of reliability (Cronbach's alpha and correlation). MT evaluation, based on n-grams or edit distance, using entropy could offer a new view on lexicon-based metrics in comparison to commonly used ones.
Klasifikace
Druh
J<sub>imp</sub> - Článek v periodiku v databázi Web of Science
CEP obor
—
OECD FORD obor
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Návaznosti výsledku
Projekt
—
Návaznosti
S - Specificky vyzkum na vysokych skolach
Ostatní
Rok uplatnění
2018
Kód důvěrnosti údajů
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Údaje specifické pro druh výsledku
Název periodika
Journal of Intelligent & Fuzzy Systems
ISSN
1064-1246
e-ISSN
—
Svazek periodika
34
Číslo periodika v rámci svazku
5
Stát vydavatele periodika
NL - Nizozemsko
Počet stran výsledku
9
Strana od-do
3225-3233
Kód UT WoS článku
000433204800036
EID výsledku v databázi Scopus
—