Towards the use of entropy as a measure for the reliability of automatic MT evaluation metrics
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216275%3A25410%2F18%3A39912365" target="_blank" >RIV/00216275:25410/18:39912365 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.3233/JIFS-169505" target="_blank" >http://dx.doi.org/10.3233/JIFS-169505</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.3233/JIFS-169505" target="_blank" >10.3233/JIFS-169505</a>
Alternative languages
Result language
angličtina
Original language name
Towards the use of entropy as a measure for the reliability of automatic MT evaluation metrics
Original language description
The study describes an experiment with different estimations of reliability. Reliability reflects the technical quality of the measurement procedure such as an automatic evaluation of Machine Translation (MT). Reliability is an indicator of accuracy, the reliability of measuring, in our case, measuring the accuracy and error rate of MT output based on automatic metrics (precision, recall, f-measure, Bleu-n, WER, PER, and CDER). The experiment showed metrics (Bleu-4 and WER) that reduce the overall reliability of the automatic evaluation of accuracy and error rate using entropy. Based on the results we can say, that the use of entropy for the estimation of reliability brings more accurate results than conventional estimations of reliability (Cronbach's alpha and correlation). MT evaluation, based on n-grams or edit distance, using entropy could offer a new view on lexicon-based metrics in comparison to commonly used ones.
Czech name
—
Czech description
—
Classification
Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2018
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
Journal of Intelligent & Fuzzy Systems
ISSN
1064-1246
e-ISSN
—
Volume of the periodical
34
Issue of the periodical within the volume
5
Country of publishing house
NL - THE KINGDOM OF THE NETHERLANDS
Number of pages
9
Pages from-to
3225-3233
UT code for WoS article
000433204800036
EID of the result in the Scopus database
—