Adding Visual Information to Improve Multimodal Machine Translation for Low-Resource Language
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F22%3AZNLLGNT4" target="_blank" >RIV/00216208:11320/22:ZNLLGNT4 - isvavai.cz</a>
Result on the web
<a href="https://www.hindawi.com/journals/mpe/2022/5483535/" target="_blank" >https://www.hindawi.com/journals/mpe/2022/5483535/</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1155/2022/5483535" target="_blank" >10.1155/2022/5483535</a>
Alternative languages
Result language
angličtina
Original language name
Adding Visual Information to Improve Multimodal Machine Translation for Low-Resource Language
Original language description
Machine translation makes it easy for people to communicate across languages. Multimodal machine translation is also one of the important directions of research in machine translation, which uses feature information such as images and audio to assist translation models in obtaining higher quality target languages. However, in the vast majority of current research work has been conducted on the basis of commonly used corpora such as English, French, German, less research has been done on low-resource languages, and this has left the translation of low-resource languages relatively behind. This paper selects the English-Hindi and English-Hausa corpus, researched on low-resource language translation. The different models we use for image feature information extraction are fusion of image features with text information in the text encoding process of translation, using image features to provide additional information, and assisting the translation model for translation. Compared with text-only machine translation, the experimental results show that our method improves 3 BLEU in the English-Hindi dataset and improves 0.47 BLEU in the English-Hausa dataset. In addition, we also analyze the effect of image feature information extracted by different feature extraction models on the translation results. Different models pay different attention to each region of the image, and ResNet model is able to extract more feature information compared to VGG model, which is more effective for translation.
Czech name
—
Czech description
—
Classification
Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2022
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
Mathematical Problems in Engineering
ISSN
1024-123X
e-ISSN
1563-5147
Volume of the periodical
2022
Issue of the periodical within the volume
2022-8-30
Country of publishing house
US - UNITED STATES
Number of pages
9
Pages from-to
1-9
UT code for WoS article
000863227700012
EID of the result in the Scopus database
2-s2.0-85137866371