Jointly Trained Transformers Models for Spoken Language Translation
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F21%3APU142914" target="_blank" >RIV/00216305:26230/21:PU142914 - isvavai.cz</a>
Result on the web
<a href="https://www.fit.vut.cz/research/publication/12522/" target="_blank" >https://www.fit.vut.cz/research/publication/12522/</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1109/ICASSP39728.2021.9414159" target="_blank" >10.1109/ICASSP39728.2021.9414159</a>
Alternative languages
Result language
angličtina
Original language name
Jointly Trained Transformers Models for Spoken Language Translation
Original language description
End-to-End and cascade (ASR-MT) spoken language translation (SLT) systems are reaching comparable performances, however, a large degradation is observed when translating the ASR hypothesis in comparison to using oracle input text. In this work, degradation in performance is reduced by creating an End-to-End differentiable pipeline between the ASR and MT systems. In this work, we train SLT systems with ASR objective as an auxiliary loss and both the networks are connected through the neural hidden representations. This training has an End-to-End differentiable path with respect to the final objective function and utilizes the ASR objective for better optimization. This architecture has improved the BLEU score from 41.21 to 44.69. Ensembling the proposed architecture with independently trained ASR and MT systems further improved the BLEU score from 44.69 to 46.9. All the experiments are reported on English-Portuguese speech translation task using the How2 corpus. The final BLEU score is on-par with the best speech translation system on How2 dataset without using any additional training data and language model and using fewer parameters.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2021
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
ISBN
978-1-7281-7605-5
ISSN
—
e-ISSN
—
Number of pages
5
Pages from-to
7513-7517
Publisher name
IEEE Signal Processing Society
Place of publication
Toronto, Ontario
Event location
Toronto, Canada
Event date
Jun 6, 2021
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000704288407158