The IWSLT 2021 BUT Speech Translation Systems

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F21%3APU144029" target="_blank" >RIV/00216305:26230/21:PU144029 - isvavai.cz</a>

Výsledek na webu

<a href="https://aclanthology.org/2021.iwslt-1.7.pdf" target="_blank" >https://aclanthology.org/2021.iwslt-1.7.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.18653/v1/2021.iwslt-1.7" target="_blank" >10.18653/v1/2021.iwslt-1.7</a>

Jazyk výsledku

angličtina

Název v původním jazyce

The IWSLT 2021 BUT Speech Translation Systems

Popis výsledku v původním jazyce

The paper describes BUTs English to German offline speech translation (ST) systems developed for IWSLT2021. They are based on jointly trained Automatic Speech Recognition- Machine Translation models. Their performances is evaluated on MustC-Common test set. In this work, we study their efficiency from the perspective of having a large amount of separate ASR training data and MT training data, and a smaller amount of speechtranslation training data. Large amounts of ASR and MT training data are utilized for pretraining the ASR and MT models. Speechtranslation data is used to jointly optimize ASR-MT models by defining an end-to-end differentiable path from speech to translations. For this purpose, we use the internal continuous representations from the ASR-decoder as the input to MT module. We show that speech translation can be further improved by training the ASR-decoder jointly with the MT-module using large amount of text-only MT training data. We also show significant improvements by training an ASR module capable of generating punctuated text, rather than leaving the punctuation task to the MT module.

Název v anglickém jazyce

The IWSLT 2021 BUT Speech Translation Systems

Popis výsledku anglicky

The paper describes BUTs English to German offline speech translation (ST) systems developed for IWSLT2021. They are based on jointly trained Automatic Speech Recognition- Machine Translation models. Their performances is evaluated on MustC-Common test set. In this work, we study their efficiency from the perspective of having a large amount of separate ASR training data and MT training data, and a smaller amount of speechtranslation training data. Large amounts of ASR and MT training data are utilized for pretraining the ASR and MT models. Speechtranslation data is used to jointly optimize ASR-MT models by defining an end-to-end differentiable path from speech to translations. For this purpose, we use the internal continuous representations from the ASR-decoder as the input to MT module. We show that speech translation can be further improved by training the ASR-decoder jointly with the MT-module using large amount of text-only MT training data. We also show significant improvements by training an ASR module capable of generating punctuated text, rather than leaving the punctuation task to the MT module.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GX19-26934X" target="_blank" >GX19-26934X: Neuronové reprezentace v multimodálním a mnohojazyčném modelování</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2021

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of 18th International Conference on Spoken Language Translation (IWSLT)

ISBN

978-1-7138-3378-9

ISSN

—

e-ISSN

—

Počet stran výsledku

9

Strana od-do

75-83

Název nakladatele

Association for Computational Linguistics

Místo vydání

Bangkok, on-line

Místo konání akce

Bangkok (on-line)

Datum konání akce

5. 8. 2021

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000694723100007