Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F23%3APU149715" target="_blank" >RIV/00216305:26230/23:PU149715 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.isca-speech.org/archive/pdfs/interspeech_2023/kesiraju23_interspeech.pdf" target="_blank" >https://www.isca-speech.org/archive/pdfs/interspeech_2023/kesiraju23_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2023-2506" target="_blank" >10.21437/Interspeech.2023-2506</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models

Popis výsledku v původním jazyce

This paper presents techniques and findings for improving the performance of low-resource speech to text translation (ST). We conducted experiments on both simulated and reallow resource setups, on language pairs English - Portuguese, and Tamasheq - French respectively. Using the encoder-decoder framework for ST, our results show that a multilingual automatic speech recognition system acts as a good initialization under low-resource scenarios. Furthermore, using the CTC as an additional objective for translation during training and decoding helps to reorder the internal representations and improves the final translation. Through our experiments, we try to identify various factors (initializations, objectives, and hyperparameters) that contribute the most for improvements in lowresource setups. With only 300 hours of pre-training data, our model achieved 7.3 BLEU score on Tamasheq - French data, outperforming prior published works from IWSLT 2022 by 1.6 points.

Název v anglickém jazyce

Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models

Popis výsledku anglicky

This paper presents techniques and findings for improving the performance of low-resource speech to text translation (ST). We conducted experiments on both simulated and reallow resource setups, on language pairs English - Portuguese, and Tamasheq - French respectively. Using the encoder-decoder framework for ST, our results show that a multilingual automatic speech recognition system acts as a good initialization under low-resource scenarios. Furthermore, using the CTC as an additional objective for translation during training and decoding helps to reorder the internal representations and improves the final translation. Through our experiments, we try to identify various factors (initializations, objectives, and hyperparameters) that contribute the most for improvements in lowresource setups. With only 300 hours of pre-training data, our model achieved 7.3 BLEU score on Tamasheq - French data, outperforming prior published works from IWSLT 2022 by 1.6 points.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

—

ISSN

1990-9772

e-ISSN

—

Počet stran výsledku

5

Strana od-do

2148-2152

Název nakladatele

International Speech Communication Association

Místo vydání

Dublin

Místo konání akce

Dublin

Datum konání akce

20. 8. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—