Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F23%3APU149715" target="_blank" >RIV/00216305:26230/23:PU149715 - isvavai.cz</a>

Result on the web

<a href="https://www.isca-speech.org/archive/pdfs/interspeech_2023/kesiraju23_interspeech.pdf" target="_blank" >https://www.isca-speech.org/archive/pdfs/interspeech_2023/kesiraju23_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2023-2506" target="_blank" >10.21437/Interspeech.2023-2506</a>

Result language

angličtina

Original language name

Strategies for Improving Low Resource Speech to Text Translation Relying on Pre-trained ASR Models

Original language description

This paper presents techniques and findings for improving the performance of low-resource speech to text translation (ST). We conducted experiments on both simulated and reallow resource setups, on language pairs English - Portuguese, and Tamasheq - French respectively. Using the encoder-decoder framework for ST, our results show that a multilingual automatic speech recognition system acts as a good initialization under low-resource scenarios. Furthermore, using the CTC as an additional objective for translation during training and decoding helps to reorder the internal representations and improves the final translation. Through our experiments, we try to identify various factors (initializations, objectives, and hyperparameters) that contribute the most for improvements in lowresource setups. With only 300 hours of pre-training data, our model achieved 7.3 BLEU score on Tamasheq - French data, outperforming prior published works from IWSLT 2022 by 1.6 points.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

Result was created during the realization of more than one project. More information in the Projects tab.

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

—

ISSN

1990-9772

e-ISSN

—

Number of pages

5

Pages from-to

2148-2152

Publisher name

International Speech Communication Association

Place of publication

Dublin

Event location

Dublin

Event date

Aug 20, 2023

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—