Usted: Improving ASR with a Unified Speech and Text Encoder-Decoder

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F22%3APU144907" target="_blank" >RIV/00216305:26230/22:PU144907 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9746554" target="_blank" >https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9746554</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICASSP43922.2022.9746554" target="_blank" >10.1109/ICASSP43922.2022.9746554</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Usted: Improving ASR with a Unified Speech and Text Encoder-Decoder

Popis výsledku v původním jazyce

Improving end-to-end speech recognition by incorporating external text data has been a longstanding research topic. There has been a recent focus on training E2E ASR models that get the performance benefits of external text data without incurring the extra cost of evaluating an external language model at inference time. In this work, we propose training ASR model jointly with a set of text-to-text auxiliary tasks with which it shares a decoder and parts of the encoder. When we jointly train ASR and masked language model with the 960-hour Librispeech and Opensubtitles data respectively, we observe WER reductions of 16% and 20% on test-other and test-clean respectively over an ASR-only baseline without any extra cost at inference time, and reductions of 6% and 8% compared to a stronger MUTE-L baseline which trains the decoder with the same text data as our model. We achieve further improvements when we train masked language model on Librispeech data or when we use machine translation as the auxiliary task, without significantly sacrificing performance on the task itself.

Název v anglickém jazyce

Usted: Improving ASR with a Unified Speech and Text Encoder-Decoder

Popis výsledku anglicky

Improving end-to-end speech recognition by incorporating external text data has been a longstanding research topic. There has been a recent focus on training E2E ASR models that get the performance benefits of external text data without incurring the extra cost of evaluating an external language model at inference time. In this work, we propose training ASR model jointly with a set of text-to-text auxiliary tasks with which it shares a decoder and parts of the encoder. When we jointly train ASR and masked language model with the 960-hour Librispeech and Opensubtitles data respectively, we observe WER reductions of 16% and 20% on test-other and test-clean respectively over an ASR-only baseline without any extra cost at inference time, and reductions of 6% and 8% compared to a stronger MUTE-L baseline which trains the decoder with the same text data as our model. We achieve further improvements when we train masked language model on Librispeech data or when we use machine translation as the auxiliary task, without significantly sacrificing performance on the task itself.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

ISBN

978-1-6654-0540-9

ISSN

—

e-ISSN

—

Počet stran výsledku

5

Strana od-do

8297-8301

Název nakladatele

IEEE Signal Processing Society

Místo vydání

Singapore

Místo konání akce

Singapore

Datum konání akce

22. 5. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000864187908121