Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F24%3APU154969" target="_blank" >RIV/00216305:26230/24:PU154969 - isvavai.cz</a>

Result on the web

<a href="https://www.isca-archive.org/interspeech_2024/yusuf24_interspeech.pdf" target="_blank" >https://www.isca-archive.org/interspeech_2024/yusuf24_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2024-298" target="_blank" >10.21437/Interspeech.2024-298</a>

Result language

angličtina

Original language name

Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models

Original language description

This paper explores speculative speech recognition (SSR), where we empower conventional automatic speech recognition (ASR) with speculation capabilities, allowing the recognizer to run ahead of audio. We introduce a metric for measuring SSR performance and we propose a model which does SSR by com bining a RNN-Transducer-based ASR system with an audioprefixed language model (LM). The ASR system transcribes ongoing audio and feeds the resulting transcripts, along with an audiodependent prefix, to the LM, which speculates likely completions for the transcriptions. We experiment with a variety of ASR datasets on which show the efficacy our method and the feasibility of SSR as a method of reducing ASR latency.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of Interspeech 2024

ISBN

—

ISSN

1990-9772

e-ISSN

—

Number of pages

5

Pages from-to

792-796

Publisher name

International Speech Communication Association

Place of publication

Kos

Event location

Kos

Event date

Sep 1, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—