Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F22%3APU146108" target="_blank" >RIV/00216305:26230/22:PU146108 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.isca-speech.org/archive/pdfs/interspeech_2022/kocour22_interspeech.pdf" target="_blank" >https://www.isca-speech.org/archive/pdfs/interspeech_2022/kocour22_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2022-10406" target="_blank" >10.21437/Interspeech.2022-10406</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model

Popis výsledku v původním jazyce

In typical multi-talker speech recognition systems, a neural network-based acoustic model predicts senone state posteriors for each speaker. These are later used by a single-talker decoder which is applied on each speaker-specific output stream separately. In this work, we argue that such a scheme is sub-optimal and propose a principled solution that decodes all speakers jointly. We modify the acoustic model to predict joint state posteriors for all speakers, enabling the network to express uncertainty about the attribution of parts of the speech signal to the speakers. We employ a joint decoder that can make use of this uncertainty together with higher-level language information. For this, we revisit decoding algorithms used in factorial generative models in early multi-talker speech recognition systems. In contrast with these early works, we replace the GMM acoustic model with DNN, which provides greater modeling power and simplifies part of the inference. We demonstrate the advantage of joint decoding in proof of concept experiments on a mixed-TIDIGITS dataset.

Název v anglickém jazyce

Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model

Popis výsledku anglicky

In typical multi-talker speech recognition systems, a neural network-based acoustic model predicts senone state posteriors for each speaker. These are later used by a single-talker decoder which is applied on each speaker-specific output stream separately. In this work, we argue that such a scheme is sub-optimal and propose a principled solution that decodes all speakers jointly. We modify the acoustic model to predict joint state posteriors for all speakers, enabling the network to express uncertainty about the attribution of parts of the speech signal to the speakers. We employ a joint decoder that can make use of this uncertainty together with higher-level language information. For this, we revisit decoding algorithms used in factorial generative models in early multi-talker speech recognition systems. In contrast with these early works, we replace the GMM acoustic model with DNN, which provides greater modeling power and simplifies part of the inference. We demonstrate the advantage of joint decoding in proof of concept experiments on a mixed-TIDIGITS dataset.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/LTAIN19087" target="_blank" >LTAIN19087: Multi-lingualita v řečových technologiích</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

—

ISSN

1990-9772

e-ISSN

—

Počet stran výsledku

5

Strana od-do

4955-4959

Název nakladatele

International Speech Communication Association

Místo vydání

Incheon

Místo konání akce

Incheon Korea

Datum konání akce

18. 9. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—