Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F22%3APU146108" target="_blank" >RIV/00216305:26230/22:PU146108 - isvavai.cz</a>

Result on the web

<a href="https://www.isca-speech.org/archive/pdfs/interspeech_2022/kocour22_interspeech.pdf" target="_blank" >https://www.isca-speech.org/archive/pdfs/interspeech_2022/kocour22_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2022-10406" target="_blank" >10.21437/Interspeech.2022-10406</a>

Result language

angličtina

Original language name

Revisiting joint decoding based multi-talker speech recognition with DNN acoustic model

Original language description

In typical multi-talker speech recognition systems, a neural network-based acoustic model predicts senone state posteriors for each speaker. These are later used by a single-talker decoder which is applied on each speaker-specific output stream separately. In this work, we argue that such a scheme is sub-optimal and propose a principled solution that decodes all speakers jointly. We modify the acoustic model to predict joint state posteriors for all speakers, enabling the network to express uncertainty about the attribution of parts of the speech signal to the speakers. We employ a joint decoder that can make use of this uncertainty together with higher-level language information. For this, we revisit decoding algorithms used in factorial generative models in early multi-talker speech recognition systems. In contrast with these early works, we replace the GMM acoustic model with DNN, which provides greater modeling power and simplifies part of the inference. We demonstrate the advantage of joint decoding in proof of concept experiments on a mixed-TIDIGITS dataset.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/LTAIN19087" target="_blank" >LTAIN19087: Multi-linguality in speech technologies</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2022

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

—

ISSN

1990-9772

e-ISSN

—

Number of pages

5

Pages from-to

4955-4959

Publisher name

International Speech Communication Association

Place of publication

Incheon

Event location

Incheon Korea

Event date

Sep 18, 2022

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—