Analysis of impact of emotions on target speech extraction and speech separation

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F22%3APU146144" target="_blank" >RIV/00216305:26230/22:PU146144 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9914718" target="_blank" >https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9914718</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IWAENC53105.2022.9914718" target="_blank" >10.1109/IWAENC53105.2022.9914718</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Analysis of impact of emotions on target speech extraction and speech separation

Popis výsledku v původním jazyce

Recently, the performance of blind speech separation (BSS) and target speech extraction (TSE) has greatly progressed. Most works, however, focus on relatively well-controlled conditions using, e.g., read speech. The performance may degrade in more realistic situations. One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech. In this paper, we investigate the influence of emotions on TSE and BSS. We create a new test dataset of emotional mixtures for the evaluation of TSE and BSS. This dataset combines LibriSpeech and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). Through controlled experiments, we can analyze the impact of different emotions on the performance of BSS and TSE.We observe that BSS is relatively robust to emotions, while TSE, which requires identifying and extracting the speech of a target speaker, is much more sensitive to emotions. On comparative speaker verification experiments we show that identifying the target speaker may be particularly challenging when dealing with emotional speech. Using our findings, we outline potential future directions that could improve the robustness of BSS and TSE systems toward emotional speech.

Název v anglickém jazyce

Analysis of impact of emotions on target speech extraction and speech separation

Popis výsledku anglicky

Recently, the performance of blind speech separation (BSS) and target speech extraction (TSE) has greatly progressed. Most works, however, focus on relatively well-controlled conditions using, e.g., read speech. The performance may degrade in more realistic situations. One of the factors causing such degradation may be intrinsic speaker variability, such as emotions, occurring commonly in realistic speech. In this paper, we investigate the influence of emotions on TSE and BSS. We create a new test dataset of emotional mixtures for the evaluation of TSE and BSS. This dataset combines LibriSpeech and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). Through controlled experiments, we can analyze the impact of different emotions on the performance of BSS and TSE.We observe that BSS is relatively robust to emotions, while TSE, which requires identifying and extracting the speech of a target speaker, is much more sensitive to emotions. On comparative speaker verification experiments we show that identifying the target speaker may be particularly challenging when dealing with emotional speech. Using our findings, we outline potential future directions that could improve the robustness of BSS and TSE systems toward emotional speech.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/LTAIN19087" target="_blank" >LTAIN19087: Multi-lingualita v řečových technologiích</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of The 17th International Workshop on Acoustic Signal Enhancement (IWAENC 2022)

ISBN

978-1-6654-6867-1

ISSN

—

e-ISSN

—

Počet stran výsledku

5

Strana od-do

1-5

Název nakladatele

IEEE Signal Processing Society

Místo vydání

Bamberg

Místo konání akce

Bamberg

Datum konání akce

5. 9. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—