Multi-Channel Extension of Pre-trained Models for Speaker Verification

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F24%3APU154930" target="_blank" >RIV/00216305:26230/24:PU154930 - isvavai.cz</a>

Result on the web

<a href="https://www.isca-archive.org/interspeech_2024/mosner24_interspeech.pdf" target="_blank" >https://www.isca-archive.org/interspeech_2024/mosner24_interspeech.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21437/Interspeech.2024-1260" target="_blank" >10.21437/Interspeech.2024-1260</a>

Result language

angličtina

Original language name

Multi-Channel Extension of Pre-trained Models for Speaker Verification

Original language description

In this work, we focus on designing a multi-channel speech processing system based on large pre-trained models. These models are typically trained for single-channel scenarios via self-supervised learning (SSL). A common approach to using the SSL models with microphone array data is to prepend it with a multi-channel speech enhancement. The downside is that spatial information can be leveraged only by the pre-processing stage, and enhancement errors get propagated to the SSL model. We aim to alleviate the issue by designing METRO, a Multi-channel ExTension of pRe-trained mOdels. It interleaves per- channel processing with cross-channel information exchange, eventually fusing channels into one. While our approach is general, here we focus on multi-channel speaker verification. Our experiments on the MultiSV corpus show noteworthy improvements over the best-published results on the dataset.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/VJ01010108" target="_blank" >VJ01010108: Robust processing of recordings for operations and security</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

ISBN

—

ISSN

1990-9772

e-ISSN

—

Number of pages

5

Pages from-to

2135-2139

Publisher name

International Speech Communication Association

Place of publication

Kos

Event location

Kos

Event date

Sep 1, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—