Evaluation Framework for Deepfake Speech Detection: A Comparative Study of State-of-the-art Deepfake Speech Detectors

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0193261" target="_blank" >RIV/00216305:26230/26:0193261 - isvavai.cz</a>

Výsledek na webu

<a href="https://cybersecurity.springeropen.com/articles/10.1186/s42400-024-00346-1" target="_blank" >https://cybersecurity.springeropen.com/articles/10.1186/s42400-024-00346-1</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1186/s42400-024-00346-1" target="_blank" >10.1186/s42400-024-00346-1</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Evaluation Framework for Deepfake Speech Detection: A Comparative Study of State-of-the-art Deepfake Speech Detectors

Popis výsledku v původním jazyce

The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection methods. Moreover, real-world incidents such as fraudulent financial transactions highlight the severity of the problem. Although numerous detectors have been developed, their evaluation remains difficult due to different methodologies and benchmark datasets, making direct comparisons impossible. This study presents a general and detailed framework for evaluating and comparing deepfake speech detectors. We further demonstrate the use of this framework to evaluate 40 state-of-the-art deepfake speech detectors under various conditions and data samples. We objectively compare these methods and identify the key attributes influencing performance the most. We also stress the issue of generalisation, as current detectors struggle to detect previously unseen deepfake speech samples or samples that have been modified. Finally, to strengthen the defence against synthetic audio content, we provide recommendations for improving the robustness of future detectors.

Název v anglickém jazyce

Evaluation Framework for Deepfake Speech Detection: A Comparative Study of State-of-the-art Deepfake Speech Detectors

Popis výsledku anglicky

The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection methods. Moreover, real-world incidents such as fraudulent financial transactions highlight the severity of the problem. Although numerous detectors have been developed, their evaluation remains difficult due to different methodologies and benchmark datasets, making direct comparisons impossible. This study presents a general and detailed framework for evaluating and comparing deepfake speech detectors. We further demonstrate the use of this framework to evaluate 40 state-of-the-art deepfake speech detectors under various conditions and data samples. We objectively compare these methods and identify the key attributes influencing performance the most. We also stress the issue of generalisation, as current detectors struggle to detect previously unseen deepfake speech samples or samples that have been modified. Finally, to strengthen the defence against synthetic audio content, we provide recommendations for improving the robustness of future detectors.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Cybersecurity

ISSN

2523-3246

e-ISSN

—

Svazek periodika

8

Číslo periodika v rámci svazku

50

Stát vydavatele periodika

CN - Čínská lidová republika

Počet stran výsledku

24

Strana od-do

1-24

Kód UT WoS článku

001541737700001

EID výsledku v databázi Scopus

2-s2.0-105012388167