Evaluation Framework for Deepfake Speech Detection: A Comparative Study of State-of-the-art Deepfake Speech Detectors

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0193261" target="_blank" >RIV/00216305:26230/26:0193261 - isvavai.cz</a>
Result on the web
<a href="https://cybersecurity.springeropen.com/articles/10.1186/s42400-024-00346-1" target="_blank" >https://cybersecurity.springeropen.com/articles/10.1186/s42400-024-00346-1</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1186/s42400-024-00346-1" target="_blank" >10.1186/s42400-024-00346-1</a>

Result language
angličtina
Original language name
Evaluation Framework for Deepfake Speech Detection: A Comparative Study of State-of-the-art Deepfake Speech Detectors
Original language description
The proliferation of deepfake speech poses a significant threat to cybersecurity, from manipulating political speeches and impersonating public figures to spoofing voice biometric systems. The increasing sophistication of adversaries increases the necessity of deploying adaptive detection methods. Moreover, real-world incidents such as fraudulent financial transactions highlight the severity of the problem. Although numerous detectors have been developed, their evaluation remains difficult due to different methodologies and benchmark datasets, making direct comparisons impossible. This study presents a general and detailed framework for evaluating and comparing deepfake speech detectors. We further demonstrate the use of this framework to evaluate 40 state-of-the-art deepfake speech detectors under various conditions and data samples. We objectively compare these methods and identify the key attributes influencing performance the most. We also stress the issue of generalisation, as current detectors struggle to detect previously unseen deepfake speech samples or samples that have been modified. Finally, to strengthen the defence against synthetic audio content, we provide recommendations for improving the robustness of future detectors.
Czech name
—
Czech description
—

Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)