Zero-Shot vs. Few-Shot Multi-Speaker TTS Using Pre-trained Czech SpeechT5 Model

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F24%3A43973107" target="_blank" >RIV/49777513:23520/24:43973107 - isvavai.cz</a>
Result on the web
<a href="https://link.springer.com/10.1007/978-3-031-70566-3_5" target="_blank" >https://link.springer.com/10.1007/978-3-031-70566-3_5</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-031-70566-3_5" target="_blank" >10.1007/978-3-031-70566-3_5</a>

Result language
angličtina
Original language name
Zero-Shot vs. Few-Shot Multi-Speaker TTS Using Pre-trained Czech SpeechT5 Model
Original language description
In this paper, we experimented with the SpeechT5 model pre-trained on large-scale datasets. We pre-trained the foundation model from scratch and fine-tuned it on a large-scale robust multi-speaker text-to-speech (TTS) task. We tested the model capabilities in a zero- and few-shot scenario. Based on two listening tests, we evaluated the synthetic audio quality and the similarity of how synthetic voices resemble real voices. Our results showed that the SpeechT5 model can generate a synthetic voice for any speaker using only one minute of the target speaker's data. We successfully demonstrated the high quality and similarity of our synthetic voices on publicly known Czech politicians and celebrities.
Czech name
—
Czech description
—

Project
<a href="/en/project/GA22-27800S" target="_blank" >GA22-27800S: Transformers of multiple modalities for more natural spoken dialog</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Text, Speech, and Dialogue. Lecture Notes in Computer Science
ISBN
978-3-031-70565-6
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
12
Pages from-to
46-57
Publisher name
Springer International Publishing
Place of publication
Cham
Event location
Brno, Czech Republic
Event date
Sep 9, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
001307848400005

Similar results(10)