Data Alignment and Duration Modelling in VITS

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F24%3A43973184" target="_blank" >RIV/49777513:23520/24:43973184 - isvavai.cz</a>
Result on the web
<a href="https://link.springer.com/chapter/10.1007/978-3-031-70566-3_11" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-031-70566-3_11</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-031-70566-3_11" target="_blank" >10.1007/978-3-031-70566-3_11</a>

Result language
angličtina
Original language name
Data Alignment and Duration Modelling in VITS
Original language description
The paper analyses data alignment and duration modelling in the modern end-to-end speech synthesis model VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech). The standard version of VITS utilizes the MAS (Monotonic Alignment Search) procedure to align input text/phones and corresponding speech during the training procedure; the alignment is also used to obtain phoneme durations for the stochastic duration predictor training. This study analyzes the resulting MAS alignment and compares it with a reference alignment obtained by an LSTM-based phonetic segmentation system. We also examine the performance of VITS when the reference phonetic segmentation replaces the default MAS alignment. The comparison shows that while the original VITS is still slightly preferred in terms of quality, it provides a less interpretative data alignment. The duration modelling is more transparent in the modified version, allowing better duration control and modifications. The analysis has been carried out on two Czech voices.
Czech name
—
Czech description
—

Project
<a href="/en/project/GA22-27800S" target="_blank" >GA22-27800S: Transformers of multiple modalities for more natural spoken dialog</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Text, Speech, and Dialogue. Lecture Notes in Computer Science
ISBN
978-3-031-70565-6
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
12
Pages from-to
118-129
Publisher name
Springer International Publishing
Place of publication
Cham
Event location
Brno, Czech Republic
Event date
Sep 9, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
001307848400011

Similar results(10)