Data Alignment and Duration Modelling in VITS

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F24%3A43973184" target="_blank" >RIV/49777513:23520/24:43973184 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007/978-3-031-70566-3_11" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-031-70566-3_11</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-70566-3_11" target="_blank" >10.1007/978-3-031-70566-3_11</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Data Alignment and Duration Modelling in VITS

Popis výsledku v původním jazyce

The paper analyses data alignment and duration modelling in the modern end-to-end speech synthesis model VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech). The standard version of VITS utilizes the MAS (Monotonic Alignment Search) procedure to align input text/phones and corresponding speech during the training procedure; the alignment is also used to obtain phoneme durations for the stochastic duration predictor training. This study analyzes the resulting MAS alignment and compares it with a reference alignment obtained by an LSTM-based phonetic segmentation system. We also examine the performance of VITS when the reference phonetic segmentation replaces the default MAS alignment. The comparison shows that while the original VITS is still slightly preferred in terms of quality, it provides a less interpretative data alignment. The duration modelling is more transparent in the modified version, allowing better duration control and modifications. The analysis has been carried out on two Czech voices.

Název v anglickém jazyce

Data Alignment and Duration Modelling in VITS

Popis výsledku anglicky

The paper analyses data alignment and duration modelling in the modern end-to-end speech synthesis model VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech). The standard version of VITS utilizes the MAS (Monotonic Alignment Search) procedure to align input text/phones and corresponding speech during the training procedure; the alignment is also used to obtain phoneme durations for the stochastic duration predictor training. This study analyzes the resulting MAS alignment and compares it with a reference alignment obtained by an LSTM-based phonetic segmentation system. We also examine the performance of VITS when the reference phonetic segmentation replaces the default MAS alignment. The comparison shows that while the original VITS is still slightly preferred in terms of quality, it provides a less interpretative data alignment. The duration modelling is more transparent in the modified version, allowing better duration control and modifications. The analysis has been carried out on two Czech voices.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

20205 - Automation and control systems

Projekt

<a href="/cs/project/GA22-27800S" target="_blank" >GA22-27800S: Využití vícemodálních Transformerů pro přirozenější hlasový dialog</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Text, Speech, and Dialogue. Lecture Notes in Computer Science

ISBN

978-3-031-70565-6

ISSN

0302-9743

e-ISSN

1611-3349

Počet stran výsledku

12

Strana od-do

118-129

Název nakladatele

Springer International Publishing

Místo vydání

Cham

Místo konání akce

Brno, Czech Republic

Datum konání akce

9. 9. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001307848400011