Long-Form End-to-End Speech Translation via Latent Alignment Segmentation
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F24%3A10511567" target="_blank" >RIV/00216208:11320/24:10511567 - isvavai.cz</a>
Result on the web
<a href="https://arxiv.org/abs/2309.11384" target="_blank" >https://arxiv.org/abs/2309.11384</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Long-Form End-to-End Speech Translation via Latent Alignment Segmentation
Original language description
Current simultaneous speech translation models can process audio only up to a few seconds long. Contemporary datasets provide an oracle segmentation into sentences based on human-annotated transcripts and translations. However, the segmentation into sentences is not available in the real world. Current speech segmentation approaches either offer poor segmentation quality or have to trade latency for quality. In this paper, we propose a novel segmentation approach for a low-latency end-to-end speech translation. We leverage the existing speech translation encoder-decoder architecture with ST CTC and show that it can perform the segmentation task without supervision or additional parameters. To the best of our knowledge, our method is the first that allows an actual end-to-end simultaneous speech translation, as the same model is used for translation and segmentation at the same time. On a diverse set of language pairs and in- and out-of-domain data, we show that the proposed approach achieves state-of-
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 2024 IEEE Spoken Language Technology Workshop (SLT)
ISBN
978-1-4799-7130-5
ISSN
—
e-ISSN
—
Number of pages
7
Pages from-to
1076-1082
Publisher name
IEEE
Place of publication
Piscataway, USA
Event location
Macau, China
Event date
Dec 2, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—