Lightweight online punctuation and capitalization restoration for streaming ASR systems

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F46747885%3A24220%2F25%3A00014132" target="_blank" >RIV/46747885:24220/25:00014132 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.sciencedirect.com/science/article/pii/S0167639325000846?pes=vor&utm_source=clarivate&getft_integrator=clarivate" target="_blank" >https://www.sciencedirect.com/science/article/pii/S0167639325000846?pes=vor&utm_source=clarivate&getft_integrator=clarivate</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.specom.2025.103269" target="_blank" >10.1016/j.specom.2025.103269</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Lightweight online punctuation and capitalization restoration for streaming ASR systems

Popis výsledku v původním jazyce

This work proposes a lightweight online approach to automatic punctuation and capitalization restoration (APCR). Our method takes pure text as input and can be utilized in real-time speech transcription systems for, e.g., live captioning of TV or radio streams. We develop and evaluate it in a series of consecutive experiments, starting with the task of automatic punctuation restoration (APR). Within that, we also compare our results to another real-time APR method, which combines textual and acoustic features. The test data that we use for this purpose contains automatic transcripts of radio talks and TV debates. In the second part of the paper, we extend our method towards the task of automatic capitalization restoration (ACR). The resulting approach uses two consecutive ELECTRA-small models complemented by simple classification heads; the first ELECTRA model restores punctuation, while the second performs capitalization. Our complete system allows for restoring question marks, commas, periods, and capitalization with a very short inference time and a low latency of just four words. We evaluate its performance for Czech and German, and also compare its results to those of another existing APCR system for English. We are also publishing the data used for our evaluation and testing.

Název v anglickém jazyce

Lightweight online punctuation and capitalization restoration for streaming ASR systems

Popis výsledku anglicky

This work proposes a lightweight online approach to automatic punctuation and capitalization restoration (APCR). Our method takes pure text as input and can be utilized in real-time speech transcription systems for, e.g., live captioning of TV or radio streams. We develop and evaluate it in a series of consecutive experiments, starting with the task of automatic punctuation restoration (APR). Within that, we also compare our results to another real-time APR method, which combines textual and acoustic features. The test data that we use for this purpose contains automatic transcripts of radio talks and TV debates. In the second part of the paper, we extend our method towards the task of automatic capitalization restoration (ACR). The resulting approach uses two consecutive ELECTRA-small models complemented by simple classification heads; the first ELECTRA model restores punctuation, while the second performs capitalization. Our complete system allows for restoring question marks, commas, periods, and capitalization with a very short inference time and a low latency of just four words. We evaluate its performance for Czech and German, and also compare its results to those of another existing APCR system for English. We are also publishing the data used for our evaluation and testing.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10307 - Acoustics

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

SPEECH COMMUNICATION>

ISSN

0167-6393

e-ISSN

—

Svazek periodika

173

Číslo periodika v rámci svazku

SEP

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

4

Strana od-do

—

Kód UT WoS článku

001529336100001

EID výsledku v databázi Scopus

2-s2.0-105009904852