Dealing with Newly Emerging OOVs in Broadcast Programs by Daily Updates of the Lexicon and Language Model

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F46747885%3A24220%2F20%3A00008351" target="_blank" >RIV/46747885:24220/20:00008351 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007/978-3-030-60276-5_10" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-030-60276-5_10</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-60276-5_10" target="_blank" >10.1007/978-3-030-60276-5_10</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Dealing with Newly Emerging OOVs in Broadcast Programs by Daily Updates of the Lexicon and Language Model

Popis výsledku v původním jazyce

This paper deals with out-of-vocabulary (OOV) word recognition in the task of 24/7 broadcast stream transcription. Here, the majority of OOVs newly emerging over time are constituted of names of politicians, athletes, major world events, disasters, etc. The absence of these content OOVs, e.g. COVID-19, is detrimental to human understanding of the recognized text and harmful to further NLP processing, such as machine translation, named entity recognition or any type of semantic or dialogue analysis. In production environments, content OOVs are of extreme importance and it is essential that their correct transcription is provided as soon as possible. For this purpose, an approach based on daily updates of the lexicon and language model is proposed. It consists of three consecutive steps: a) the identification of new content OOVs from already existing text sources, b) their controlled addition into the lexicon of the transcription system and c) proper tuning of the language model. Experimental evaluation is performed on an extensive data-set compiled from various Czech broadcast programs. This data was produced by a real transcription platform over the course of 300 days in 2019. Detailed statistics and analysis of new content OOVs emerging within this period are also provided.

Název v anglickém jazyce

Dealing with Newly Emerging OOVs in Broadcast Programs by Daily Updates of the Lexicon and Language Model

Popis výsledku anglicky

This paper deals with out-of-vocabulary (OOV) word recognition in the task of 24/7 broadcast stream transcription. Here, the majority of OOVs newly emerging over time are constituted of names of politicians, athletes, major world events, disasters, etc. The absence of these content OOVs, e.g. COVID-19, is detrimental to human understanding of the recognized text and harmful to further NLP processing, such as machine translation, named entity recognition or any type of semantic or dialogue analysis. In production environments, content OOVs are of extreme importance and it is essential that their correct transcription is provided as soon as possible. For this purpose, an approach based on daily updates of the lexicon and language model is proposed. It consists of three consecutive steps: a) the identification of new content OOVs from already existing text sources, b) their controlled addition into the lexicon of the transcription system and c) proper tuning of the language model. Experimental evaluation is performed on an extensive data-set compiled from various Czech broadcast programs. This data was produced by a real transcription platform over the course of 300 days in 2019. Detailed statistics and analysis of new content OOVs emerging within this period are also provided.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/TH03010018" target="_blank" >TH03010018: DeepSpot - Multilingvální technologie pro detekci a včasné upozornění</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) - 22nd International Conference on Speech and Computer, SPECOM 2020

ISBN

978-303060275-8

ISSN

0302-9743

e-ISSN

—

Počet stran výsledku

11

Strana od-do

97-107

Název nakladatele

Springer Nature Switzerland

Místo vydání

Switzerland

Místo konání akce

(on-line) St. Petersburg, Russian

Datum konání akce

1. 1. 2020

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—