Dealing with Newly Emerging OOVs in Broadcast Programs by Daily Updates of the Lexicon and Language Model

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F46747885%3A24220%2F20%3A00008351" target="_blank" >RIV/46747885:24220/20:00008351 - isvavai.cz</a>

Result on the web

<a href="https://link.springer.com/chapter/10.1007/978-3-030-60276-5_10" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-030-60276-5_10</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-60276-5_10" target="_blank" >10.1007/978-3-030-60276-5_10</a>

Result language

angličtina

Original language name

Dealing with Newly Emerging OOVs in Broadcast Programs by Daily Updates of the Lexicon and Language Model

Original language description

This paper deals with out-of-vocabulary (OOV) word recognition in the task of 24/7 broadcast stream transcription. Here, the majority of OOVs newly emerging over time are constituted of names of politicians, athletes, major world events, disasters, etc. The absence of these content OOVs, e.g. COVID-19, is detrimental to human understanding of the recognized text and harmful to further NLP processing, such as machine translation, named entity recognition or any type of semantic or dialogue analysis. In production environments, content OOVs are of extreme importance and it is essential that their correct transcription is provided as soon as possible. For this purpose, an approach based on daily updates of the lexicon and language model is proposed. It consists of three consecutive steps: a) the identification of new content OOVs from already existing text sources, b) their controlled addition into the lexicon of the transcription system and c) proper tuning of the language model. Experimental evaluation is performed on an extensive data-set compiled from various Czech broadcast programs. This data was produced by a real transcription platform over the course of 300 days in 2019. Detailed statistics and analysis of new content OOVs emerging within this period are also provided.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/TH03010018" target="_blank" >TH03010018: DeepSpot - Multilingual technology for spotting and instant alerting</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2020

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) - 22nd International Conference on Speech and Computer, SPECOM 2020

ISBN

978-303060275-8

ISSN

0302-9743

e-ISSN

—

Number of pages

11

Pages from-to

97-107

Publisher name

Springer Nature Switzerland

Place of publication

Switzerland

Event location

(on-line) St. Petersburg, Russian

Event date

Jan 1, 2020

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—