Designing Advanced Geometric Features for Automatic Russian Visual Speech Recognition

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F18%3A43952639" target="_blank" >RIV/49777513:23520/18:43952639 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007%2F978-3-319-99579-3_26" target="_blank" >https://link.springer.com/chapter/10.1007%2F978-3-319-99579-3_26</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-319-99579-3_26" target="_blank" >10.1007/978-3-319-99579-3_26</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Designing Advanced Geometric Features for Automatic Russian Visual Speech Recognition

Popis výsledku v původním jazyce

The use of video information plays an increasingly important role for automatic speech recognition. Nowadays, audio-only based systems have reached a certain accuracy threshold and many researchers see a solution to the problem in the use of visual modality to obtain better results. Despite the fact that audio modality of speech is much more representative than video, their proper fusion can improve both quality and robustness of the entire recognition system that was proved in practice by many researches. However, no agreement between researchers on the optimal set of visual features was reached. In this paper, we investigate this issue in more detail and propose advanced geometry-based visual features for automatic Russian lip-reading system. The experi-ments were conducted using collected HAVRUS audio-visual speech database. The average viseme recognition accuracy of our system trained on the entire corpus is 40.62%. We also tested the main state-of-the-art methods for visual speech recognition, applying them to continuous Russian speech with high-speed recordings (200 frames per seconds).

Název v anglickém jazyce

Designing Advanced Geometric Features for Automatic Russian Visual Speech Recognition

Popis výsledku anglicky

The use of video information plays an increasingly important role for automatic speech recognition. Nowadays, audio-only based systems have reached a certain accuracy threshold and many researchers see a solution to the problem in the use of visual modality to obtain better results. Despite the fact that audio modality of speech is much more representative than video, their proper fusion can improve both quality and robustness of the entire recognition system that was proved in practice by many researches. However, no agreement between researchers on the optimal set of visual features was reached. In this paper, we investigate this issue in more detail and propose advanced geometry-based visual features for automatic Russian lip-reading system. The experi-ments were conducted using collected HAVRUS audio-visual speech database. The average viseme recognition accuracy of our system trained on the entire corpus is 40.62%. We also tested the main state-of-the-art methods for visual speech recognition, applying them to continuous Russian speech with high-speed recordings (200 frames per seconds).

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

20205 - Automation and control systems

Projekt

<a href="/cs/project/LO1506" target="_blank" >LO1506: Podpora udržitelnosti centra NTIS - Nové technologie pro informační společnost</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2018

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Speech and Computer 20th International Conference, SPECOM 2018 Leipzig, Germany, September 18–22, 2018, Proceedings

ISBN

978-3-319-99578-6

ISSN

0302-9743

e-ISSN

1611-3349

Počet stran výsledku

10

Strana od-do

245-254

Název nakladatele

Springer Nature Switzerland AG

Místo vydání

Cham

Místo konání akce

Leipzig, Germany

Datum konání akce

18. 9. 2018

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—