Mobile AR Solution for Deaf People: Correlation Between Face Detection and Speech Recognition

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F62690094%3A18450%2F19%3A50015843" target="_blank" >RIV/62690094:18450/19:50015843 - isvavai.cz</a>

Result on the web

<a href="https://link.springer.com/chapter/10.1007/978-3-030-27192-3_19" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-030-27192-3_19</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-27192-3_19" target="_blank" >10.1007/978-3-030-27192-3_19</a>

Result language

angličtina

Original language name

Mobile AR Solution for Deaf People: Correlation Between Face Detection and Speech Recognition

Original language description

In the last two years, the authors’ research has been focused on designing a smart solution to compensate for hearing or visual deficiencies using the Google Glass hardware device and its own software architecture. This paper presents a solution aimed on deaf people or people with hearing impairment. At the beginning of this paper there is a brief explanation of the architecture of the designed solution, a description of the user interface through Google Glass. Related work to face detection, visual activity detection and speech recognition is presented with many possible approaches to these research areas. The principle of the solution lies in the combination of face detection and subsequent assignment of the recognized speech to the mouth of the correct face in the image. The aim of the solution is to digitally capture the ambient sound and based on its evaluation using neural networks, to present detected speech in a text form assigned to detected face from camera stream. The testing has shown that the solution is beneficial, and it is working as expected. Machine Learning Kit provides good results in face detection and communication with Google Cloud Speech API is fast enough for smooth user experience.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2019

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Mobile Web and Intelligent Information Systems

ISBN

978-3-030-27191-6

ISSN

0302-9743

e-ISSN

1611-3349

Number of pages

12

Pages from-to

243-254

Publisher name

Springer Nature Switzerland AG 2019

Place of publication

Springer, Cham

Event location

Istanbul, Turkey

Event date

Aug 26, 2019

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—