NN-Based Czech Sign Language Synthesis

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F19%3A43956396" target="_blank" >RIV/49777513:23520/19:43956396 - isvavai.cz</a>
Result on the web
<a href="https://link.springer.com/chapter/10.1007%2F978-3-030-26061-3_57" target="_blank" >https://link.springer.com/chapter/10.1007%2F978-3-030-26061-3_57</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-030-26061-3_57" target="_blank" >10.1007/978-3-030-26061-3_57</a>

Result language
angličtina
Original language name
NN-Based Czech Sign Language Synthesis
Original language description
This paper describes our Czech sign language synthesis that converts a Czech text into a series of skeletal poses. Our main goal is to avoid demanding handcrafted annotations of videos and to avoid a manual mapping between sign language glosses and skeletal poses. Thus, instead of solving these task separately, we join a model of an implicit neural-network-based translator and a model of the mapping between sign language glosses and we train both models together. For this purpose, we propose a simple differentiable operation that decomposes input symbols and it allows to produce a required series without any recurrent mechanism. We used The OpenPose toolbox to automatically extract skeletal poses and we designed a gradient-descend-based algorithm that converts a 2D skeleton model to a 3D skeleton model in order to fix misplaced and missing joints. Weather forecast parts of The daily news in Czech sign language were used to obtain our training and testing data. Our experiments demonstrate the benefit of the implicit translator and an ability of the designed sign language synthesis system to produce naturally formed skeletal poses.
Czech name
—
Czech description
—

Project
<a href="/en/project/EF15_003%2F0000466" target="_blank" >EF15_003/0000466: Artificial Intelligence and Reasoning</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2019
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Speech and Computer, 21st International Conference, SPECOM 2019, Istanbul, turkey, August 20-25,2019, Proceedings
ISBN
978-3-030-26060-6
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
10
Pages from-to
559-568
Publisher name
Springer
Place of publication
Cham
Event location
Instanbul, Turkey
Event date
Aug 20, 2019
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—

Similar results(10)