NN-Based Czech Sign Language Synthesis

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F19%3A43956396" target="_blank" >RIV/49777513:23520/19:43956396 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/chapter/10.1007%2F978-3-030-26061-3_57" target="_blank" >https://link.springer.com/chapter/10.1007%2F978-3-030-26061-3_57</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-26061-3_57" target="_blank" >10.1007/978-3-030-26061-3_57</a>

Jazyk výsledku

angličtina

Název v původním jazyce

NN-Based Czech Sign Language Synthesis

Popis výsledku v původním jazyce

This paper describes our Czech sign language synthesis that converts a Czech text into a series of skeletal poses. Our main goal is to avoid demanding handcrafted annotations of videos and to avoid a manual mapping between sign language glosses and skeletal poses. Thus, instead of solving these task separately, we join a model of an implicit neural-network-based translator and a model of the mapping between sign language glosses and we train both models together. For this purpose, we propose a simple differentiable operation that decomposes input symbols and it allows to produce a required series without any recurrent mechanism. We used The OpenPose toolbox to automatically extract skeletal poses and we designed a gradient-descend-based algorithm that converts a 2D skeleton model to a 3D skeleton model in order to fix misplaced and missing joints. Weather forecast parts of The daily news in Czech sign language were used to obtain our training and testing data. Our experiments demonstrate the benefit of the implicit translator and an ability of the designed sign language synthesis system to produce naturally formed skeletal poses.

Název v anglickém jazyce

NN-Based Czech Sign Language Synthesis

Popis výsledku anglicky

This paper describes our Czech sign language synthesis that converts a Czech text into a series of skeletal poses. Our main goal is to avoid demanding handcrafted annotations of videos and to avoid a manual mapping between sign language glosses and skeletal poses. Thus, instead of solving these task separately, we join a model of an implicit neural-network-based translator and a model of the mapping between sign language glosses and we train both models together. For this purpose, we propose a simple differentiable operation that decomposes input symbols and it allows to produce a required series without any recurrent mechanism. We used The OpenPose toolbox to automatically extract skeletal poses and we designed a gradient-descend-based algorithm that converts a 2D skeleton model to a 3D skeleton model in order to fix misplaced and missing joints. Weather forecast parts of The daily news in Czech sign language were used to obtain our training and testing data. Our experiments demonstrate the benefit of the implicit translator and an ability of the designed sign language synthesis system to produce naturally formed skeletal poses.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

20205 - Automation and control systems

Projekt

<a href="/cs/project/EF15_003%2F0000466" target="_blank" >EF15_003/0000466: Umělá inteligence a uvažování</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Speech and Computer, 21st International Conference, SPECOM 2019, Istanbul, turkey, August 20-25,2019, Proceedings

ISBN

978-3-030-26060-6

ISSN

0302-9743

e-ISSN

1611-3349

Počet stran výsledku

10

Strana od-do

559-568

Název nakladatele

Springer

Místo vydání

Cham

Místo konání akce

Instanbul, Turkey

Datum konání akce

20. 8. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—