Weight Initialization Possibilities for Feedforward Neural Network with Linear Saturated Activation Functions

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216275%3A25530%2F16%3A39902015" target="_blank" >RIV/00216275:25530/16:39902015 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1016/j.ifacol.2016.12.009" target="_blank" >http://dx.doi.org/10.1016/j.ifacol.2016.12.009</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.ifacol.2016.12.009" target="_blank" >10.1016/j.ifacol.2016.12.009</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Weight Initialization Possibilities for Feedforward Neural Network with Linear Saturated Activation Functions

Popis výsledku v původním jazyce

Initial weight choice is an important aspect of the training mechanism for feedforward neural networks. This paper deals with a particular topology of a feedforward neural network, where symmetric linear saturated activation functions are used in a hidden layer. Training of such a topology is a tricky procedure, since the activation functions are not fully differentiable. Thus, a proper initialization method for that case is even more important, than dealing with neural networks with sigmoid activation functions. Therefore, several initialization possibilities are examined and tested here. As a result, particular initialization methods are recommended for application, according to the class of the task to be solved.

Název v anglickém jazyce

Weight Initialization Possibilities for Feedforward Neural Network with Linear Saturated Activation Functions

Popis výsledku anglicky

Initial weight choice is an important aspect of the training mechanism for feedforward neural networks. This paper deals with a particular topology of a feedforward neural network, where symmetric linear saturated activation functions are used in a hidden layer. Training of such a topology is a tricky procedure, since the activation functions are not fully differentiable. Thus, a proper initialization method for that case is even more important, than dealing with neural networks with sigmoid activation functions. Therefore, several initialization possibilities are examined and tested here. As a result, particular initialization methods are recommended for application, according to the class of the task to be solved.

Druh

J_x - Nezařazeno - Článek v odborném periodiku (Jimp, Jsc a Jost)

CEP obor

BC - Teorie a systémy řízení

OECD FORD obor

—

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2016

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IFAC Papersonline

ISSN

2405-8963

e-ISSN

—

Svazek periodika

49

Číslo periodika v rámci svazku

25

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

6

Strana od-do

"49?54"

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85006964860