Weight Initialization Possibilities for Feedforward Neural Network with Linear Saturated Activation Functions
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216275%3A25530%2F16%3A39902015" target="_blank" >RIV/00216275:25530/16:39902015 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1016/j.ifacol.2016.12.009" target="_blank" >http://dx.doi.org/10.1016/j.ifacol.2016.12.009</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1016/j.ifacol.2016.12.009" target="_blank" >10.1016/j.ifacol.2016.12.009</a>
Alternative languages
Result language
angličtina
Original language name
Weight Initialization Possibilities for Feedforward Neural Network with Linear Saturated Activation Functions
Original language description
Initial weight choice is an important aspect of the training mechanism for feedforward neural networks. This paper deals with a particular topology of a feedforward neural network, where symmetric linear saturated activation functions are used in a hidden layer. Training of such a topology is a tricky procedure, since the activation functions are not fully differentiable. Thus, a proper initialization method for that case is even more important, than dealing with neural networks with sigmoid activation functions. Therefore, several initialization possibilities are examined and tested here. As a result, particular initialization methods are recommended for application, according to the class of the task to be solved.
Czech name
—
Czech description
—
Classification
Type
J<sub>x</sub> - Unclassified - Peer-reviewed scientific article (Jimp, Jsc and Jost)
CEP classification
BC - Theory and management systems
OECD FORD branch
—
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2016
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
IFAC Papersonline
ISSN
2405-8963
e-ISSN
—
Volume of the periodical
49
Issue of the periodical within the volume
25
Country of publishing house
NL - THE KINGDOM OF THE NETHERLANDS
Number of pages
6
Pages from-to
"49?54"
UT code for WoS article
—
EID of the result in the Scopus database
2-s2.0-85006964860