Stochastic Normalizations as Bayesian Learning
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F19%3A00332292" target="_blank" >RIV/68407700:21230/19:00332292 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-030-20890-5_30" target="_blank" >http://dx.doi.org/10.1007/978-3-030-20890-5_30</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-030-20890-5_30" target="_blank" >10.1007/978-3-030-20890-5_30</a>
Alternative languages
Result language
angličtina
Original language name
Stochastic Normalizations as Bayesian Learning
Original language description
In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods, is randomness of batch statistics. This randomness appears in the parameters rather than in activations and admits an interpretation as a practical Bayesian learning. We apply this idea to other (deterministic) normalization techniques that are oblivious to the batch size. We show that their generalization performance can be improved significantly by Bayesian learning of the same form. We obtain test performance comparable to BN and, at the same time, better validation losses suitable for subsequent output uncertainty estimation through approximate Bayesian posterior.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2019
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
ACCV 2018: Proceedings of the 14th Asian Conference on Computer Vision, Part II
ISBN
978-3-030-20889-9
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
17
Pages from-to
463-479
Publisher name
Springer
Place of publication
—
Event location
Perth
Event date
Dec 4, 2018
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000492902300030