UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F19%3A00337712" target="_blank" >RIV/68407700:21730/19:00337712 - isvavai.cz</a>

Výsledek na webu

<a href="https://dl.acm.org/doi/abs/10.1145/3372422.3372443" target="_blank" >https://dl.acm.org/doi/abs/10.1145/3372422.3372443</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3372422.3372443" target="_blank" >10.1145/3372422.3372443</a>

Jazyk výsledku

angličtina

Název v původním jazyce

UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks

Popis výsledku v původním jazyce

Artificial neural networks, in particular the deep end-to-end architectures trained by error backpropagation (BP), are currently the topmost used learning systems. However, learning in such systems is only loosely inspired by the actual neural mechanisms. Algorithms based on local activation differences were designed as a biologically plausible alternative to BP. We propose Universal Bidirectional Activation-based Learning, a novel neural model which enhances the contrastive Hebbian learning rule with special hyperparameters yielding a single learning rule that can perform multiple ways of learning, similarly to what is assumed about learning in the brain. Unlike others, our model consists of mutually dependent, yet separate weight matrices for different directions of activation propagation. We show that UBAL can learn different tasks (such as pattern retrieval, denoising, or classification) with different setups of the learning hyperparameters. We also demonstrate the performance of our algorithm on a machine learning benchmark (MNIST). The experimental results presented in this paper confirm that UBAL is comparable with a basic version BP-trained multilayer network and the related biologically-motivated models.

Název v anglickém jazyce

UBAL: a Universal Bidirectional Activation-based Learning Rule for Neural Networks

Popis výsledku anglicky

Artificial neural networks, in particular the deep end-to-end architectures trained by error backpropagation (BP), are currently the topmost used learning systems. However, learning in such systems is only loosely inspired by the actual neural mechanisms. Algorithms based on local activation differences were designed as a biologically plausible alternative to BP. We propose Universal Bidirectional Activation-based Learning, a novel neural model which enhances the contrastive Hebbian learning rule with special hyperparameters yielding a single learning rule that can perform multiple ways of learning, similarly to what is assumed about learning in the brain. Unlike others, our model consists of mutually dependent, yet separate weight matrices for different directions of activation propagation. We show that UBAL can learn different tasks (such as pattern retrieval, denoising, or classification) with different setups of the learning hyperparameters. We also demonstrate the performance of our algorithm on a machine learning benchmark (MNIST). The experimental results presented in this paper confirm that UBAL is comparable with a basic version BP-trained multilayer network and the related biologically-motivated models.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

20204 - Robotics and automatic control

Projekt

<a href="/cs/project/VI20172019082" target="_blank" >VI20172019082: Smart Camera - Dohledové centrum nové generace</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 2019 International Conference on Computational Intelligence and Intelligent Systems

ISBN

978-1-4503-7259-6

ISSN

—

e-ISSN

—

Počet stran výsledku

6

Strana od-do

57-62

Název nakladatele

Association for Computing Machinery

Místo vydání

New York

Místo konání akce

Bangkok

Datum konání akce

23. 11. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—