Energy Complexity of Recurrent Neural Networks
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F14%3A00393985" target="_blank" >RIV/67985807:_____/14:00393985 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1162/NECO_a_00579" target="_blank" >http://dx.doi.org/10.1162/NECO_a_00579</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1162/NECO_a_00579" target="_blank" >10.1162/NECO_a_00579</a>
Alternative languages
Result language
angličtina
Original language name
Energy Complexity of Recurrent Neural Networks
Original language description
Recently, a new so-called energy complexity measure has been introduced and studied for feedforward perceptron networks. This measure is inspired by the fact that biological neurons require more energy to transmit a spike than not to fire, and the activity of neurons in the brain is quite sparse, with only about 1% of neurons firing. In this paper, we investigate the energy complexity of recurrent networks which counts the number of active neurons at any time instant of a computation. We prove that anydeterministic finite automaton with m states can be simulated by a neural network of optimal size s=Theta(sqrt{m}) with the time overhead of tau=O(s/e) per one input bit, using the energy O(e), for any e such that e=Omega(log s) and e=O(s), which shows the time-energy tradeoff in recurrent networks. In addition, for the time overhead tau satisfying tau^tau=o(s), we obtain the lower bound of s^{c/tau} on the energy of such a simulation, for some constant c>0 and for infinitely ma
Czech name
—
Czech description
—
Classification
Type
J<sub>x</sub> - Unclassified - Peer-reviewed scientific article (Jimp, Jsc and Jost)
CEP classification
IN - Informatics
OECD FORD branch
—
Result continuities
Project
<a href="/en/project/GAP202%2F10%2F1333" target="_blank" >GAP202/10/1333: NoSCoM: Non-Standard Computational Models and Their Applications in Complexity, Linguistics, and Learning</a><br>
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2014
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
Neural Computation
ISSN
0899-7667
e-ISSN
—
Volume of the periodical
26
Issue of the periodical within the volume
5
Country of publishing house
US - UNITED STATES
Number of pages
21
Pages from-to
953-973
UT code for WoS article
000334027800005
EID of the result in the Scopus database
—