Subrecursive Neural Networks
Identifikátory výsledku
Kód výsledku v IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F19%3A00490203" target="_blank" >RIV/67985807:_____/19:00490203 - isvavai.cz</a>
Výsledek na webu
<a href="http://dx.doi.org/10.1016/j.neunet.2019.04.019" target="_blank" >http://dx.doi.org/10.1016/j.neunet.2019.04.019</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1016/j.neunet.2019.04.019" target="_blank" >10.1016/j.neunet.2019.04.019</a>
Alternativní jazyky
Jazyk výsledku
angličtina
Název v původním jazyce
Subrecursive Neural Networks
Popis výsledku v původním jazyce
It has been known for discrete-time recurrent neural networks (NNs) that binary-state models using the Heaviside activation function (with Boolean outputs 0 or 1) are equivalent to finite automata (level 3 in the Chomsky hierarchy), while analog-state NNs with rational weights, employing the saturated-linear function (with real-number outputs in the interval ), are Turing complete (Chomsky level 0) even for three analog units. However, it is as yet unknown whether there exist subrecursive (i.e. sub-Turing) NN models which occur on Chomsky levels 1 or 2. In this paper, we provide such a model which is a binary-state NN extended with one extra analog unit (1ANN). We achieve a syntactic characterization of languages that are accepted online by 1ANNs in terms of so-called cut languages which are combined in a certain way by usual operations. We employ this characterization for proving that languages accepted by 1ANNs with rational weights are context-sensitive (Chomsky level 1) and we present explicit examples of such languages that are not context-free (i.e. are above Chomsky level 2). In addition, we formulate a sufficient condition when a 1ANN recognizes a regular language (Chomsky level 3) in terms of quasi-periodicity of parameters derived from its real weights, which is satisfied e.g. for rational weights provided that the inverse of the real self-loop weight of the analog unit is a Pisot number.
Název v anglickém jazyce
Subrecursive Neural Networks
Popis výsledku anglicky
It has been known for discrete-time recurrent neural networks (NNs) that binary-state models using the Heaviside activation function (with Boolean outputs 0 or 1) are equivalent to finite automata (level 3 in the Chomsky hierarchy), while analog-state NNs with rational weights, employing the saturated-linear function (with real-number outputs in the interval ), are Turing complete (Chomsky level 0) even for three analog units. However, it is as yet unknown whether there exist subrecursive (i.e. sub-Turing) NN models which occur on Chomsky levels 1 or 2. In this paper, we provide such a model which is a binary-state NN extended with one extra analog unit (1ANN). We achieve a syntactic characterization of languages that are accepted online by 1ANNs in terms of so-called cut languages which are combined in a certain way by usual operations. We employ this characterization for proving that languages accepted by 1ANNs with rational weights are context-sensitive (Chomsky level 1) and we present explicit examples of such languages that are not context-free (i.e. are above Chomsky level 2). In addition, we formulate a sufficient condition when a 1ANN recognizes a regular language (Chomsky level 3) in terms of quasi-periodicity of parameters derived from its real weights, which is satisfied e.g. for rational weights provided that the inverse of the real self-loop weight of the analog unit is a Pisot number.
Klasifikace
Druh
J<sub>imp</sub> - Článek v periodiku v databázi Web of Science
CEP obor
—
OECD FORD obor
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Návaznosti výsledku
Projekt
<a href="/cs/project/GA19-05704S" target="_blank" >GA19-05704S: FoNeCo: Analytické základy neurovýpočtů</a><br>
Návaznosti
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Ostatní
Rok uplatnění
2019
Kód důvěrnosti údajů
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Údaje specifické pro druh výsledku
Název periodika
Neural Networks
ISSN
0893-6080
e-ISSN
—
Svazek periodika
116
Číslo periodika v rámci svazku
August
Stát vydavatele periodika
GB - Spojené království Velké Británie a Severního Irska
Počet stran výsledku
16
Strana od-do
208-223
Kód UT WoS článku
000471669900018
EID výsledku v databázi Scopus
2-s2.0-85065759931