Evolutionary Echo State Network: evolving reservoirs in the Fourier space

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989100%3A27240%2F22%3A10253425" target="_blank" >RIV/61989100:27240/22:10253425 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/document/9892892" target="_blank" >https://ieeexplore.ieee.org/document/9892892</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IJCNN55064.2022.9892892" target="_blank" >10.1109/IJCNN55064.2022.9892892</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Evolutionary Echo State Network: evolving reservoirs in the Fourier space

Popis výsledku v původním jazyce

The Echo State Network (ESN) is a class of Recurrent Neural Network with a large number of hidden-hidden weights (in the so-called reservoir). Canonical ESN and its variations have recently received significant attention due to their remarkable success in the modeling of non-linear dynamical systems. The reservoir is randomly connected with fixed weights that don&apos;t change in the learning process. Only the weights from reservoir to output are trained. Since the reservoir is fixed during the training procedure, we may wonder if the computational power of the recurrent structure is fully harnessed. In this article, we propose a new computational model of the ESN type, that represents the reservoir weights in the Fourier space and performs a fine-tuning of these weights applying genetic algorithms in the frequency domain. The main interest is that this procedure will work in a much smaller space compared to the classical ESN, thus providing a dimensionality reduction transformation of the initial method. The proposed technique allows us to exploit the benefits of the large recurrent structure avoiding the training problems of gradient-based method. We provide a detailed experimental study that demonstrates the good performances of our approach with well-known chaotic systems and real-world data.

Název v anglickém jazyce

Evolutionary Echo State Network: evolving reservoirs in the Fourier space

Popis výsledku anglicky

The Echo State Network (ESN) is a class of Recurrent Neural Network with a large number of hidden-hidden weights (in the so-called reservoir). Canonical ESN and its variations have recently received significant attention due to their remarkable success in the modeling of non-linear dynamical systems. The reservoir is randomly connected with fixed weights that don&apos;t change in the learning process. Only the weights from reservoir to output are trained. Since the reservoir is fixed during the training procedure, we may wonder if the computational power of the recurrent structure is fully harnessed. In this article, we propose a new computational model of the ESN type, that represents the reservoir weights in the Fourier space and performs a fine-tuning of these weights applying genetic algorithms in the frequency domain. The main interest is that this procedure will work in a much smaller space compared to the classical ESN, thus providing a dimensionality reduction transformation of the initial method. The proposed technique allows us to exploit the benefits of the large recurrent structure avoiding the training problems of gradient-based method. We provide a detailed experimental study that demonstrates the good performances of our approach with well-known chaotic systems and real-world data.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GF21-33574K" target="_blank" >GF21-33574K: Celoživotní strojové učení z datových proudů</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2022 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN)

ISBN

978-1-72818-671-9

ISSN

2161-4393

e-ISSN

—

Počet stran výsledku

8

Strana od-do

—

Název nakladatele

IEEE

Místo vydání

NEW YORK

Místo konání akce

Padua

Datum konání akce

18. 7. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000867070908004