Efficient Text Classification with Echo State Networks

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F21%3A00542211" target="_blank" >RIV/67985807:_____/21:00542211 - isvavai.cz</a>

Result on the web

<a href="http://dx.doi.org/10.1109/IJCNN52387.2021.9533958" target="_blank" >http://dx.doi.org/10.1109/IJCNN52387.2021.9533958</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IJCNN52387.2021.9533958" target="_blank" >10.1109/IJCNN52387.2021.9533958</a>

Result language

angličtina

Original language name

Efficient Text Classification with Echo State Networks

Original language description

We consider echo state networks (ESNs) for text classification. More specifically, we investigate the learning capabilities of ESNs with pre-trained word embedding as input features, trained on the IMDb and TREC sentiment and question classification datasets, respectively. First, we introduce a customized training paradigm for the processing of multiple input time series (the inputs texts) associated with categorical targets (their corresponding classes). For sentiment tasks, we use an additional frozen attention mechanism which is based on an external lexicon, and hence requires only negligible computational cost. Within this paradigm, ESNs can be trained in tens of seconds on a GPU. We show that ESNs significantly outperform their Ridge regression baselines provided with the same embedded features. ESNs also compete with classical Bi-LSTM networks while keeping a training time of up to 23 times faster. These results show that ESNs can be considered as robust, efficient and fast candidates for text classification tasks. Overall, this study falls within the context of light and fast-to-train models for NLP.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GA19-05704S" target="_blank" >GA19-05704S: FoNeCo: Analytical Foundations of Neurocomputing</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

IJCNN 2021. The International Joint Conference on Neural Networks Proceedings

ISBN

978-0-7381-3366-9

ISSN

—

e-ISSN

—

Number of pages

8

Pages from-to

1-8

Publisher name

IEEE

Place of publication

Piscataway

Event location

Virtual

Event date

Jul 18, 2021

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

000722581705038