Learning Entropy of Adaptive Filters via Clustering Techniques

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21220%2F20%3A00344466" target="_blank" >RIV/68407700:21220/20:00344466 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/SSPD47486.2020.9272138" target="_blank" >https://doi.org/10.1109/SSPD47486.2020.9272138</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/SSPD47486.2020.9272138" target="_blank" >10.1109/SSPD47486.2020.9272138</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Learning Entropy of Adaptive Filters via Clustering Techniques

Popis výsledku v původním jazyce

Learning Entropy (LE) was initially introduced as a measure for sample point novelty by unusually large learning effort of an online learning system. The key concept is that LE is based on pre-training and further online learning, and the novelty measure is not necessarily correlated to the prediction error. Most recently, the idea of LE was revised as a novel non-probabilistic, i.e., machine-learning-based information measure. This measure is high when a learning system is not familiar with a given data point, so the learning activity to learn novel data points is unusual (regardless of the prediction error), i.e., the learning increments display unusual patterns during adaptation. In this paper, we propose concepts of the learning state and the learning state space so that LE can be approximated via neighbourhood analysis in the learning space. Further, two novel clustering-based techniques for approximation of sample point LE are proposed. The first one is based on the sum of K nearest neighbour distances. The second one is based on multiscale neighbourhood cumulative sum. Also, we preprocess the learning space with dimensionality reduction that is promising for research of LE even with neural networks and potentially with deep neural networks. The performance of novelty detection with the clustering-based sample point LE with dimensionality reduction is compared to the original algorithms of LE, and its potentials are discussed.

Název v anglickém jazyce

Learning Entropy of Adaptive Filters via Clustering Techniques

Popis výsledku anglicky

Learning Entropy (LE) was initially introduced as a measure for sample point novelty by unusually large learning effort of an online learning system. The key concept is that LE is based on pre-training and further online learning, and the novelty measure is not necessarily correlated to the prediction error. Most recently, the idea of LE was revised as a novel non-probabilistic, i.e., machine-learning-based information measure. This measure is high when a learning system is not familiar with a given data point, so the learning activity to learn novel data points is unusual (regardless of the prediction error), i.e., the learning increments display unusual patterns during adaptation. In this paper, we propose concepts of the learning state and the learning state space so that LE can be approximated via neighbourhood analysis in the learning space. Further, two novel clustering-based techniques for approximation of sample point LE are proposed. The first one is based on the sum of K nearest neighbour distances. The second one is based on multiscale neighbourhood cumulative sum. Also, we preprocess the learning space with dimensionality reduction that is promising for research of LE even with neural networks and potentially with deep neural networks. The performance of novelty detection with the clustering-based sample point LE with dimensionality reduction is compared to the original algorithms of LE, and its potentials are discussed.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF16_019%2F0000753" target="_blank" >EF16_019/0000753: Centrum výzkumu nízkouhlíkových energetických technologií</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2020 SENSOR SIGNAL PROCESSING FOR DEFENCE CONFERENCE (SSPD)

ISBN

9781728138107

ISSN

—

e-ISSN

—

Počet stran výsledku

5

Strana od-do

—

Název nakladatele

IEEE Industrial Electronic Society

Místo vydání

???

Místo konání akce

Edinburgh

Datum konání akce

14. 12. 2020

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—