Learning to Optimize with Dynamic Mode Decomposition

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21240%2F22%3A00360159" target="_blank" >RIV/68407700:21240/22:00360159 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1109/IJCNN55064.2022.9892364" target="_blank" >https://doi.org/10.1109/IJCNN55064.2022.9892364</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IJCNN55064.2022.9892364" target="_blank" >10.1109/IJCNN55064.2022.9892364</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Learning to Optimize with Dynamic Mode Decomposition

Popis výsledku v původním jazyce

Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.

Název v anglickém jazyce

Learning to Optimize with Dynamic Mode Decomposition

Popis výsledku anglicky

Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effectively include the dynamics of the optimization process during training. They either omit it entirely or only implicitly assume the dynamics of an isolated parameter. In this paper, we show how to utilize the dynamic mode decomposition method for extracting informative features about optimization dynamics. By employing those features, we show that our learned optimizer generalizes much better to unseen optimization problems in short. The improved generalization is illustrated on multiple tasks where training the optimizer on one neural network generalizes to different architectures and distinct datasets.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2022 International Joint Conference on Neural Networks (IJCNN)

ISBN

978-1-7281-8671-9

ISSN

—

e-ISSN

2161-4407

Počet stran výsledku

8

Strana od-do

1-8

Název nakladatele

IEEE Industrial Electronic Society

Místo vydání

Vienna

Místo konání akce

Padua

Datum konání akce

18. 7. 2022

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000867070903123