Reinforcement Learning with Symbolic Input-Output Models

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F18%3A00324409" target="_blank" >RIV/68407700:21230/18:00324409 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/68407700:21730/18:00324409

Výsledek na webu

<a href="http://dx.doi.org/10.1109/IROS.2018.8593881" target="_blank" >http://dx.doi.org/10.1109/IROS.2018.8593881</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IROS.2018.8593881" target="_blank" >10.1109/IROS.2018.8593881</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Reinforcement Learning with Symbolic Input-Output Models

Popis výsledku v původním jazyce

It is well known that reinforcement learning (RL) can benefit from the use of a dynamic prediction model which is learned on data samples collected online from the process to be controlled. Most RL algorithms are formulated in the statespace domain and use state-space models. However, learning state-space models is difficult, mainly because in the vast majority of problems the full state cannot be measured on the system or reconstructed from the measurements. To circumvent this limitation, we propose to use input–output models of the NARX (nonlinear autoregressive with exogenous input) type. Symbolic regression is employed to construct parsimonious models and the corresponding value functions. Thanks to this approach, we can learn accurate models and compute optimal policies even from small amounts of training data. We demonstrate the approach on two simulated examples, a hopping robot and a 1-DOF robot arm, and on a real inverted pendulum system. Results show that our proposed method can reliably determine a good control policy based on a symbolic input–output process model and value function.

Název v anglickém jazyce

Reinforcement Learning with Symbolic Input-Output Models

Popis výsledku anglicky

It is well known that reinforcement learning (RL) can benefit from the use of a dynamic prediction model which is learned on data samples collected online from the process to be controlled. Most RL algorithms are formulated in the statespace domain and use state-space models. However, learning state-space models is difficult, mainly because in the vast majority of problems the full state cannot be measured on the system or reconstructed from the measurements. To circumvent this limitation, we propose to use input–output models of the NARX (nonlinear autoregressive with exogenous input) type. Symbolic regression is employed to construct parsimonious models and the corresponding value functions. Thanks to this approach, we can learn accurate models and compute optimal policies even from small amounts of training data. We demonstrate the approach on two simulated examples, a hopping robot and a 1-DOF robot arm, and on a real inverted pendulum system. Results show that our proposed method can reliably determine a good control policy based on a symbolic input–output process model and value function.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

20204 - Robotics and automatic control

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2018

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

ISBN

978-1-5386-8094-0

ISSN

2153-0858

e-ISSN

2153-0866

Počet stran výsledku

6

Strana od-do

3004-3009

Název nakladatele

IEEE Press

Místo vydání

New York

Místo konání akce

Madrid

Datum konání akce

1. 10. 2018

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

000458872702122