Policy derivation methods for critic-only reinforcement learning in continuous spaces

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F18%3A00316441" target="_blank" >RIV/68407700:21730/18:00316441 - isvavai.cz</a>
Result on the web
<a href="https://www.sciencedirect.com/science/article/pii/S0952197617302993" target="_blank" >https://www.sciencedirect.com/science/article/pii/S0952197617302993</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1016/j.engappai.2017.12.004" target="_blank" >10.1016/j.engappai.2017.12.004</a>

Result language
angličtina
Original language name
Policy derivation methods for critic-only reinforcement learning in continuous spaces
Original language description
This paper addresses the problem of deriving a policy from the value function in the context of critic-only reinforcement learning (RL) in continuous state and action spaces. With continuous-valued states, RL algorithms have to rely on a numerical approximator to represent the value function. Numerical approximation due to its nature virtually always exhibits artifacts which damage the overall performance of the controlled system. In addition, when continuous-valued action is used, the most common approach is to discretize the action space and exhaustively search for the action that maximizes the right-hand side of the Bellman equation. Such a policy derivation procedure is computationally involved and results in steady-state error due to the lack of continuity. In this work, we propose policy derivation methods which alleviate the above problems by means of action space refinement, continuous approximation, and post-processing of the V-function by using symbolic regression. The proposed methods are tested on nonlinear control problems: 1-DOF and 2-DOF pendulum swing-up problems, and on magnetic manipulation. The results show significantly improved performance in terms of cumulative return and computational complexity.
Czech name
—
Czech description
—

Type
J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database
CEP classification
—
OECD FORD branch
20205 - Automation and control systems

Project
<a href="/en/project/GA15-22731S" target="_blank" >GA15-22731S: Symbolic Regression for Reinforcement Learning in Continuous Spaces</a><br>
Continuities
S - Specificky vyzkum na vysokych skolach

Publication year
2018
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)