Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processes

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F20%3A00114279" target="_blank" >RIV/00216224:14330/20:00114279 - isvavai.cz</a>
Result on the web
<a href="https://aaai.org/ojs/index.php/AAAI/article/view/6531" target="_blank" >https://aaai.org/ojs/index.php/AAAI/article/view/6531</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1609/aaai.v34i06.6531" target="_blank" >10.1609/aaai.v34i06.6531</a>

Result language
angličtina
Original language name
Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processes
Original language description
Markov decision processes (MDPs) are the defacto framework for sequential decision making in the presence of stochastic uncertainty. A classical optimization criterion for MDPs is to maximize the expected discounted-sum payoff, which ignores low probability catastrophic events with highly negative impact on the system. On the other hand, risk-averse policies require the probability of undesirable events to be below a given threshold, but they do not account for optimization of the expected payoff. We consider MDPs with discounted-sum payoff with failure states which represent catastrophic outcomes. The objective of risk-constrained planning is to maximize the expected discounted-sum payoff among risk-averse policies that ensure the probability to encounter a failure state is below a desired threshold. Our main contribution is an efficient risk-constrained planning algorithm that combines UCT-like search with a predictor learned through interaction with the MDP (in the style of AlphaZero) and with a risk-constrained action selection via linear programming. We demonstrate the effectiveness of our approach with experiments on classical MDPs from the literature, including benchmarks with an order of 10^6 states.
Czech name
—
Czech description
—

Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Publication year
2020
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020
ISBN
9781577358237
ISSN
—
e-ISSN
—
Number of pages
8
Pages from-to
9794-9801
Publisher name
AAAI Press
Place of publication
Palo Alto, California, USA
Event location
New York
Event date
Feb 7, 2020
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—

Similar results(10)