One-Counter Markov Decision Processes
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F10%3A00043501" target="_blank" >RIV/00216224:14330/10:00043501 - isvavai.cz</a>
Result on the web
—
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
One-Counter Markov Decision Processes
Original language description
We study One-Counter Markov Decision Processes (OC-MDPs), which extend finite-state MDPs with an unbounded counter. The counter can be incremented, decremented, or not changed during each state transition. Basic objectives for OC-MDPs include ``termination'' (Does the OC-MDP reach counter 0?) and ``limit'' questions (Is the limsup value infinity?). We may ask what is the optimal probability of such objectives, or ask for the existence and synthesis of optimal strategies. We show that several quantitative and almost-sure limit problems can be answered in polynomial time, and that almost-sure termination problems (without selection of desired terminal states) can also be answered in polynomial time. On the other hand, we show that the almost-sure termination problem with selected terminal states is PSPACE-hard and we provide an exponential time algorithm for this problem. We also characterize classes of strategies that suffice for optimality in several of these settings.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
IN - Informatics
OECD FORD branch
—
Result continuities
Project
<a href="/en/project/1M0545" target="_blank" >1M0545: Institute for Theoretical Computer Science</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>Z - Vyzkumny zamer (s odkazem do CEZ)
Others
Publication year
2010
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms
ISBN
978-0-89871-698-6
ISSN
—
e-ISSN
—
Number of pages
12
Pages from-to
—
Publisher name
SIAM
Place of publication
Neuveden
Event location
Austin (Texas, USA)
Event date
Jan 1, 2010
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000280699900070