Shielding in Resource-Constrained Goal POMDPs

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F23%3A00131270" target="_blank" >RIV/00216224:14330/23:00131270 - isvavai.cz</a>

Výsledek na webu

<a href="https://ojs.aaai.org/index.php/AAAI/article/view/26715" target="_blank" >https://ojs.aaai.org/index.php/AAAI/article/view/26715</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1609/aaai.v37i12.26715" target="_blank" >10.1609/aaai.v37i12.26715</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Shielding in Resource-Constrained Goal POMDPs

Popis výsledku v původním jazyce

We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.

Název v anglickém jazyce

Shielding in Resource-Constrained Goal POMDPs

Popis výsledku anglicky

We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by the agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10200 - Computer and information sciences

Projekt

<a href="/cs/project/GA21-24711S" target="_blank" >GA21-24711S: Efektivní analýza a optimalizace pravděpodobnostních systémů a her</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 37th AAAI Conference on Artificial Intelligence

ISBN

9781577358800

ISSN

2159-5399

e-ISSN

2374-3468

Počet stran výsledku

9

Strana od-do

14674-14682

Název nakladatele

AAAI Press

Místo vydání

Washington, DC, USA

Místo konání akce

Washington, DC, USA

Datum konání akce

1. 1. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—