Robust Finite-Memory Policy Gradients for Hidden-Model POMDPs

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F26%3A0198874" target="_blank" >RIV/00216305:26230/26:0198874 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.24963/ijcai.2025/947" target="_blank" >https://doi.org/10.24963/ijcai.2025/947</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.24963/ijcai.2025/947" target="_blank" >10.24963/ijcai.2025/947</a>

Result language

angličtina

Original language name

Robust Finite-Memory Policy Gradients for Hidden-Model POMDPs

Original language description

Partially observable Markov decision processes (POMDPs) model specific environments in sequential decision-making under uncertainty. Critically, optimal policies for POMDPs may not be robust against perturbations in the environment. Hidden-model POMDPs (HM-POMDPs) capture sets of different environment models, that is, POMDPs with a shared action and observation space. The intuition is that the true model is hidden among a set of potential models, and it is unknown which model will be the environment at execution time. A policy is robust for a given HM-POMDP if it achieves sufficient performance for each of its POMDPs. We compute such robust policies by combining two orthogonal techniques: (1) a deductive formal verification technique that supports tractable robust policy evaluation by computing a worst-case POMDP within the HM-POMDP, and (2) subgradient ascent to optimize the candidate policy for a worst-case POMDP. The empirical evaluation shows that, compared to various baselines, our approach (1) produces policies that are more robust and generalize better to unseen POMDPs, and (2) scales to HM-POMDPs that consist of over a hundred thousand environments.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GA23-06963S" target="_blank" >GA23-06963S: VESCAA: Verifiable and Efficient Synthesis of Controllers for Autonomous Agents</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence

ISBN

978-1-956792-06-5

ISSN

—

e-ISSN

—

Number of pages

9

Pages from-to

8518-8526

Publisher name

International Joint Conferences on Artificial Intelligence Organization

Place of publication

—

Event location

Montreal, Canada

Event date

Aug 16, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001634925300305