Closed Loop Interactive Embodied Reasoning for Robot Manipulation

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00381860" target="_blank" >RIV/68407700:21230/25:00381860 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/68407700:21730/25:00381860

Výsledek na webu

<a href="https://doi.org/10.1109/ICRA55743.2025.11127480" target="_blank" >https://doi.org/10.1109/ICRA55743.2025.11127480</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/ICRA55743.2025.11127480" target="_blank" >10.1109/ICRA55743.2025.11127480</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Closed Loop Interactive Embodied Reasoning for Robot Manipulation

Popis výsledku v původním jazyce

Embodied reasoning systems integrate robotic hardware and cognitive processes to perform complex tasks, typically in response to a natural language query about a specific physical environment. This usually involves changing the belief about the scene or physically interacting and changing the scene (e.g. sort the objects from lightest to heaviest). In order to facilitate the development of such systems we introduce a new modular Closed Loop Interactive Embodied Reasoning (CLIER) approach that takes into account the measurements of non-visual object properties, changes in the scene caused by external disturbances as well as uncertain outcomes of robotic actions. CLIER performs multi-modal reasoning and action planning and generates a sequence of primitive actions that can be executed by a robot manipulator. Our method operates in a closed loop, responding to changes in the environment. Our approach is developed with the use of MuBle simulation environment and tested in 10 interactive benchmark scenarios. We extensively evaluate our reasoning approach in simulation and in real-world manipulation tasks with a success rate above 76% and 64%, respectively.

Název v anglickém jazyce

Closed Loop Interactive Embodied Reasoning for Robot Manipulation

Popis výsledku anglicky

Embodied reasoning systems integrate robotic hardware and cognitive processes to perform complex tasks, typically in response to a natural language query about a specific physical environment. This usually involves changing the belief about the scene or physically interacting and changing the scene (e.g. sort the objects from lightest to heaviest). In order to facilitate the development of such systems we introduce a new modular Closed Loop Interactive Embodied Reasoning (CLIER) approach that takes into account the measurements of non-visual object properties, changes in the scene caused by external disturbances as well as uncertain outcomes of robotic actions. CLIER performs multi-modal reasoning and action planning and generates a sequence of primitive actions that can be executed by a robot manipulator. Our method operates in a closed loop, responding to changes in the environment. Our approach is developed with the use of MuBle simulation environment and tested in 10 interactive benchmark scenarios. We extensively evaluate our reasoning approach in simulation and in real-world manipulation tasks with a success rate above 76% and 64%, respectively.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

2025 IEEE International Conference on Robotics and Automation (ICRA)

ISBN

979-8-3315-4140-8

ISSN

1050-4729

e-ISSN

1050-4729

Počet stran výsledku

8

Strana od-do

13722-13729

Název nakladatele

IEEE Industrial Electronic Society

Místo vydání

Vienna

Místo konání akce

Atlanta

Datum konání akce

19. 5. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001614889900214