Wandering around: a bioinspired approach to visual attention through object motion sensitivity

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F25%3A00384001" target="_blank" >RIV/68407700:21230/25:00384001 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1088/2634-4386/addc90" target="_blank" >https://doi.org/10.1088/2634-4386/addc90</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1088/2634-4386/addc90" target="_blank" >10.1088/2634-4386/addc90</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Wandering around: a bioinspired approach to visual attention through object motion sensitivity

Popis výsledku v původním jazyce

Active vision enables dynamic and robust visual perception, offering an alternative to the static, passive nature of feedforward architectures commonly used in computer vision, which depend on large datasets and high computational resources. Biological selective attention mechanisms allow agents to focus on salient regions of interest (ROIs), reducing computational demand while maintaining real-time responsiveness. Event-based cameras, inspired by the mammalian retina, further enhance this capability by capturing asynchronous scene changes, enabling efficient, low-latency processing. To distinguish moving objects while the event-based camera is also in motion, the agent requires an object motion segmentation mechanism to accurately detect targets and position them at the centre of the visual field (fovea). Integrating event-based sensors with neuromorphic algorithms represents a paradigm shift, using spiking neural networks (SNNs) to parallelise computation and adapt to dynamic environments. This work presents a spiking convolutional neural network bioinspired attention system for selective attention through object motion sensitivity. The system generates events via fixational eye movements using a dynamic vision sensor integrated into the Speck neuromorphic hardware, mounted on a Pan–Tilt unit, to identify the ROI and saccade toward it. The system, characterised using ideal gratings and benchmarked against the event camera motion segmentation dataset, reaches a mean IoU of 82.2% and a mean structural similarity index of 96% in multi-object motion segmentation. Additionally, the detection of salient objects reaches an accuracy of 88.8% in office scenarios and 89.8% in challenging indoor and outdoor low-light conditions, as evaluated on the event-assisted low-light video object segmentation dataset. A real-time demonstrator showcases the system’s capabilities of detecting the salient object through object motion sensitivity in 0.124 s in dynamic scenes. Its learning-free design ensures robustness across diverse perceptual scenes, making it a reliable foundation for real-time robotic applications and serving as a basis for more complex architectures.

Název v anglickém jazyce

Wandering around: a bioinspired approach to visual attention through object motion sensitivity

Popis výsledku anglicky

Active vision enables dynamic and robust visual perception, offering an alternative to the static, passive nature of feedforward architectures commonly used in computer vision, which depend on large datasets and high computational resources. Biological selective attention mechanisms allow agents to focus on salient regions of interest (ROIs), reducing computational demand while maintaining real-time responsiveness. Event-based cameras, inspired by the mammalian retina, further enhance this capability by capturing asynchronous scene changes, enabling efficient, low-latency processing. To distinguish moving objects while the event-based camera is also in motion, the agent requires an object motion segmentation mechanism to accurately detect targets and position them at the centre of the visual field (fovea). Integrating event-based sensors with neuromorphic algorithms represents a paradigm shift, using spiking neural networks (SNNs) to parallelise computation and adapt to dynamic environments. This work presents a spiking convolutional neural network bioinspired attention system for selective attention through object motion sensitivity. The system generates events via fixational eye movements using a dynamic vision sensor integrated into the Speck neuromorphic hardware, mounted on a Pan–Tilt unit, to identify the ROI and saccade toward it. The system, characterised using ideal gratings and benchmarked against the event camera motion segmentation dataset, reaches a mean IoU of 82.2% and a mean structural similarity index of 96% in multi-object motion segmentation. Additionally, the detection of salient objects reaches an accuracy of 88.8% in office scenarios and 89.8% in challenging indoor and outdoor low-light conditions, as evaluated on the event-assisted low-light video object segmentation dataset. A real-time demonstrator showcases the system’s capabilities of detecting the salient object through object motion sensitivity in 0.124 s in dynamic scenes. Its learning-free design ensures robustness across diverse perceptual scenes, making it a reliable foundation for real-time robotic applications and serving as a basis for more complex architectures.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EH22_008%2F0004590" target="_blank" >EH22_008/0004590: Robotika a pokročilá průmyslová výroba</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Neuromorphic Computing and Engineering

ISSN

2634-4386

e-ISSN

2634-4386

Svazek periodika

5

Číslo periodika v rámci svazku

2

Stát vydavatele periodika

GB - Spojené království Velké Británie a Severního Irska

Počet stran výsledku

25

Strana od-do

—

Kód UT WoS článku

001511110000001

EID výsledku v databázi Scopus

2-s2.0-105008463792