A Multimodal Connectionist Architecture for Unsupervised Grounding of Spatial Language

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F14%3A00212230" target="_blank" >RIV/68407700:21230/14:00212230 - isvavai.cz</a>

Výsledek na webu

<a href="http://link.springer.com/article/10.1007/s12559-013-9212-5" target="_blank" >http://link.springer.com/article/10.1007/s12559-013-9212-5</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s12559-013-9212-5" target="_blank" >10.1007/s12559-013-9212-5</a>

Jazyk výsledku

angličtina

Název v původním jazyce

A Multimodal Connectionist Architecture for Unsupervised Grounding of Spatial Language

Popis výsledku v původním jazyce

We propose a bio-inspired unsupervised connectionist architecture and apply it to grounding the spatial phrases. The two-layer architecture combines by concatenation the information from the visual and the phonological inputs. In the first layer, the visual pathway employs separate `what? and `where? subsystems that represent the identity and spatial relations of two objects in 2D space, respectively. The bitmap images are presented to an artificial retina and the phonologically encoded five-word sentences describing the image serve as the phonological input. The visual scene is hence represented by several self-organizing maps (SOMs) and the phonological description is processed by the Recursive SOM that learns to topographically represent the spatialphrases, represented as five-word sentences (e.g., `blue ball above red cup?). Primary representations from the first-layer modules are unambiguously integrated in a multimodal second-layer module, implemented by the SOM or the `neural g

Název v anglickém jazyce

A Multimodal Connectionist Architecture for Unsupervised Grounding of Spatial Language

Popis výsledku anglicky

We propose a bio-inspired unsupervised connectionist architecture and apply it to grounding the spatial phrases. The two-layer architecture combines by concatenation the information from the visual and the phonological inputs. In the first layer, the visual pathway employs separate `what? and `where? subsystems that represent the identity and spatial relations of two objects in 2D space, respectively. The bitmap images are presented to an artificial retina and the phonologically encoded five-word sentences describing the image serve as the phonological input. The visual scene is hence represented by several self-organizing maps (SOMs) and the phonological description is processed by the Recursive SOM that learns to topographically represent the spatialphrases, represented as five-word sentences (e.g., `blue ball above red cup?). Primary representations from the first-layer modules are unambiguously integrated in a multimodal second-layer module, implemented by the SOM or the `neural g

Druh

J_x - Nezařazeno - Článek v odborném periodiku (Jimp, Jsc a Jost)

CEP obor

AN - Psychologie

OECD FORD obor

—

Projekt

<a href="/cs/project/GPP407%2F11%2FP696" target="_blank" >GPP407/11/P696: Referenční rámce v 3D prostředí a jejich EEG analýza</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2014

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Cognitive Computation

ISSN

1866-9956

e-ISSN

—

Svazek periodika

6

Číslo periodika v rámci svazku

1

Stát vydavatele periodika

DE - Spolková republika Německo

Počet stran výsledku

12

Strana od-do

101-112

Kód UT WoS článku

000332454800008

EID výsledku v databázi Scopus

—