A Multimodal Connectionist Architecture for Unsupervised Grounding of Spatial Language

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F14%3A00212230" target="_blank" >RIV/68407700:21230/14:00212230 - isvavai.cz</a>

Result on the web

<a href="http://link.springer.com/article/10.1007/s12559-013-9212-5" target="_blank" >http://link.springer.com/article/10.1007/s12559-013-9212-5</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s12559-013-9212-5" target="_blank" >10.1007/s12559-013-9212-5</a>

Result language

angličtina

Original language name

A Multimodal Connectionist Architecture for Unsupervised Grounding of Spatial Language

Original language description

We propose a bio-inspired unsupervised connectionist architecture and apply it to grounding the spatial phrases. The two-layer architecture combines by concatenation the information from the visual and the phonological inputs. In the first layer, the visual pathway employs separate `what? and `where? subsystems that represent the identity and spatial relations of two objects in 2D space, respectively. The bitmap images are presented to an artificial retina and the phonologically encoded five-word sentences describing the image serve as the phonological input. The visual scene is hence represented by several self-organizing maps (SOMs) and the phonological description is processed by the Recursive SOM that learns to topographically represent the spatialphrases, represented as five-word sentences (e.g., `blue ball above red cup?). Primary representations from the first-layer modules are unambiguously integrated in a multimodal second-layer module, implemented by the SOM or the `neural g

Czech name

—

Czech description

—

Type

J_x - Unclassified - Peer-reviewed scientific article (Jimp, Jsc and Jost)

CEP classification

AN - Psychology

OECD FORD branch

—

Project

<a href="/en/project/GPP407%2F11%2FP696" target="_blank" >GPP407/11/P696: Frames of Reference Processing in the 3D Environment and its EEG Analysis</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2014

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Cognitive Computation

ISSN

1866-9956

e-ISSN

—

Volume of the periodical

6

Issue of the periodical within the volume

1

Country of publishing house

DE - GERMANY

Number of pages

12

Pages from-to

101-112

UT code for WoS article

000332454800008

EID of the result in the Scopus database

—