Will multimodal large language models ever achieve deep understanding of the world?

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F25%3A00387695" target="_blank" >RIV/68407700:21730/25:00387695 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.3389/fnsys.2025.1683133" target="_blank" >https://doi.org/10.3389/fnsys.2025.1683133</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.3389/fnsys.2025.1683133" target="_blank" >10.3389/fnsys.2025.1683133</a>

Result language

angličtina

Original language name

Will multimodal large language models ever achieve deep understanding of the world?

Original language description

Despite impressive performance in various tasks, large language models (LLMs) are subject to the symbol grounding problem, so from the cognitive science perspective, one can argue that they are merely statistics-driven distributional models without a deeper understanding. Modern multimodal versions of LLMs (MLLMs) are trying to avoid this problem by linking language knowledge with other modalities such as vision (Vision Language Models called VLM) or action (Vision Language Action Models called VLA) when, for instance, a robotic agent, is acting in the world. If eventually successful, MLLMs could be taken as pathway for symbol grounding. In this work, we explore the extent to which MLLMs integrated with embodied agents can achieve such grounded understanding through interaction with the physical world. We argue that closing the gap between symbolic tokens, neural representations, and embodied experience will require deeper developmental integration of continuous sensory data, goal-directed behavior, and adaptive neural learning in real-world environments. We raise a concern that MLLMs do not currently achieve a human-like level of deep understanding, largely because their random learning trajectory deviates significantly from human cognitive development. Humans typically acquire knowledge incrementally, building complex concepts upon simpler ones in a structured developmental progression. In contrast, MLLMs are often trained on vast, randomly ordered datasets. This non-developmental approach, which circumvents a structured simple-to-complex conceptual scaffolding, inhibits the ability to build a deep and meaningful grounded knowledge base, posing a significant challenge to achieving human-like semantic comprehension.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GF23-04080L" target="_blank" >GF23-04080L: Intuitive Collaboration with Household Robots in Everyday Settings</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Frontiers in systems neuroscience

ISSN

1662-5137

e-ISSN

1662-5137

Volume of the periodical

19

Issue of the periodical within the volume

1683133

Country of publishing house

CH - SWITZERLAND

Number of pages

4

Pages from-to

—

UT code for WoS article

001631126800001

EID of the result in the Scopus database

—