Geometric Reasoning in the Embedding Space

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F25%3A00387606" target="_blank" >RIV/68407700:21730/25:00387606 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/61988987:17610/25:A2603BLJ

Výsledek na webu

<a href="https://doi.org/10.3390/make7030093" target="_blank" >https://doi.org/10.3390/make7030093</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.3390/make7030093" target="_blank" >10.3390/make7030093</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Geometric Reasoning in the Embedding Space

Popis výsledku v původním jazyce

While neural networks can solve complex geometric problems, as demonstrated by systems like AlphaGeometry, we have limited understanding of how they internally represent and reason about spatial relationships. In this work, we investigate how neural networks develop internal spatial understanding by training Graph Neural Networks and Transformers to predict point positions on a discrete 2D grid from geometric constraints that describe hidden figures. We show that both models develop interpretable internal representations that mirror the geometric structure of the problems they solve. Specifically, we observe that point embeddings self-organize into 2D grid structures during training, and during inference, the models iteratively construct the hidden geometric figures within their embedding spaces. Our analysis reveals how reasoning complexity correlates with prediction accuracy, and shows that models solve constraints through an iterative refinement process, which might resemble continuous optimization. We also find that Graph Neural Networks prove more suitable than Transformers for this type of structured constraint reasoning and scale more effectively to larger problems. These findings provide initial insights into how neural networks can develop structured understanding and contribute to their interpretability.

Název v anglickém jazyce

Geometric Reasoning in the Embedding Space

Popis výsledku anglicky

While neural networks can solve complex geometric problems, as demonstrated by systems like AlphaGeometry, we have limited understanding of how they internally represent and reason about spatial relationships. In this work, we investigate how neural networks develop internal spatial understanding by training Graph Neural Networks and Transformers to predict point positions on a discrete 2D grid from geometric constraints that describe hidden figures. We show that both models develop interpretable internal representations that mirror the geometric structure of the problems they solve. Specifically, we observe that point embeddings self-organize into 2D grid structures during training, and during inference, the models iteratively construct the hidden geometric figures within their embedding spaces. Our analysis reveals how reasoning complexity correlates with prediction accuracy, and shows that models solve constraints through an iterative refinement process, which might resemble continuous optimization. We also find that Graph Neural Networks prove more suitable than Transformers for this type of structured constraint reasoning and scale more effectively to larger problems. These findings provide initial insights into how neural networks can develop structured understanding and contribute to their interpretability.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Machine Learning and Knowledge Extraction

ISSN

2504-4990

e-ISSN

2504-4990

Svazek periodika

7

Číslo periodika v rámci svazku

3

Stát vydavatele periodika

CH - Švýcarská konfederace

Počet stran výsledku

34

Strana od-do

—

Kód UT WoS článku

001580457600001

EID výsledku v databázi Scopus

2-s2.0-105017425258