Large Language Models and the Extended Church-Turing Thesis

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00598590" target="_blank" >RIV/67985807:_____/24:00598590 - isvavai.cz</a>

Výsledek na webu

<a href="https://cgi.cse.unsw.edu.au/~eptcs/paper.cgi?NCMA2024.14" target="_blank" >https://cgi.cse.unsw.edu.au/~eptcs/paper.cgi?NCMA2024.14</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.4204/EPTCS.407.14" target="_blank" >10.4204/EPTCS.407.14</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Large Language Models and the Extended Church-Turing Thesis

Popis výsledku v původním jazyce

The Extended Church-Turing Thesis (ECTT) posits that all effective information processing, including unbounded and non-uniform interactive computations, can be described in terms of interactive Turing machines with advice. Does this assertion also apply to the abilities of contemporary large language models (LLMs)? From a broader perspective, this question calls for an investigation of the computational power of LLMs by the classical means of computability and computational complexity theory, especially the theory of automata. Along these lines, we establish a number of fundamental results. Firstly, we argue that any fixed (non-adaptive) LLM is computationally equivalent to a, possibly very large, deterministic finite-state transducer. This characterizes the base level of LLMs. We extend this to a key result concerning the simulation of space-bounded Turing machines by LLMs. Secondly, we show that lineages of evolving LLMs are computationally equivalent to interactive Turing machines with advice. The latter finding confirms the validity of the ECTT for lineages of LLMs. From a computability viewpoint, it also suggests that lineages of LLMs possess super-Turing computational power. Consequently, in our computational model knowledge generation is in general a non-algorithmic process realized by lineages of LLMs. Finally, we discuss the merits of our findings in the broader context of several related disciplines and philosophies.

Název v anglickém jazyce

Large Language Models and the Extended Church-Turing Thesis

Popis výsledku anglicky

The Extended Church-Turing Thesis (ECTT) posits that all effective information processing, including unbounded and non-uniform interactive computations, can be described in terms of interactive Turing machines with advice. Does this assertion also apply to the abilities of contemporary large language models (LLMs)? From a broader perspective, this question calls for an investigation of the computational power of LLMs by the classical means of computability and computational complexity theory, especially the theory of automata. Along these lines, we establish a number of fundamental results. Firstly, we argue that any fixed (non-adaptive) LLM is computationally equivalent to a, possibly very large, deterministic finite-state transducer. This characterizes the base level of LLMs. We extend this to a key result concerning the simulation of space-bounded Turing machines by LLMs. Secondly, we show that lineages of evolving LLMs are computationally equivalent to interactive Turing machines with advice. The latter finding confirms the validity of the ECTT for lineages of LLMs. From a computability viewpoint, it also suggests that lineages of LLMs possess super-Turing computational power. Consequently, in our computational model knowledge generation is in general a non-algorithmic process realized by lineages of LLMs. Finally, we discuss the merits of our findings in the broader context of several related disciplines and philosophies.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/CK04000150" target="_blank" >CK04000150: Hodnocení chování automatizovaných vozidel z pohledu dodržování etických a právních principů ve smíšeném provozu</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings 14th International Workshop on Non-Classical Models of Automata and Applications (NCMA 2024)

ISBN

—

ISSN

2075-2180

e-ISSN

—

Počet stran výsledku

16

Strana od-do

198-213

Název nakladatele

Open Publishing Association

Místo vydání

Waterloo

Místo konání akce

Göttingen

Datum konání akce

12. 8. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001318849600015