How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10490653" target="_blank" >RIV/00216208:11320/25:10490653 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1007/978-3-031-75107-3_13" target="_blank" >https://doi.org/10.1007/978-3-031-75107-3_13</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-75107-3_13" target="_blank" >10.1007/978-3-031-75107-3_13</a>

Jazyk výsledku

angličtina

Název v původním jazyce

How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures

Popis výsledku v původním jazyce

Ensemble-based component systems have been used for many years to develop collective adaptive systems (CAS). The DEECo component model offers a framework for modeling and implementing ensemble-based component systems. Being expressive enough and having semantics specifically tailored towards dynamically evolving systems, DEECo has proven to be fairly powerful in modeling complex and dynamic architectures. At the same time, its specific semantics turned out to be a hurdle for newcomers when expressing their design intention and understanding its implications and side effects. We see quite a potential in employing large language models (LLMs) to simplify creating and refining the DEECo architectures. Since this constitutes a large research scope, we focus in this paper on initial experiments demonstrating how well generic LLMs understand the advanced concepts of ensemble-based CAS embodied in DEECo and asses what expectation from LLMs is realistic in this context. Our results indicate that LLMs can indeed understand ensemble-based architectures, but it depends on the form in which the architecture is presented in the textual form. Using external DSL, which is very self-explanatory gave good results out of the box. Specifications embedded in existing programming languages needed prior explanation of how to interpret them.

Název v anglickém jazyce

How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures

Popis výsledku anglicky

Ensemble-based component systems have been used for many years to develop collective adaptive systems (CAS). The DEECo component model offers a framework for modeling and implementing ensemble-based component systems. Being expressive enough and having semantics specifically tailored towards dynamically evolving systems, DEECo has proven to be fairly powerful in modeling complex and dynamic architectures. At the same time, its specific semantics turned out to be a hurdle for newcomers when expressing their design intention and understanding its implications and side effects. We see quite a potential in employing large language models (LLMs) to simplify creating and refining the DEECo architectures. Since this constitutes a large research scope, we focus in this paper on initial experiments demonstrating how well generic LLMs understand the advanced concepts of ensemble-based CAS embodied in DEECo and asses what expectation from LLMs is realistic in this context. Our results indicate that LLMs can indeed understand ensemble-based architectures, but it depends on the form in which the architecture is presented in the textual form. Using external DSL, which is very self-explanatory gave good results out of the box. Specifications embedded in existing programming languages needed prior explanation of how to interpret them.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ISBN

978-3-031-75106-6

ISSN

—

e-ISSN

1611-3349

Počet stran výsledku

16

Strana od-do

208-223

Název nakladatele

Springer

Místo vydání

Cham, Germany

Místo konání akce

Crete, Greece

Datum konání akce

27. 10. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001419019500013