How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10490653" target="_blank" >RIV/00216208:11320/25:10490653 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1007/978-3-031-75107-3_13" target="_blank" >https://doi.org/10.1007/978-3-031-75107-3_13</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-031-75107-3_13" target="_blank" >10.1007/978-3-031-75107-3_13</a>

Result language

angličtina

Original language name

How Well Do LLMs Understand DEECo Ensemble-Based Component Architectures

Original language description

Ensemble-based component systems have been used for many years to develop collective adaptive systems (CAS). The DEECo component model offers a framework for modeling and implementing ensemble-based component systems. Being expressive enough and having semantics specifically tailored towards dynamically evolving systems, DEECo has proven to be fairly powerful in modeling complex and dynamic architectures. At the same time, its specific semantics turned out to be a hurdle for newcomers when expressing their design intention and understanding its implications and side effects. We see quite a potential in employing large language models (LLMs) to simplify creating and refining the DEECo architectures. Since this constitutes a large research scope, we focus in this paper on initial experiments demonstrating how well generic LLMs understand the advanced concepts of ensemble-based CAS embodied in DEECo and asses what expectation from LLMs is realistic in this context. Our results indicate that LLMs can indeed understand ensemble-based architectures, but it depends on the form in which the architecture is presented in the textual form. Using external DSL, which is very self-explanatory gave good results out of the box. Specifications embedded in existing programming languages needed prior explanation of how to interpret them.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ISBN

978-3-031-75106-6

ISSN

—

e-ISSN

1611-3349

Number of pages

16

Pages from-to

208-223

Publisher name

Springer

Place of publication

Cham, Germany

Event location

Crete, Greece

Event date

Oct 27, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

001419019500013