Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F60461373%3A22340%2F25%3A43933083" target="_blank" >RIV/60461373:22340/25:43933083 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/content/pdf/10.1007/978-3-032-02936-2.pdf" target="_blank" >https://link.springer.com/content/pdf/10.1007/978-3-032-02936-2.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-032-02936-2_25" target="_blank" >10.1007/978-3-032-02936-2_25</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education

Popis výsledku v původním jazyce

As Large Language Models (LLMs) become increasingly available to students, BPM educators face a new challenge: how to design assignments that remain pedagogically effective and resistant to superficial AI-generated answers. This paper presents the results of two experiments that simulate common BPMN-related homework tasks and test how LLMs respond to them. The first experiment focused on answering comprehension questions based on five BPMN models, each provided either in PNG or XML format. The second asked the models to detect modeling errors in 30 flawed BPMN diagrams. In both cases, we evaluated the outputs of ChatGPT-4o and Gemini Flash, analyzing the correctness, reasoning, and completeness of their responses. Our findings show that while current LLMs are not yet fully capable of reliably solving BPMN assignments-especially those involving deeper process logic-they can already provide partially correct and plausible responses. This raises questions about the future of BPM education, the design of AI-aware assignments, and the role of LLMs as potential learning assistants. Alongside insights and lessons learned, we provide materials to help instructors adapt their teaching to an AI-enabled environment.

Název v anglickém jazyce

Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education

Popis výsledku anglicky

As Large Language Models (LLMs) become increasingly available to students, BPM educators face a new challenge: how to design assignments that remain pedagogically effective and resistant to superficial AI-generated answers. This paper presents the results of two experiments that simulate common BPMN-related homework tasks and test how LLMs respond to them. The first experiment focused on answering comprehension questions based on five BPMN models, each provided either in PNG or XML format. The second asked the models to detect modeling errors in 30 flawed BPMN diagrams. In both cases, we evaluated the outputs of ChatGPT-4o and Gemini Flash, analyzing the correctness, reasoning, and completeness of their responses. Our findings show that while current LLMs are not yet fully capable of reliably solving BPMN assignments-especially those involving deeper process logic-they can already provide partially correct and plausible responses. This raises questions about the future of BPM education, the design of AI-aware assignments, and the role of LLMs as potential learning assistants. Alongside insights and lessons learned, we provide materials to help instructors adapt their teaching to an AI-enabled environment.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10200 - Computer and information sciences

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Lecture Notes in Business Information Processing

ISBN

978-3-031-70444-4

ISSN

1865-1348

e-ISSN

1865-1356

Počet stran výsledku

15

Strana od-do

358-372

Název nakladatele

Springer Science and Business Media Deutschland GmbH

Místo vydání

Cham

Místo konání akce

AGH Univ Krakow

Datum konání akce

1. 9. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001588122500023