Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F60461373%3A22340%2F25%3A43933083" target="_blank" >RIV/60461373:22340/25:43933083 - isvavai.cz</a>
Result on the web
<a href="https://link.springer.com/content/pdf/10.1007/978-3-032-02936-2.pdf" target="_blank" >https://link.springer.com/content/pdf/10.1007/978-3-032-02936-2.pdf</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-032-02936-2_25" target="_blank" >10.1007/978-3-032-02936-2_25</a>

Result language
angličtina
Original language name
Revisiting BPMN Assignments with AI in Mind: Insights from Experiments with Large Language Models in Process Modeling Education
Original language description
As Large Language Models (LLMs) become increasingly available to students, BPM educators face a new challenge: how to design assignments that remain pedagogically effective and resistant to superficial AI-generated answers. This paper presents the results of two experiments that simulate common BPMN-related homework tasks and test how LLMs respond to them. The first experiment focused on answering comprehension questions based on five BPMN models, each provided either in PNG or XML format. The second asked the models to detect modeling errors in 30 flawed BPMN diagrams. In both cases, we evaluated the outputs of ChatGPT-4o and Gemini Flash, analyzing the correctness, reasoning, and completeness of their responses. Our findings show that while current LLMs are not yet fully capable of reliably solving BPMN assignments-especially those involving deeper process logic-they can already provide partially correct and plausible responses. This raises questions about the future of BPM education, the design of AI-aware assignments, and the role of LLMs as potential learning assistants. Alongside insights and lessons learned, we provide materials to help instructors adapt their teaching to an AI-enabled environment.
Czech name
—
Czech description
—

Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)