Using artificial intelligence to assess students' programming knowledge

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F60076658%3A12310%2F25%3A43911205" target="_blank" >RIV/60076658:12310/25:43911205 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.21125/edulearn.2025.2153" target="_blank" >http://dx.doi.org/10.21125/edulearn.2025.2153</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.21125/edulearn.2025.2153" target="_blank" >10.21125/edulearn.2025.2153</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Using artificial intelligence to assess students' programming knowledge

Popis výsledku v původním jazyce

Modern artificial intelligence tools, particularly large language models (LLM), also find applications in evaluating students&apos; knowledge in IT fields. A specific subset in this area is the teaching of programming. Traditionally, classical programming tasks are evaluated using so-called autograders, but their code assessment capabilities are mainly based on verifying the functional correctness of the code. However, artificial intelligence, especially LLMs, provides opportunities to automate the assessment process and refine and extend the assessment of programming knowledge and skills. One way is, for example, to generate tests for auto-grading. More interesting, however, is the possibility of assessing code quality and style and adherence to required procedures or algorithm efficiency.This paper discusses a particular application of using LLM to automate the assessment in the teaching of programming. A pilot solution that enables automatic evaluation of programming tasks without the teacher&apos;s need for manual intervention will be presented. The system supports the assignment of tasks, the interactive definition of evaluation criteria and their weights, and subsequent analysis of the submitted code and generation of feedback, while running the code is not a necessary part of the process. Thus, using language models increases the assessment&apos;s efficiency and objectivity while reducing the educator&apos;s involvement. The presented application has been tested in a real classroom with positive results. The paper will also discuss the experience gained and possible ways for future development and expansion of the application.

Název v anglickém jazyce

Using artificial intelligence to assess students' programming knowledge

Popis výsledku anglicky

Modern artificial intelligence tools, particularly large language models (LLM), also find applications in evaluating students&apos; knowledge in IT fields. A specific subset in this area is the teaching of programming. Traditionally, classical programming tasks are evaluated using so-called autograders, but their code assessment capabilities are mainly based on verifying the functional correctness of the code. However, artificial intelligence, especially LLMs, provides opportunities to automate the assessment process and refine and extend the assessment of programming knowledge and skills. One way is, for example, to generate tests for auto-grading. More interesting, however, is the possibility of assessing code quality and style and adherence to required procedures or algorithm efficiency.This paper discusses a particular application of using LLM to automate the assessment in the teaching of programming. A pilot solution that enables automatic evaluation of programming tasks without the teacher&apos;s need for manual intervention will be presented. The system supports the assignment of tasks, the interactive definition of evaluation criteria and their weights, and subsequent analysis of the submitted code and generation of feedback, while running the code is not a necessary part of the process. Thus, using language models increases the assessment&apos;s efficiency and objectivity while reducing the educator&apos;s involvement. The presented application has been tested in a real classroom with positive results. The paper will also discuss the experience gained and possible ways for future development and expansion of the application.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

EDULEARN25 Proceedings

ISBN

978-84-09-74218-9

ISSN

2340-1117

e-ISSN

—

Počet stran výsledku

5

Strana od-do

8359-8364

Název nakladatele

IATED Academy

Místo vydání

Palma, Spain

Místo konání akce

Palma, Spain

Datum konání akce

30. 6. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—