Using artificial intelligence to assess students' programming knowledge
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F60076658%3A12310%2F25%3A43911205" target="_blank" >RIV/60076658:12310/25:43911205 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.21125/edulearn.2025.2153" target="_blank" >http://dx.doi.org/10.21125/edulearn.2025.2153</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.21125/edulearn.2025.2153" target="_blank" >10.21125/edulearn.2025.2153</a>
Alternative languages
Result language
angličtina
Original language name
Using artificial intelligence to assess students' programming knowledge
Original language description
Modern artificial intelligence tools, particularly large language models (LLM), also find applications in evaluating students' knowledge in IT fields. A specific subset in this area is the teaching of programming. Traditionally, classical programming tasks are evaluated using so-called autograders, but their code assessment capabilities are mainly based on verifying the functional correctness of the code. However, artificial intelligence, especially LLMs, provides opportunities to automate the assessment process and refine and extend the assessment of programming knowledge and skills. One way is, for example, to generate tests for auto-grading. More interesting, however, is the possibility of assessing code quality and style and adherence to required procedures or algorithm efficiency.This paper discusses a particular application of using LLM to automate the assessment in the teaching of programming. A pilot solution that enables automatic evaluation of programming tasks without the teacher's need for manual intervention will be presented. The system supports the assignment of tasks, the interactive definition of evaluation criteria and their weights, and subsequent analysis of the submitted code and generation of feedback, while running the code is not a necessary part of the process. Thus, using language models increases the assessment's efficiency and objectivity while reducing the educator's involvement. The presented application has been tested in a real classroom with positive results. The paper will also discuss the experience gained and possible ways for future development and expansion of the application.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
EDULEARN25 Proceedings
ISBN
978-84-09-74218-9
ISSN
2340-1117
e-ISSN
—
Number of pages
5
Pages from-to
8359-8364
Publisher name
IATED Academy
Place of publication
Palma, Spain
Event location
Palma, Spain
Event date
Jun 30, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—