Analysis of Student Interactions with a Large Language Model in an Introductory Physics Lab Setting

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10508873" target="_blank" >RIV/00216208:11320/25:10508873 - isvavai.cz</a>

Result on the web

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=0siDxd4VPQ" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=0siDxd4VPQ</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s40593-025-00489-3" target="_blank" >10.1007/s40593-025-00489-3</a>

Result language

angličtina

Original language name

Analysis of Student Interactions with a Large Language Model in an Introductory Physics Lab Setting

Original language description

This study investigates using Large Language Models (LLMs), specifically OpenAI’s GPT-4, in an introductory physics lab setting. Researchers at Portland State University designed an artificial intelligence lab assistant—a custom web interface to support students during an in-person lab exploring the moment of inertia of different objects. The study aimed to determine the positive and negative aspects of LLM integration from students’ points of view and experts’ assessment of the interactions between students and the LLM assistant. For the latter, we analyzed the accuracy and helpfulness of the LLM responses from the transcript of all interactions between students and the LLM assistant. We found that students’ use of the LLM was beneficial in various ways such as answer verification, guiding students to the correct answer, and providing support with theoretical questions. However, the LLM responses were also detrimental in more than one out of ten cases by providing incorrect feedback, or misleading or confusing students. Overall, students reported a positive experience with using the LLM assistant, highlighting its potential utility and benefits in enhancing their education. This study is an example of how LLMs can be integrated into a physics lab environment and we discuss the implications of the findings for future research and practical application of science education. The study contributes to the ongoing discourse on sound pedagogical approaches to the use of LLMs in teaching.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

50301 - Education, general; including training, pedagogy, didactics [and education systems]

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

International Journal of Artificial Intelligence in Education

ISSN

1560-4292

e-ISSN

1560-4306

Volume of the periodical

35

Issue of the periodical within the volume

5

Country of publishing house

NL - THE KINGDOM OF THE NETHERLANDS

Number of pages

24

Pages from-to

2993-3016

UT code for WoS article

001502645200001

EID of the result in the Scopus database

2-s2.0-105007291681