Analysis of Student Interactions with a Large Language Model in an Introductory Physics Lab Setting

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A10508873" target="_blank" >RIV/00216208:11320/25:10508873 - isvavai.cz</a>

Výsledek na webu

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=0siDxd4VPQ" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=0siDxd4VPQ</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s40593-025-00489-3" target="_blank" >10.1007/s40593-025-00489-3</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Analysis of Student Interactions with a Large Language Model in an Introductory Physics Lab Setting

Popis výsledku v původním jazyce

This study investigates using Large Language Models (LLMs), specifically OpenAI’s GPT-4, in an introductory physics lab setting. Researchers at Portland State University designed an artificial intelligence lab assistant—a custom web interface to support students during an in-person lab exploring the moment of inertia of different objects. The study aimed to determine the positive and negative aspects of LLM integration from students’ points of view and experts’ assessment of the interactions between students and the LLM assistant. For the latter, we analyzed the accuracy and helpfulness of the LLM responses from the transcript of all interactions between students and the LLM assistant. We found that students’ use of the LLM was beneficial in various ways such as answer verification, guiding students to the correct answer, and providing support with theoretical questions. However, the LLM responses were also detrimental in more than one out of ten cases by providing incorrect feedback, or misleading or confusing students. Overall, students reported a positive experience with using the LLM assistant, highlighting its potential utility and benefits in enhancing their education. This study is an example of how LLMs can be integrated into a physics lab environment and we discuss the implications of the findings for future research and practical application of science education. The study contributes to the ongoing discourse on sound pedagogical approaches to the use of LLMs in teaching.

Název v anglickém jazyce

Analysis of Student Interactions with a Large Language Model in an Introductory Physics Lab Setting

Popis výsledku anglicky

This study investigates using Large Language Models (LLMs), specifically OpenAI’s GPT-4, in an introductory physics lab setting. Researchers at Portland State University designed an artificial intelligence lab assistant—a custom web interface to support students during an in-person lab exploring the moment of inertia of different objects. The study aimed to determine the positive and negative aspects of LLM integration from students’ points of view and experts’ assessment of the interactions between students and the LLM assistant. For the latter, we analyzed the accuracy and helpfulness of the LLM responses from the transcript of all interactions between students and the LLM assistant. We found that students’ use of the LLM was beneficial in various ways such as answer verification, guiding students to the correct answer, and providing support with theoretical questions. However, the LLM responses were also detrimental in more than one out of ten cases by providing incorrect feedback, or misleading or confusing students. Overall, students reported a positive experience with using the LLM assistant, highlighting its potential utility and benefits in enhancing their education. This study is an example of how LLMs can be integrated into a physics lab environment and we discuss the implications of the findings for future research and practical application of science education. The study contributes to the ongoing discourse on sound pedagogical approaches to the use of LLMs in teaching.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

50301 - Education, general; including training, pedagogy, didactics [and education systems]

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

International Journal of Artificial Intelligence in Education

ISSN

1560-4292

e-ISSN

1560-4306

Svazek periodika

35

Číslo periodika v rámci svazku

5

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

24

Strana od-do

2993-3016

Kód UT WoS článku

001502645200001

EID výsledku v databázi Scopus

2-s2.0-105007291681