ChatGPT improves creative problem-solving performance in university students: An experimental study

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68081740%3A_____%2F24%3A00584616" target="_blank" >RIV/68081740:_____/24:00584616 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/00216208:11210/24:10478949 RIV/00216208:11240/24:10478949 RIV/00216208:11320/24:10478949

Výsledek na webu

<a href="https://www.sciencedirect.com/science/article/pii/S0360131524000459?via%3Dihub" target="_blank" >https://www.sciencedirect.com/science/article/pii/S0360131524000459?via%3Dihub</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.compedu.2024.105031" target="_blank" >10.1016/j.compedu.2024.105031</a>

Jazyk výsledku

angličtina

Název v původním jazyce

ChatGPT improves creative problem-solving performance in university students: An experimental study

Popis výsledku v původním jazyce

University students often employ generative artificial intelligence tools such as ChatGPT in resolution of ill-defined problem-solving tasks. However, the experimental evidence about effects of ChatGPT on complex problem-solving performance is still missing. In this preregistered experiment, the impact of ChatGPT on performance in a complex creative problem-solving task was investigated in 77 university students solving a task with ChatGPT in comparison to 68 students solving a task without it. ChatGPT use significantly improved self-efficacy for task resolution (d = 0.65) and enhanced the quality (d = 0.69), elaboration (d = 0.61), and originality (d = 0.55) of solutions. Moreover, participants with ChatGPT assistance perceived task as easier (d = 0.56) and requiring less mental effort (d = 0.58). However, use of ChatGPT did not make task resolution more interesting (d = 0.08), and the impact of ChatGPT on metacognitive monitoring accuracy was unclear. Although there were no significant differences in absolute accuracy between students solving the task with and without the assistance of ChatGPT, the absence of correlation between self-evaluation judgments and performance suggests that participants struggled to calibrate their self-evaluations when using ChatGPT. Notably, the perceived usefulness of ChatGPT appeared to inform self-evaluation judgments, resulting in higher inaccuracy. The implications for hybrid human-AI regulation (HHAIR) theory are discussed. To regulate effectively, students using AI tools should focus on valid metacognitive cues instead of the perceived ease of ChatGPT-assisted problem-solving.

Název v anglickém jazyce

ChatGPT improves creative problem-solving performance in university students: An experimental study

Popis výsledku anglicky

University students often employ generative artificial intelligence tools such as ChatGPT in resolution of ill-defined problem-solving tasks. However, the experimental evidence about effects of ChatGPT on complex problem-solving performance is still missing. In this preregistered experiment, the impact of ChatGPT on performance in a complex creative problem-solving task was investigated in 77 university students solving a task with ChatGPT in comparison to 68 students solving a task without it. ChatGPT use significantly improved self-efficacy for task resolution (d = 0.65) and enhanced the quality (d = 0.69), elaboration (d = 0.61), and originality (d = 0.55) of solutions. Moreover, participants with ChatGPT assistance perceived task as easier (d = 0.56) and requiring less mental effort (d = 0.58). However, use of ChatGPT did not make task resolution more interesting (d = 0.08), and the impact of ChatGPT on metacognitive monitoring accuracy was unclear. Although there were no significant differences in absolute accuracy between students solving the task with and without the assistance of ChatGPT, the absence of correlation between self-evaluation judgments and performance suggests that participants struggled to calibrate their self-evaluations when using ChatGPT. Notably, the perceived usefulness of ChatGPT appeared to inform self-evaluation judgments, resulting in higher inaccuracy. The implications for hybrid human-AI regulation (HHAIR) theory are discussed. To regulate effectively, students using AI tools should focus on valid metacognitive cues instead of the perceived ease of ChatGPT-assisted problem-solving.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

50101 - Psychology (including human - machine relations)

Projekt

<a href="/cs/project/GA24-11515S" target="_blank" >GA24-11515S: Učení se s ChatGPT: proč a jak studující využívají ChatGPT</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Computers & Education

ISSN

0360-1315

e-ISSN

1873-782X

Svazek periodika

215

Číslo periodika v rámci svazku

červenec

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

37

Strana od-do

105031

Kód UT WoS článku

001218625800001

EID výsledku v databázi Scopus

2-s2.0-85189012818