Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F62690094%3A18450%2F24%3A50021939" target="_blank" >RIV/62690094:18450/24:50021939 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.sciencedirect.com/science/article/pii/S1877050924025882?via%3Dihub" target="_blank" >https://www.sciencedirect.com/science/article/pii/S1877050924025882?via%3Dihub</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.procs.2024.09.543" target="_blank" >10.1016/j.procs.2024.09.543</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators

Popis výsledku v původním jazyce

With the increasing development of artificial intelligence (AI), educators are exploring various AI-driven tools to enhance language teaching and learning processes. This study investigates the reliability of ChatGPT as an AI-powered assessment tool in the context of foreign language (FL) education. Using a mixed-methods approach, the research examines ChatGPT’s evaluative abilities compared to human evaluators, focusing on its assessment of students’ writing skills. The study employed a total of two research samples: one comprising eight students and another consisting of eight teachers of English language at the university where the research took place. The findings reveal that while ChatGPT demonstrates potential in providing feedback, its limitations, particularly in accurate assessment of grammatical nuances, emphasize the need for supplementation with human evaluation. The method section provides detailed information on the data collection process, including the procedures followed for evaluating student summaries and conducting interviews with teachers. The study highlights the importance of acknowledging AI’s limitations and adopting a complementary approach to its integration in language education.

Název v anglickém jazyce

Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators

Popis výsledku anglicky

With the increasing development of artificial intelligence (AI), educators are exploring various AI-driven tools to enhance language teaching and learning processes. This study investigates the reliability of ChatGPT as an AI-powered assessment tool in the context of foreign language (FL) education. Using a mixed-methods approach, the research examines ChatGPT’s evaluative abilities compared to human evaluators, focusing on its assessment of students’ writing skills. The study employed a total of two research samples: one comprising eight students and another consisting of eight teachers of English language at the university where the research took place. The findings reveal that while ChatGPT demonstrates potential in providing feedback, its limitations, particularly in accurate assessment of grammatical nuances, emphasize the need for supplementation with human evaluation. The method section provides detailed information on the data collection process, including the procedures followed for evaluating student summaries and conducting interviews with teachers. The study highlights the importance of acknowledging AI’s limitations and adopting a complementary approach to its integration in language education.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

50301 - Education, general; including training, pedagogy, didactics [and education systems]

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Procedia computer science

ISBN

—

ISSN

1877-0509

e-ISSN

1877-0509

Počet stran výsledku

10

Strana od-do

2332-2341

Název nakladatele

Elsevier

Místo vydání

Amsterdam

Místo konání akce

Seville

Datum konání akce

11. 9. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—