Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F62690094%3A18450%2F24%3A50021939" target="_blank" >RIV/62690094:18450/24:50021939 - isvavai.cz</a>
Result on the web
<a href="https://www.sciencedirect.com/science/article/pii/S1877050924025882?via%3Dihub" target="_blank" >https://www.sciencedirect.com/science/article/pii/S1877050924025882?via%3Dihub</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1016/j.procs.2024.09.543" target="_blank" >10.1016/j.procs.2024.09.543</a>
Alternative languages
Result language
angličtina
Original language name
Examining the Reliability of ChatGPT as an Assessment Tool Compared to Human Evaluators
Original language description
With the increasing development of artificial intelligence (AI), educators are exploring various AI-driven tools to enhance language teaching and learning processes. This study investigates the reliability of ChatGPT as an AI-powered assessment tool in the context of foreign language (FL) education. Using a mixed-methods approach, the research examines ChatGPT’s evaluative abilities compared to human evaluators, focusing on its assessment of students’ writing skills. The study employed a total of two research samples: one comprising eight students and another consisting of eight teachers of English language at the university where the research took place. The findings reveal that while ChatGPT demonstrates potential in providing feedback, its limitations, particularly in accurate assessment of grammatical nuances, emphasize the need for supplementation with human evaluation. The method section provides detailed information on the data collection process, including the procedures followed for evaluating student summaries and conducting interviews with teachers. The study highlights the importance of acknowledging AI’s limitations and adopting a complementary approach to its integration in language education.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
50301 - Education, general; including training, pedagogy, didactics [and education systems]
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Procedia computer science
ISBN
—
ISSN
1877-0509
e-ISSN
1877-0509
Number of pages
10
Pages from-to
2332-2341
Publisher name
Elsevier
Place of publication
Amsterdam
Event location
Seville
Event date
Sep 11, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—