Predicting item difficulty with text analysis and machine learning in different languages and item types

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00601204" target="_blank" >RIV/67985807:_____/24:00601204 - isvavai.cz</a>

Result on the web

<a href="https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=522" target="_blank" >https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=522</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Predicting item difficulty with text analysis and machine learning in different languages and item types

Original language description

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 309-309. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: In standardized testing, predicting item difficulty from item wording is useful both for test development as well as for deeper understanding of what makes an item a difficult one. Many features may influence item difficulty, such as the length of answer choices, their similarity with the item question, difficulty of the words used, etc., and different machine learning models may be used to predict item difficulty from item features (Štěpánek et al., 2023). However, differences and challenges may arise when building models for different item types (including those involving audio, or visual components), and for different languages. In this work, we extract item features from various types of test items from the English, German, and French as foreign languages Czech matura exams into various item features, and train numerous different machine learning models to predict their difficulty. We compare and analyze the models and features in order to create a tool that can analyze and suggest changes during test development to help achieve an optimal item difficulty.

Czech name

—

Czech description

—

Type

O - Miscellaneous

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/TL05000008" target="_blank" >TL05000008: Advances in educational assessment: Analytical support for educational test development</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů