Predicting item difficulty with text analysis and machine learning in different languages and item types

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00601204" target="_blank" >RIV/67985807:_____/24:00601204 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=522" target="_blank" >https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=522</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Predicting item difficulty with text analysis and machine learning in different languages and item types

Popis výsledku v původním jazyce

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 309-309. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: In standardized testing, predicting item difficulty from item wording is useful both for test development as well as for deeper understanding of what makes an item a difficult one. Many features may influence item difficulty, such as the length of answer choices, their similarity with the item question, difficulty of the words used, etc., and different machine learning models may be used to predict item difficulty from item features (Štěpánek et al., 2023). However, differences and challenges may arise when building models for different item types (including those involving audio, or visual components), and for different languages. In this work, we extract item features from various types of test items from the English, German, and French as foreign languages Czech matura exams into various item features, and train numerous different machine learning models to predict their difficulty. We compare and analyze the models and features in order to create a tool that can analyze and suggest changes during test development to help achieve an optimal item difficulty.

Název v anglickém jazyce

Predicting item difficulty with text analysis and machine learning in different languages and item types

Popis výsledku anglicky

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 309-309. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: In standardized testing, predicting item difficulty from item wording is useful both for test development as well as for deeper understanding of what makes an item a difficult one. Many features may influence item difficulty, such as the length of answer choices, their similarity with the item question, difficulty of the words used, etc., and different machine learning models may be used to predict item difficulty from item features (Štěpánek et al., 2023). However, differences and challenges may arise when building models for different item types (including those involving audio, or visual components), and for different languages. In this work, we extract item features from various types of test items from the English, German, and French as foreign languages Czech matura exams into various item features, and train numerous different machine learning models to predict their difficulty. We compare and analyze the models and features in order to create a tool that can analyze and suggest changes during test development to help achieve an optimal item difficulty.

Druh

O - Ostatní výsledky

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/TL05000008" target="_blank" >TL05000008: Výzvy pro hodnocení znalostí: Analytická podpora tvorby znalostních testů</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů