Effect sizes for detection of differential item functioning

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00601182" target="_blank" >RIV/67985807:_____/24:00601182 - isvavai.cz</a>

Výsledek na webu

<a href="https://as.mf.uni-lj.si/uploads/pdf/as2024book.pdf" target="_blank" >https://as.mf.uni-lj.si/uploads/pdf/as2024book.pdf</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Effect sizes for detection of differential item functioning

Popis výsledku v původním jazyce

ZÁKLADNÍ ÚDAJE: APPLIED STATISTICS 2024. Program and Abstracts. Koper: Statistical Society of Slovenia, 2024 - (Kastrin, A., Lusa, L.). s. 39-39. ISBN 978-961-94283-4-4. [Applied Statistics 2024 /20./. 23.09.2024-25.09.2024, Koper]. ABSTRAKT: An analysis of multi-item measurements in educational or psychological assessments includes testing for Differential Item Functioning (DIF), that is, identifying items that function differently for distinct population groups, indicating potential bias. The presence of DIF can be tested using various statistical methods, such as those based on contingency tables, logistic regression, and item response theory models. For items with statistically significant DIF, it is important to assess the practical significance by quantifying the magnitude of DIF with an appropriate effect size measure. This is necessary because even differences with no practical impact may be statistically significant. In this work, we review existing DIF effect size measures and the cut-off values used to classify the effect size magnitudes for the Mantel-Haenszel test, SIBTEST, and model-based approaches. We conduct a simulation study to investigate the properties of these effect size measures and their existing classification guidelines. Based on the simulation study, we suggest updating some of the values. Additionally, we suggest appropriate cut-off values for effect size measures based on the area between the item characteristic curves, taking those based on the Mantel-Haenszel approach as as a reference. Furthermore, we investigate the newly proposed cut-off values through an additional simulation study. We propose solutions to observed inconsistencies and issues, focusing on the practical implementation using the R software.

Název v anglickém jazyce

Effect sizes for detection of differential item functioning

Popis výsledku anglicky

ZÁKLADNÍ ÚDAJE: APPLIED STATISTICS 2024. Program and Abstracts. Koper: Statistical Society of Slovenia, 2024 - (Kastrin, A., Lusa, L.). s. 39-39. ISBN 978-961-94283-4-4. [Applied Statistics 2024 /20./. 23.09.2024-25.09.2024, Koper]. ABSTRAKT: An analysis of multi-item measurements in educational or psychological assessments includes testing for Differential Item Functioning (DIF), that is, identifying items that function differently for distinct population groups, indicating potential bias. The presence of DIF can be tested using various statistical methods, such as those based on contingency tables, logistic regression, and item response theory models. For items with statistically significant DIF, it is important to assess the practical significance by quantifying the magnitude of DIF with an appropriate effect size measure. This is necessary because even differences with no practical impact may be statistically significant. In this work, we review existing DIF effect size measures and the cut-off values used to classify the effect size magnitudes for the Mantel-Haenszel test, SIBTEST, and model-based approaches. We conduct a simulation study to investigate the properties of these effect size measures and their existing classification guidelines. Based on the simulation study, we suggest updating some of the values. Additionally, we suggest appropriate cut-off values for effect size measures based on the area between the item characteristic curves, taking those based on the Mantel-Haenszel approach as as a reference. Furthermore, we investigate the newly proposed cut-off values through an additional simulation study. We propose solutions to observed inconsistencies and issues, focusing on the practical implementation using the R software.

Druh

O - Ostatní výsledky

CEP obor

—

OECD FORD obor

10103 - Statistics and probability

Projekt

<a href="/cs/project/EH22_008%2F0004583" target="_blank" >EH22_008/0004583: Excelentní výzkum v oblasti digitálních technologií a wellbeingu</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů