Differential item functioning: Effect sizes classification

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00601140" target="_blank" >RIV/67985807:_____/24:00601140 - isvavai.cz</a>

Result on the web

<a href="https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=467" target="_blank" >https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=467</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Differential item functioning: Effect sizes classification

Original language description

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 429-429. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: An important aspect of subsequent analysis in multi-item measurements involves checking for Differential Item Functioning (DIF), that is, identifying potentially biased items that function differently across distinct population groups. Numerous statistical procedures have been developed to detect DIF by testing the statistical significance of the item-level differences between groups. However, besides the statistical significance of DIF, it is also vital to assess its practical significance by examining the magnitude of the corresponding effect size measure. This is necessary because even practically “negligible” differences can be statistically significant. In this work, we review existing DIF effect size measures and the cut-off values used to classify the effect size magnitudes as “negligible” (A), “moderate” (B), and “large” (C) for logistic regression models and Item Response Theory models, both types of models in the case of binary items. The properties of the effect size measures and cut-off values are evaluated through a simulation study for both uniform and non-uniform DIF. Based on the simulation study, several effect size measures seem to display unsatisfactory properties (e.g., dependence on sample size, inconsistent classification of the underlying DIF, underestimating of the true underlying effect size measure). We propose solutions to observed inconsistencies and issues.

Czech name

—

Czech description

—

Type

O - Miscellaneous

CEP classification

—

OECD FORD branch

10103 - Statistics and probability

Project

<a href="/en/project/EH22_008%2F0004583" target="_blank" >EH22_008/0004583: Research of Excellence on Digital Technologies and Wellbeing</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů