Differential item functioning: Effect sizes classification

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00601140" target="_blank" >RIV/67985807:_____/24:00601140 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=467" target="_blank" >https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2024_abstracts.pdf?1720733361#page=467</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Differential item functioning: Effect sizes classification

Popis výsledku v původním jazyce

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 429-429. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: An important aspect of subsequent analysis in multi-item measurements involves checking for Differential Item Functioning (DIF), that is, identifying potentially biased items that function differently across distinct population groups. Numerous statistical procedures have been developed to detect DIF by testing the statistical significance of the item-level differences between groups. However, besides the statistical significance of DIF, it is also vital to assess its practical significance by examining the magnitude of the corresponding effect size measure. This is necessary because even practically “negligible” differences can be statistically significant. In this work, we review existing DIF effect size measures and the cut-off values used to classify the effect size magnitudes as “negligible” (A), “moderate” (B), and “large” (C) for logistic regression models and Item Response Theory models, both types of models in the case of binary items. The properties of the effect size measures and cut-off values are evaluated through a simulation study for both uniform and non-uniform DIF. Based on the simulation study, several effect size measures seem to display unsatisfactory properties (e.g., dependence on sample size, inconsistent classification of the underlying DIF, underestimating of the true underlying effect size measure). We propose solutions to observed inconsistencies and issues.

Název v anglickém jazyce

Differential item functioning: Effect sizes classification

Popis výsledku anglicky

ZÁKLADNÍ ÚDAJE: IMPS 2024 Abstracts. Prague: IMPS, 2024. s. 429-429. [IMPS 2024: Annual Meeting of the Psychometric Society. 16.07.2024-19.07.2024, Prague]. ABSTRAKT: An important aspect of subsequent analysis in multi-item measurements involves checking for Differential Item Functioning (DIF), that is, identifying potentially biased items that function differently across distinct population groups. Numerous statistical procedures have been developed to detect DIF by testing the statistical significance of the item-level differences between groups. However, besides the statistical significance of DIF, it is also vital to assess its practical significance by examining the magnitude of the corresponding effect size measure. This is necessary because even practically “negligible” differences can be statistically significant. In this work, we review existing DIF effect size measures and the cut-off values used to classify the effect size magnitudes as “negligible” (A), “moderate” (B), and “large” (C) for logistic regression models and Item Response Theory models, both types of models in the case of binary items. The properties of the effect size measures and cut-off values are evaluated through a simulation study for both uniform and non-uniform DIF. Based on the simulation study, several effect size measures seem to display unsatisfactory properties (e.g., dependence on sample size, inconsistent classification of the underlying DIF, underestimating of the true underlying effect size measure). We propose solutions to observed inconsistencies and issues.

Druh

O - Ostatní výsledky

CEP obor

—

OECD FORD obor

10103 - Statistics and probability

Projekt

<a href="/cs/project/EH22_008%2F0004583" target="_blank" >EH22_008/0004583: Excelentní výzkum v oblasti digitálních technologií a wellbeingu</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů