Bayesian location-scale model for assessing reliability differences with ordinal ratings

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F23%3A00580425" target="_blank" >RIV/67985807:_____/23:00580425 - isvavai.cz</a>

Result on the web

<a href="https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2023-abstracts.pdf" target="_blank" >https://www.psychometricsociety.org/sites/main/files/file-attachments/imps2023-abstracts.pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Bayesian location-scale model for assessing reliability differences with ordinal ratings

Original language description

ZÁKLADNÍ ÚDAJE: IMPS 2023: Abstract Book: Talks. Psychometric Society, 2023. s. 88-88. [IMPS 2023: International Meeting of the Psychometric Society. 23.07.2023-28.07.2023, Washington DC]. ABSTRAKT: The quality of ratings and quantitative assessments depends on the reliability of the rating instrument. Especially important is the measurement error – a high measurement error results in high uncertainty of the resulting scores. Detected systematic differences in measurement error due to applicant/raters-related characteristics might provide guidance on which groups to focus on in interventions designed to lower the measurement error. A flexible approach for detecting differences in measurement error was proposed in Martinková et al., 2023) for cases when scores are assumed to be continuous. In this work, we build on this approach by focusing on ordinal ratings. We highlight cases where treating ordinal rating as continuous might result in biased estimates and outline a Bayesian cumulative probit multi-level location-scale model to mitigate the issue. We use spike-andslab prior distributions to obtain inclusion Bayes factors of individual predictors and model-averaged posterior distributions within a single model fit. We demonstrate the superiority of the proposed ordinal approach with a simulation study.

Czech name

—

Czech description

—

Type

O - Miscellaneous

CEP classification

—

OECD FORD branch

10103 - Statistics and probability

Project

<a href="/en/project/GA21-03658S" target="_blank" >GA21-03658S: Theoretical foundations of computational psychometrics</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů