Testing Heterogeneity in Inter-Rater Reliability

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F20%3A00531172" target="_blank" >RIV/67985807:_____/20:00531172 - isvavai.cz</a>
Alternative codes found
RIV/00216208:11210/20:10416702 RIV/00216208:11410/20:10416702
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-030-43469-4_26" target="_blank" >http://dx.doi.org/10.1007/978-3-030-43469-4_26</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-030-43469-4_26" target="_blank" >10.1007/978-3-030-43469-4_26</a>

Result language
angličtina
Original language name
Testing Heterogeneity in Inter-Rater Reliability
Original language description
Estimating the inter-rater reliability (IRR) is important for assessing and improving the quality of ratings. In some cases, the IRR may differ between groups due to their features. To test heterogeneity in IRR, the second-order generalized estimating equations (GEE2) and linear mixed-effects models (LME) were already used. Another method capable of estimating the components for IRR is generalized additive models (GAM). This paper presents a simulation study evaluating the performance of these methods in estimating variance components and in testing heterogeneity in IRR. We consider a wide range of sample sizes and various scenarios leading to heterogenous IRR. The results show, that while the LME and GAM models perform similarly and yield reliable estimates, the GEE2 models may lead to incorrect results.
Czech name
—
Czech description
—

Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year
2020
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Similar results(10)