Vše

Co hledáte?

Vše
Projekty
Výsledky výzkumu
Subjekty

Rychlé hledání

  • Projekty podpořené TA ČR
  • Významné projekty
  • Projekty s nejvyšší státní podporou
  • Aktuálně běžící projekty

Chytré vyhledávání

  • Takto najdu konkrétní +slovo
  • Takto z výsledků -slovo zcela vynechám
  • “Takto můžu najít celou frázi”

Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability

Identifikátory výsledku

  • Kód výsledku v IS VaVaI

    <a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11410%2F18%3A10380400" target="_blank" >RIV/00216208:11410/18:10380400 - isvavai.cz</a>

  • Nalezeny alternativní kódy

    RIV/67985807:_____/18:00494344

  • Výsledek na webu

    <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0203002" target="_blank" >https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0203002</a>

  • DOI - Digital Object Identifier

    <a href="http://dx.doi.org/10.1371/journal.pone.0203002" target="_blank" >10.1371/journal.pone.0203002</a>

Alternativní jazyky

  • Jazyk výsledku

    angličtina

  • Název v původním jazyce

    Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability

  • Popis výsledku v původním jazyce

    Ratings are present in many areas of assessment including peer review of research proposals and journal articles, teacher observations, university admissions and selection of new hires. One feature present in any rating process with multiple raters is that different raters often assign different scores to the same assessee, with the potential for bias and inconsistencies related to rater or assessee covariates. This paper analyzes disparities in ratings of internal and external applicants to teaching positions using applicant data from Spokane Public Schools. We first test for biases in rating while accounting for measures of teacher applicant qualifications and quality. Then, we develop model-based inter-rater reliability (IRR) estimates that allow us to account for various sources of measurement error, the hierarchical structure of the data, and to test whether covariates, such as applicant status, moderate IRR. We find that applicants external to the district receive lower ratings for job applications compared to internal applicants. This gap in ratings remains significant even after including measures of qualifications and quality such as experience, state licensure scores, or estimated teacher value added. With model-based IRR, we further show that consistency between raters is significantly lower when rating external applicants. We conclude the paper by discussing policy implications and possible applications of our model-based IRR estimate for hiring and selection practices in and out of the teacher labor market.

  • Název v anglickém jazyce

    Disparities in ratings of internal and external applicants: A case for model-based inter-rater reliability

  • Popis výsledku anglicky

    Ratings are present in many areas of assessment including peer review of research proposals and journal articles, teacher observations, university admissions and selection of new hires. One feature present in any rating process with multiple raters is that different raters often assign different scores to the same assessee, with the potential for bias and inconsistencies related to rater or assessee covariates. This paper analyzes disparities in ratings of internal and external applicants to teaching positions using applicant data from Spokane Public Schools. We first test for biases in rating while accounting for measures of teacher applicant qualifications and quality. Then, we develop model-based inter-rater reliability (IRR) estimates that allow us to account for various sources of measurement error, the hierarchical structure of the data, and to test whether covariates, such as applicant status, moderate IRR. We find that applicants external to the district receive lower ratings for job applications compared to internal applicants. This gap in ratings remains significant even after including measures of qualifications and quality such as experience, state licensure scores, or estimated teacher value added. With model-based IRR, we further show that consistency between raters is significantly lower when rating external applicants. We conclude the paper by discussing policy implications and possible applications of our model-based IRR estimate for hiring and selection practices in and out of the teacher labor market.

Klasifikace

  • Druh

    J<sub>imp</sub> - Článek v periodiku v databázi Web of Science

  • CEP obor

  • OECD FORD obor

    50301 - Education, general; including training, pedagogy, didactics [and education systems]

Návaznosti výsledku

  • Projekt

    <a href="/cs/project/GJ15-15856Y" target="_blank" >GJ15-15856Y: Odhad psychometrických vlastností jako součást vývoje přijímacích testů</a><br>

  • Návaznosti

    I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Ostatní

  • Rok uplatnění

    2018

  • Kód důvěrnosti údajů

    S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Údaje specifické pro druh výsledku

  • Název periodika

    PLoS One

  • ISSN

    1932-6203

  • e-ISSN

  • Svazek periodika

    13

  • Číslo periodika v rámci svazku

    10

  • Stát vydavatele periodika

    US - Spojené státy americké

  • Počet stran výsledku

    17

  • Strana od-do

    1-17

  • Kód UT WoS článku

    000446545500004

  • EID výsledku v databázi Scopus

    2-s2.0-85054464213