Measuring predictive performance of user models: The details matter

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F17%3A00100554" target="_blank" >RIV/00216224:14330/17:00100554 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1145/3099023.3099042" target="_blank" >http://dx.doi.org/10.1145/3099023.3099042</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3099023.3099042" target="_blank" >10.1145/3099023.3099042</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Measuring predictive performance of user models: The details matter

Popis výsledku v původním jazyce

Evaluation of user modeling techniques is often based on the predictive accuracy of models. The quantification of predictive accuracy is done using performance metrics. We show that the choice of a performance metric is important and that even details of metric computation matter. We analyze in detail two commonly used metrics (AUC, RMSE) in the context of student modeling. We discuss different approaches to their computation (global, averaging across skill, averaging across students) and show that these methods have different properties. An analysis of recent research papers shows that the reported descriptions of metric computation are often insufficient. To make research conclusions valid and reproducible, researchers need to pay more attention to the choice of performance metrics and they need to describe more explicitly details of their computation

Název v anglickém jazyce

Measuring predictive performance of user models: The details matter

Popis výsledku anglicky

Evaluation of user modeling techniques is often based on the predictive accuracy of models. The quantification of predictive accuracy is done using performance metrics. We show that the choice of a performance metric is important and that even details of metric computation matter. We analyze in detail two commonly used metrics (AUC, RMSE) in the context of student modeling. We discuss different approaches to their computation (global, averaging across skill, averaging across students) and show that these methods have different properties. An analysis of recent research papers shows that the reported descriptions of metric computation are often insufficient. To make research conclusions valid and reproducible, researchers need to pay more attention to the choice of performance metrics and they need to describe more explicitly details of their computation

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2017

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization

ISBN

9781450350679

ISSN

—

e-ISSN

—

Počet stran výsledku

5

Strana od-do

197-201

Název nakladatele

ACM

Místo vydání

USA

Místo konání akce

2017

Datum konání akce

1. 1. 2017

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—