Measuring predictive performance of user models: The details matter
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F17%3A00100554" target="_blank" >RIV/00216224:14330/17:00100554 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1145/3099023.3099042" target="_blank" >http://dx.doi.org/10.1145/3099023.3099042</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1145/3099023.3099042" target="_blank" >10.1145/3099023.3099042</a>
Alternative languages
Result language
angličtina
Original language name
Measuring predictive performance of user models: The details matter
Original language description
Evaluation of user modeling techniques is often based on the predictive accuracy of models. The quantification of predictive accuracy is done using performance metrics. We show that the choice of a performance metric is important and that even details of metric computation matter. We analyze in detail two commonly used metrics (AUC, RMSE) in the context of student modeling. We discuss different approaches to their computation (global, averaging across skill, averaging across students) and show that these methods have different properties. An analysis of recent research papers shows that the reported descriptions of metric computation are often insufficient. To make research conclusions valid and reproducible, researchers need to pay more attention to the choice of performance metrics and they need to describe more explicitly details of their computation
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2017
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization
ISBN
9781450350679
ISSN
—
e-ISSN
—
Number of pages
5
Pages from-to
197-201
Publisher name
ACM
Place of publication
USA
Event location
2017
Event date
Jan 1, 2017
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—