Coupled or Decoupled Evaluation for Group Recommendation Methods?

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F21%3A10432911" target="_blank" >RIV/00216208:11320/21:10432911 - isvavai.cz</a>

Result on the web

<a href="http://ceur-ws.org/Vol-2955/paper1.pdf" target="_blank" >http://ceur-ws.org/Vol-2955/paper1.pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Coupled or Decoupled Evaluation for Group Recommendation Methods?

Original language description

Group recommendations are a sub-domain of recommender systems (RS), where the final recommendations should comply with preferences of all members of the group. Usually, group recommendations are built on top of common&quot; single-user&quot; RS via aggregating models or predictions for multiple users with some notions of fairness and relevance in mind. So far, group recommendations were usually evaluated off-line either as a tightly coupled pair with the underlying RS or in a decoupled fashion, where the relevance scores estimated by underlying RS serves as a ground truth. Both evaluation types may suffer from different biases that provide illicit advantages to some classes of group recommending strategies. In experimental part, we evaluate several recent group recommendation models and show that the evaluation process itself significantly affects their perceived usability. While coupled evaluation favors group RS that tend to select per-user best items, decoupled evaluation favors strategies aiming to find items with (some degree of) overall agreement. We further evaluate methods wrt several variants of inverse propensity based de-biasing scenario in order to reduce the popularity bias of coupled evaluations. Also in this case, if groups of similar users are considered, the magnitude of de-biasing has a determining effect on the ordering of individual methods.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GJ19-22071Y" target="_blank" >GJ19-22071Y: Flexible models for known-item search in large video collections</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Perspectives on the Evaluation of Recommender Systems Workshop

ISBN

—

ISSN

1613-0073

e-ISSN

—

Number of pages

13

Pages from-to

—

Publisher name

CEUR Workshop Proceedings (CEUR-WS.org)

Place of publication

Neuveden

Event location

Amsterdam, Netherlands

Event date

Sep 27, 2021

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—