Coupled or Decoupled Evaluation for Group Recommendation Methods?

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F21%3A10432911" target="_blank" >RIV/00216208:11320/21:10432911 - isvavai.cz</a>

Výsledek na webu

<a href="http://ceur-ws.org/Vol-2955/paper1.pdf" target="_blank" >http://ceur-ws.org/Vol-2955/paper1.pdf</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Coupled or Decoupled Evaluation for Group Recommendation Methods?

Popis výsledku v původním jazyce

Group recommendations are a sub-domain of recommender systems (RS), where the final recommendations should comply with preferences of all members of the group. Usually, group recommendations are built on top of common&quot; single-user&quot; RS via aggregating models or predictions for multiple users with some notions of fairness and relevance in mind. So far, group recommendations were usually evaluated off-line either as a tightly coupled pair with the underlying RS or in a decoupled fashion, where the relevance scores estimated by underlying RS serves as a ground truth. Both evaluation types may suffer from different biases that provide illicit advantages to some classes of group recommending strategies. In experimental part, we evaluate several recent group recommendation models and show that the evaluation process itself significantly affects their perceived usability. While coupled evaluation favors group RS that tend to select per-user best items, decoupled evaluation favors strategies aiming to find items with (some degree of) overall agreement. We further evaluate methods wrt several variants of inverse propensity based de-biasing scenario in order to reduce the popularity bias of coupled evaluations. Also in this case, if groups of similar users are considered, the magnitude of de-biasing has a determining effect on the ordering of individual methods.

Název v anglickém jazyce

Coupled or Decoupled Evaluation for Group Recommendation Methods?

Popis výsledku anglicky

Group recommendations are a sub-domain of recommender systems (RS), where the final recommendations should comply with preferences of all members of the group. Usually, group recommendations are built on top of common&quot; single-user&quot; RS via aggregating models or predictions for multiple users with some notions of fairness and relevance in mind. So far, group recommendations were usually evaluated off-line either as a tightly coupled pair with the underlying RS or in a decoupled fashion, where the relevance scores estimated by underlying RS serves as a ground truth. Both evaluation types may suffer from different biases that provide illicit advantages to some classes of group recommending strategies. In experimental part, we evaluate several recent group recommendation models and show that the evaluation process itself significantly affects their perceived usability. While coupled evaluation favors group RS that tend to select per-user best items, decoupled evaluation favors strategies aiming to find items with (some degree of) overall agreement. We further evaluate methods wrt several variants of inverse propensity based de-biasing scenario in order to reduce the popularity bias of coupled evaluations. Also in this case, if groups of similar users are considered, the magnitude of de-biasing has a determining effect on the ordering of individual methods.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GJ19-22071Y" target="_blank" >GJ19-22071Y: Flexibilní modely pro hledání známé scény v rozsáhlých kolekcích videa</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2021

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Perspectives on the Evaluation of Recommender Systems Workshop

ISBN

—

ISSN

1613-0073

e-ISSN

—

Počet stran výsledku

13

Strana od-do

—

Název nakladatele

CEUR Workshop Proceedings (CEUR-WS.org)

Místo vydání

Neuveden

Místo konání akce

Amsterdam, Netherlands

Datum konání akce

27. 9. 2021

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—