Revealing data leakage in protein interaction benchmarks

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21730%2F24%3A00380615" target="_blank" >RIV/68407700:21730/24:00380615 - isvavai.cz</a>

Result on the web

<a href="https://openreview.net/attachment?id=ORMXYUK5IY&name=pdf" target="_blank" >https://openreview.net/attachment?id=ORMXYUK5IY&name=pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Revealing data leakage in protein interaction benchmarks

Original language description

In recent years, there has been remarkable progress in machine learning for protein–protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein–protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceeding The Twelfth International Conference on Learning Representations (ICLR 2024)

ISBN

9781713898658

ISSN

—

e-ISSN

—

Number of pages

13

Pages from-to

—

Publisher name

International Conference on Learning Representations

Place of publication

—

Event location

Vídeň

Event date

May 7, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—