Revealing data leakage in protein interaction benchmarks

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61388963%3A_____%2F24%3A00585960" target="_blank" >RIV/61388963:_____/24:00585960 - isvavai.cz</a>
Alternative codes found
RIV/68407700:21730/24:00380615
Result on the web
<a href="https://openreview.net/forum?id=ORMXYUK5IY" target="_blank" >https://openreview.net/forum?id=ORMXYUK5IY</a>
DOI - Digital Object Identifier
—

Result language
angličtina
Original language name
Revealing data leakage in protein interaction benchmarks
Original language description
In recent years, there has been remarkable progress in machine learning for protein-protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein-protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.
Czech name
—
Czech description
—

Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
GEM-2024. ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design
ISBN
9781713898658
ISSN
—
e-ISSN
—
Number of pages
13
Pages from-to
—
Publisher name
ICLR
Place of publication
—
Event location
Vídeň
Event date
May 7, 2024
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—

Similar results(10)