Revealing data leakage in protein interaction benchmarks

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61388963%3A_____%2F24%3A00585960" target="_blank" >RIV/61388963:_____/24:00585960 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/68407700:21730/24:00380615

Výsledek na webu

<a href="https://openreview.net/forum?id=ORMXYUK5IY" target="_blank" >https://openreview.net/forum?id=ORMXYUK5IY</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Revealing data leakage in protein interaction benchmarks

Popis výsledku v původním jazyce

In recent years, there has been remarkable progress in machine learning for protein-protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein-protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.

Název v anglickém jazyce

Revealing data leakage in protein interaction benchmarks

Popis výsledku anglicky

In recent years, there has been remarkable progress in machine learning for protein-protein interactions. However, prior work has predominantly focused on improving learning algorithms, with less attention paid to evaluation strategies and data preparation. Here, we demonstrate that further development of machine learning methods may be hindered by the quality of existing train-test splits. Specifically, we find that commonly used splitting strategies for protein complexes, based on protein sequence or metadata similarity, introduce major data leakage. This may result in overoptimistic evaluation of generalization, as well as unfair benchmarking of the models, biased towards assessing their overfitting capacity rather than practical utility. To overcome the data leakage, we recommend constructing data splits based on 3D structural similarity of protein-protein interfaces and suggest corresponding algorithms. We believe that addressing the data leakage problem is critical for further progress in this research area.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10620 - Other biological topics

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

GEM-2024. ICLR 2024 Workshop on Generative and Experimental Perspectives for Biomolecular Design

ISBN

9781713898658

ISSN

—

e-ISSN

—

Počet stran výsledku

13

Strana od-do

—

Název nakladatele

ICLR

Místo vydání

—

Místo konání akce

Vídeň

Datum konání akce

7. 5. 2024

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—