All

What are you looking for?

All
Projects
Results
Organizations

Quick search

  • Projects supported by TA ČR
  • Excellent projects
  • Projects with the highest public support
  • Current projects

Smart search

  • That is how I find a specific +word
  • That is how I leave the -word out of the results
  • “That is how I can find the whole phrase”

A Domain-region Based Evaluation of ML Performance Robustness to Covariate Shift

The result's identifiers

  • Result code in IS VaVaI

    <a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F23%3A00370203" target="_blank" >RIV/68407700:21230/23:00370203 - isvavai.cz</a>

  • Result on the web

    <a href="https://doi.org/10.1007/s00521-023-08622-w" target="_blank" >https://doi.org/10.1007/s00521-023-08622-w</a>

  • DOI - Digital Object Identifier

    <a href="http://dx.doi.org/10.1007/s00521-023-08622-w" target="_blank" >10.1007/s00521-023-08622-w</a>

Alternative languages

  • Result language

    angličtina

  • Original language name

    A Domain-region Based Evaluation of ML Performance Robustness to Covariate Shift

  • Original language description

    Most machine learning methods assume that the input data distribution is the same in the training and testing phases. However, in practice, this stationarity is usually not met and the distribution of inputs differs, leading to unexpected performance of the learned model in deployment. The issue in which the training and test data inputs follow different probability distributions while the input-output relationship remains unchanged is referred to as covariate shift. In this paper, the performance of conventional machine learning models was experimentally evaluated in the presence of covariate shift. Furthermore, a region-based evaluation was performed by decomposing the domain of probability density function of the input data to assess the classifier's performance per domain region. Distributional changes were simulated in a two-dimensional classification problem. Subsequently, a higher four-dimensional experiments were conducted. Based on the experimental analysis, the Random Forests algorithm is the most robust classifier in the two-dimensional case, showing the lowest degradation rate for accuracy and F1-score metrics, with a range between 0.1% and 2.08%. Moreover, the results reveal that in higher-dimensional experiments, the performance of the models is predominantly influenced by the complexity of the classification function, leading to degradation rates exceeding 25% in most cases. It is also concluded that the models exhibit high bias toward the region with high density in the input space domain of the training samples.

  • Czech name

  • Czech description

Classification

  • Type

    J<sub>imp</sub> - Article in a specialist periodical, which is included in the Web of Science database

  • CEP classification

  • OECD FORD branch

    10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Result continuities

  • Project

  • Continuities

    I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Others

  • Publication year

    2023

  • Confidentiality

    S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Data specific for result type

  • Name of the periodical

    Neural Computing and Applications

  • ISSN

    0941-0643

  • e-ISSN

    1433-3058

  • Volume of the periodical

    35

  • Issue of the periodical within the volume

    24

  • Country of publishing house

    GB - UNITED KINGDOM

  • Number of pages

    23

  • Pages from-to

    17555-17577

  • UT code for WoS article

    000988193400003

  • EID of the result in the Scopus database

    2-s2.0-85159332578