Trading-off Data Fit and Complexity in Training Gaussian Processes with Multiple Kernels

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989592%3A15310%2F19%3A73598355" target="_blank" >RIV/61989592:15310/19:73598355 - isvavai.cz</a>

Result on the web

<a href="https://link.springer.com/chapter/10.1007%2F978-3-030-37599-7_48" target="_blank" >https://link.springer.com/chapter/10.1007%2F978-3-030-37599-7_48</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-37599-7_48" target="_blank" >10.1007/978-3-030-37599-7_48</a>

Result language

angličtina

Original language name

Trading-off Data Fit and Complexity in Training Gaussian Processes with Multiple Kernels

Original language description

Gaussian processes (GPs) belong to a class of probabilistic techniques that have been successfully used in different domains of machine learning and optimization. They are popular because they provide uncertainties in predictions, which sets them apart from other modelling methods providing only point predictions. The uncertainty is particularly useful for decision making as we can gauge how reliable a prediction is. One of the fundamental challenges in using GPs is that the efficacy of a model is conferred by selecting an appropriate kernel and the associated hyperparameter values for a given problem. Furthermore, the training of GPs, that is optimizing the hyperparameters using a data set is traditionally performed using a cost function that is a weighted sum of data fit and model complexity, and the underlying trade-off is completely ignored. Addressing these challenges and shortcomings, in this article, we propose the following automated training scheme. Firstly, we use a weighted product of multiple kernels with a view to relieve the users from choosing an appropriate kernel for the problem at hand without any domain specific knowledge. Secondly, for the first time, we modify GP training by using a multi-objective optimizer to tune the hyperparameters and weights of multiple kernels and extract an approximation of the complete trade-off front between data-fit and model complexity. We then propose to use a novel solution selection strategy based on mean standardized log loss (MSLL) to select a solution from the estimated trade-off front and finalise training of a GP model. The results on three data sets and comparison with the standard approach clearly show the potential benefit of the proposed approach of using multi-objective optimization with multiple kernels.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10102 - Applied mathematics

Project

<a href="/en/project/EF17_049%2F0008408" target="_blank" >EF17_049/0008408: Hydrodynamic design of pumps</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2019

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Nicosia G., Pardalos P., Umeton R., Giuffrida G., Sciacca V. (eds) Machine Learning, Optimization, and Data Science. LOD 2019.

ISBN

978-3-030-37598-0

ISSN

0302-9743

e-ISSN

—

Number of pages

13

Pages from-to

579-591

Publisher name

Springer

Place of publication

Cham

Event location

Certosa di Pontignano, Siena – Tuscany, Italy

Event date

Sep 10, 2019

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—