Decentralized Bayesian Learning with Metropolis-adjusted Hamiltonian Monte Carlo

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F23%3A00373589" target="_blank" >RIV/68407700:21230/23:00373589 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1007/s10994-023-06345-6" target="_blank" >https://doi.org/10.1007/s10994-023-06345-6</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s10994-023-06345-6" target="_blank" >10.1007/s10994-023-06345-6</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Decentralized Bayesian Learning with Metropolis-adjusted Hamiltonian Monte Carlo

Popis výsledku v původním jazyce

Federated learning performed by a decentralized networks of agents is becoming increasingly important with the prevalence of embedded software on autonomous devices. Bayesian approaches to learning benefit from offering more information as to the uncertainty of a random quantity, and Langevin and Hamiltonian methods are effective at realizing sampling from an uncertain distribution with large parameter dimensions. Such methods have only recently appeared in the decentralized setting, and either exclusively use stochastic gradient Langevin and Hamiltonian Monte Carlo approaches that require a diminishing stepsize to asymptotically sample from the posterior and are known in practice to characterize uncertainty less faithfully than constant step-size methods with a Metropolis adjustment, or assume strong convexity properties of the potential function. We present the first approach to incorporating constant stepsize Metropolis-adjusted HMC in the decentralized sampling framework, show theoretical guarantees for consensus and probability distance to the posterior stationary distribution, and demonstrate their effectiveness numerically on standard real world problems, including decentralized learning of neural networks which is known to be highly non-convex.

Název v anglickém jazyce

Decentralized Bayesian Learning with Metropolis-adjusted Hamiltonian Monte Carlo

Popis výsledku anglicky

Federated learning performed by a decentralized networks of agents is becoming increasingly important with the prevalence of embedded software on autonomous devices. Bayesian approaches to learning benefit from offering more information as to the uncertainty of a random quantity, and Langevin and Hamiltonian methods are effective at realizing sampling from an uncertain distribution with large parameter dimensions. Such methods have only recently appeared in the decentralized setting, and either exclusively use stochastic gradient Langevin and Hamiltonian Monte Carlo approaches that require a diminishing stepsize to asymptotically sample from the posterior and are known in practice to characterize uncertainty less faithfully than constant step-size methods with a Metropolis adjustment, or assume strong convexity properties of the potential function. We present the first approach to incorporating constant stepsize Metropolis-adjusted HMC in the decentralized sampling framework, show theoretical guarantees for consensus and probability distance to the posterior stationary distribution, and demonstrate their effectiveness numerically on standard real world problems, including decentralized learning of neural networks which is known to be highly non-convex.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/EF16_019%2F0000765" target="_blank" >EF16_019/0000765: Výzkumné centrum informatiky</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Machine Learning

ISSN

0885-6125

e-ISSN

1573-0565

Svazek periodika

112

Číslo periodika v rámci svazku

8

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

29

Strana od-do

2791-2819

Kód UT WoS článku

001015523900002

EID výsledku v databázi Scopus

2-s2.0-85162198728