Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles between Client Data Subspaces

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F23%3A00367592" target="_blank" >RIV/68407700:21230/23:00367592 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1609/aaai.v37i8.26197" target="_blank" >https://doi.org/10.1609/aaai.v37i8.26197</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1609/aaai.v37i8.26197" target="_blank" >10.1609/aaai.v37i8.26197</a>

Result language

angličtina

Original language name

Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles between Client Data Subspaces

Original language description

Clustered federated learning (FL) has been shown to produce promising results by grouping clients into clusters. This is especially effective in scenarios where separate groups of clients have significant differences in the distributions of their local data. Existing clustered FL algorithms are essentially trying to group together clients with similar distributions so that clients in the same cluster can leverage each other's data to better perform federated learning. However, prior clustered FL algorithms attempt to learn these distribution similarities indirectly during training, which can be quite time consuming as many rounds of federated learning may be required until the formation of clusters is stabilized. In this paper, we propose a new approach to federated learning that directly aims to efficiently identify distribution similarities among clients by analyzing the principal angles between the client data subspaces. Each client applies a truncated singular value decomposition (SVD) step on its local data in a single-shot manner to derive a small set of principal vectors, which provides a signature that succinctly captures the main characteristics of the underlying distribution. This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters. This is achieved by comparing the similarities of the principal angles between the client data subspaces spanned by those principal vectors. The approach provides a simple, yet effective clustered FL framework that addresses a broad range of data heterogeneity issues beyond simpler forms of Non-IIDness like label skews. Our clustered FL approach also enables convergence guarantees for non-convex objectives.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 37th AAAI Conference on Artificial Intelligence

ISBN

978-1-57735-880-0

ISSN

2159-5399

e-ISSN

2374-3468

Number of pages

10

Pages from-to

10043-10052

Publisher name

AAAI Press

Place of publication

Menlo Park

Event location

Washington, DC

Event date

Feb 7, 2023

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—