External Validation of a Hip-Worn Accelerometry-Based Machine Learning Model for Physical Behavior Classification in Free-Living Conditions

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F62690094%3A18470%2F25%3A50023032" target="_blank" >RIV/62690094:18470/25:50023032 - isvavai.cz</a>

Výsledek na webu

<a href="https://journals.humankinetics.com/view/journals/jmpb/8/1/article-jmpb.2025-0030.xml" target="_blank" >https://journals.humankinetics.com/view/journals/jmpb/8/1/article-jmpb.2025-0030.xml</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1123/jmpb.2025-0030" target="_blank" >10.1123/jmpb.2025-0030</a>

Jazyk výsledku

angličtina

Název v původním jazyce

External Validation of a Hip-Worn Accelerometry-Based Machine Learning Model for Physical Behavior Classification in Free-Living Conditions

Popis výsledku v původním jazyce

Background: Accurate classification of physical behavior from accelerometer data is crucial for health and behavioral research. While machine learning models often perform well within the populations they are trained on, they are rarely validated on independent populations, and their generalizability remains poorly understood. Therefore, we aimed to externally validate a widely used random forest model for physical behavior classification, and to assess whether its performance varied by participants’ age, sex, or body mass index. Methods: We validated the random forest classifier, trained by Ellis et al., which achieved a balanced accuracy of 79% for classifying sitting, standing, and walking/running from hip-worn accelerometer data in the original training population. For the external validation, we obtained ActiGraph recordings for 610 participants from four European countries from the WEALTH (WEarable sensor Assessment of physicaL and eaTing beHaviors) project, which were labeled with the corresponding free-living behavior using ecological momentary assessment. Classifier performance was assessed using confusion matrices, precision, recall, F-score, and balanced accuracy. Results: In the WEALTH population, the random forest classifier achieved a balanced accuracy of 40% and an average F-score of 0.33. Precision and recall were highest for sitting, followed by walking/running and standing. Performance was consistent across subpopulations defined by age, sex, and body mass index. Conclusion: The substantial reduction in accuracy demonstrates the limited generalizability of the existing random forest classifier. Our findings underscore the need for external validation and more diverse training data to ensure robust application of machine learning models in physical behavior research. © 2025 Human Kinetics, Inc.

Název v anglickém jazyce

External Validation of a Hip-Worn Accelerometry-Based Machine Learning Model for Physical Behavior Classification in Free-Living Conditions

Popis výsledku anglicky

Background: Accurate classification of physical behavior from accelerometer data is crucial for health and behavioral research. While machine learning models often perform well within the populations they are trained on, they are rarely validated on independent populations, and their generalizability remains poorly understood. Therefore, we aimed to externally validate a widely used random forest model for physical behavior classification, and to assess whether its performance varied by participants’ age, sex, or body mass index. Methods: We validated the random forest classifier, trained by Ellis et al., which achieved a balanced accuracy of 79% for classifying sitting, standing, and walking/running from hip-worn accelerometer data in the original training population. For the external validation, we obtained ActiGraph recordings for 610 participants from four European countries from the WEALTH (WEarable sensor Assessment of physicaL and eaTing beHaviors) project, which were labeled with the corresponding free-living behavior using ecological momentary assessment. Classifier performance was assessed using confusion matrices, precision, recall, F-score, and balanced accuracy. Results: In the WEALTH population, the random forest classifier achieved a balanced accuracy of 40% and an average F-score of 0.33. Precision and recall were highest for sitting, followed by walking/running and standing. Performance was consistent across subpopulations defined by age, sex, and body mass index. Conclusion: The substantial reduction in accuracy demonstrates the limited generalizability of the existing random forest classifier. Our findings underscore the need for external validation and more diverse training data to ensure robust application of machine learning models in physical behavior research. © 2025 Human Kinetics, Inc.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

R - Projekt Ramcoveho programu EK

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Journal for the Measurement of Physical Behaviour

ISSN

2575-6605

e-ISSN

2575-6613

Svazek periodika

8

Číslo periodika v rámci svazku

1

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

9

Strana od-do

"Article Number: 20250030"

Kód UT WoS článku

001678562800007

EID výsledku v databázi Scopus

2-s2.0-105027529261