Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985556%3A_____%2F23%3A00572279" target="_blank" >RIV/67985556:_____/23:00572279 - isvavai.cz</a>

Nalezeny alternativní kódy

RIV/67985807:_____/23:00572279 RIV/00216224:14310/23:00130636 RIV/00216208:11120/23:43925400

Výsledek na webu

<a href="https://dx.doi.org/10.7554/eLife.81916" target="_blank" >https://dx.doi.org/10.7554/eLife.81916</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.7554/eLife.81916" target="_blank" >10.7554/eLife.81916</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Popis výsledku v původním jazyce

BACKGROUND: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts’ predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022. METHODS: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance. RESULTS: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models. CONCLUSIONS: Our results support combining independent models into an ensemble forecast to improve epidemiological predictions, and suggest that median averages yield better performance than methods based on means. We highlight that forecast consumers should place more weight on incident death forecasts than case forecasts at horizons greater than two weeks. FUNDING: European Commission, Ministerio de Ciencia, Innovación y Universidades, FEDER66666 - Agència de Qualitat i Avaluació Sanitàries de Catalunya - Netzwerk Universitätsmedizin - Health Protection Research Unit - Wellcome Trust - European Centre for Disease Prevention and Control - Ministry of Science and Higher Education of Poland - Federal Ministry of Education and Research - Los Alamos National Laboratory - German Free State of Saxony - NCBiR - FISR 2020 Covid-19 I Fase - Spanish Ministry of Health / REACT-UE (FEDER) - National Institutes of General Medical Sciences - Ministerio de Sanidad/ISCIII - PERISCOPE European H2020 - PERISCOPE European H2021 - InPresa - National Institutes of Health, NSF, US Centers for Disease Control and Prevention, Google, University of Virginia, Defense Threat Reduction Agency.

Název v anglickém jazyce

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

Popis výsledku anglicky

BACKGROUND: Short-term forecasts of infectious disease contribute to situational awareness and capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise forecasts’ predictive performance by combining independent models into an ensemble. Here we report the performance of ensemble predictions of COVID-19 cases and deaths across Europe from March 2021 to March 2022. METHODS: We created the European COVID-19 Forecast Hub, an online open-access platform where modellers upload weekly forecasts for 32 countries with results publicly visualised and evaluated. We created a weekly ensemble forecast from the equally-weighted average across individual models' predictive quantiles. We measured forecast accuracy using a baseline and relative Weighted Interval Score (rWIS). We retrospectively explored ensemble methods, including weighting by past performance. RESULTS: We collected weekly forecasts from 48 models, of which we evaluated 29 models alongside the ensemble model. The ensemble had a consistently strong performance across countries over time, performing better on rWIS than 91% of forecasts for deaths (N=763 predictions from 20 models), and 83% forecasts for cases (N=886 predictions from 23 models). Performance remained stable over a 4-week horizon for death forecasts but declined with longer horizons for cases. Among ensemble methods, the most influential choice came from using a median average instead of the mean, regardless of weighting component models. CONCLUSIONS: Our results support combining independent models into an ensemble forecast to improve epidemiological predictions, and suggest that median averages yield better performance than methods based on means. We highlight that forecast consumers should place more weight on incident death forecasts than case forecasts at horizons greater than two weeks. FUNDING: European Commission, Ministerio de Ciencia, Innovación y Universidades, FEDER66666 - Agència de Qualitat i Avaluació Sanitàries de Catalunya - Netzwerk Universitätsmedizin - Health Protection Research Unit - Wellcome Trust - European Centre for Disease Prevention and Control - Ministry of Science and Higher Education of Poland - Federal Ministry of Education and Research - Los Alamos National Laboratory - German Free State of Saxony - NCBiR - FISR 2020 Covid-19 I Fase - Spanish Ministry of Health / REACT-UE (FEDER) - National Institutes of General Medical Sciences - Ministerio de Sanidad/ISCIII - PERISCOPE European H2020 - PERISCOPE European H2021 - InPresa - National Institutes of Health, NSF, US Centers for Disease Control and Prevention, Google, University of Virginia, Defense Threat Reduction Agency.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10103 - Statistics and probability

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

eLife

ISSN

2050-084X

e-ISSN

2050-084X

Svazek periodika

12

Číslo periodika v rámci svazku

April 2023

Stát vydavatele periodika

GB - Spojené království Velké Británie a Severního Irska

Počet stran výsledku

19

Strana od-do

e81916

Kód UT WoS článku

001009734700001

EID výsledku v databázi Scopus

2-s2.0-85158061738