Optimal out-of-sample forecast evaluation under stationarity

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11640%2F23%3A00577629" target="_blank" >RIV/00216208:11640/23:00577629 - isvavai.cz</a>

Výsledek na webu

<a href="https://doi.org/10.1002/for.3013" target="_blank" >https://doi.org/10.1002/for.3013</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1002/for.3013" target="_blank" >10.1002/for.3013</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Optimal out-of-sample forecast evaluation under stationarity

Popis výsledku v původním jazyce

It is a common practice to split a time series into an in-sample and pseudo-out-of-sample segments and estimate the out-of-sample loss for a given statistical model by evaluating forecasting performance over the pseudo-out-of-sample segment. We propose an alternative estimator of the out-of-sample loss, which, contrary to the conventional wisdom, utilizes criteria measured both in- and out-of-sample via a carefully constructed system of affine weights. We prove that, provided that the time series is stationary, the proposed estimator is the best linear unbiased estimator of the out-of-sample loss and outperforms the conventional estimator in terms of sampling variability. Application of the optimal estimator to Diebold–Mariano type tests of predictive ability leads to a substantial power gain without increasing finite sample size distortions. An extensive evaluation on real-world time series from the M4 forecasting competition confirms superiority of the proposed estimator and also demonstrates substantial robustness to violations of the underlying assumption of stationarity.

Název v anglickém jazyce

Optimal out-of-sample forecast evaluation under stationarity

Popis výsledku anglicky

It is a common practice to split a time series into an in-sample and pseudo-out-of-sample segments and estimate the out-of-sample loss for a given statistical model by evaluating forecasting performance over the pseudo-out-of-sample segment. We propose an alternative estimator of the out-of-sample loss, which, contrary to the conventional wisdom, utilizes criteria measured both in- and out-of-sample via a carefully constructed system of affine weights. We prove that, provided that the time series is stationary, the proposed estimator is the best linear unbiased estimator of the out-of-sample loss and outperforms the conventional estimator in terms of sampling variability. Application of the optimal estimator to Diebold–Mariano type tests of predictive ability leads to a substantial power gain without increasing finite sample size distortions. An extensive evaluation on real-world time series from the M4 forecasting competition confirms superiority of the proposed estimator and also demonstrates substantial robustness to violations of the underlying assumption of stationarity.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

50202 - Applied Economics, Econometrics

Projekt

—

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Journal of Forecasting

ISSN

0277-6693

e-ISSN

1099-131X

Svazek periodika

42

Číslo periodika v rámci svazku

8

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

31

Strana od-do

2249-2279

Kód UT WoS článku

001038631000001

EID výsledku v databázi Scopus

2-s2.0-85166584585