Journal of Economics and Financial Analysis, 1 (1), pp. 1-13, [2017]
URI: https://ojs.tripaledu.com/index.php/jefa/article/view/3/1

Does Statistical Significance Help to Evaluate Predictive Performance of Competing Models?


Georgia State University, United States


DOI: http://dx.doi.org/10.1991/jefa.v1i1.a1

Abstract

In Monte Carlo experiment with simulated data, we show that as a point forecast criterion, the Clark and West's (2006) unconditional test of mean squared prediction errors does not reflect the relative performance of a superior model over a relatively weaker one. The simulation results show that even though the mean squared prediction errors of a constructed superior model is far below a weaker alternative, the Clark- West test does not reflect this in their test statistics. Therefore, studies that use this statistic in testing the predictive accuracy of alternative exchange rate models, stock return predictability, inflation forecasting, and unemployment forecasting should not weight too much on the magnitude of the statistically significant Clark-West tests statistics.

Keywords

Model comparison; Predictive accuracy; Point-forecast criterion; The Clark and West test; Monte-Carlo methods; Forecast comparison.

JEL Classification

F37, F47, G17, C52.

Full Text:

PDF
HTML

References

Arai, N. (2014). Using forecast evaluation to improve the accuracy of the Greenbook forecast. International Journal of Forecasting, 30 (1), 12-19.

Clark, T.E., and West, K.D. (2006). Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis. Journal of Econometrics, 135, 155–186.

Diebold, F.X. (2015). Comparing predictive accuracy, twenty years later: A personal perspective on the use and abuse of Diebold–Mariano tests. Journal of Business & Economic Statistics, 33 (1), 1-24.

Diebold, F., and Mariano, R. (1995). Comparing predictive accuracy. Journal of Business and Economic Statistics, 13, 253–263.

Dimpfl, T., and Jank, S. (2015). Can internet search queries help to predict stock market volatility?. European Financial Management. doi: 10.1111/eufm.12058.

Engel, C., Mark, N.C., and West, K.D. (2015). Factor model forecasts of exchange rates. Econometric Reviews, 34 (1-2), 32-55.

Faust, J., and Wright, J.H. (2013). Forecasting inflation. Handbook of economic forecasting, 2 (Part A), 3-56.

Ferraro, D., Rogoff, K., and Rossi, B. (2015). Can oil prices forecast exchange rates? An empirical analysis of the relationship between commodity prices and exchange rates. Journal of International Money and Finance, 54, 116-141.

Gregory, A.W., and Zhu, H. (2014). Testing the value of lead information in forecasting monthly changes in employment from the Bureau of Labor Statistics. Applied Financial Economics, 24 (7), 505-514.

Hutter, C., and Weber, E. (2015). Constructing a new leading indicator for unemployment from a survey among German employment agencies. Applied Economics, 47 (33), 3540-3558.

Löffler, G. (2013). Tower Building and Stock Market Returns. Journal of Financial Research, 36 (3), 413-434.

Nikolsko-Rzhevskyy, A., and Prodan, R. (2012). Markov switching and exchange rate predictability. International Journal of Forecasting, 28 (2), 353-365.

Sousa, R.M., Vivian, A., and Wohar, M.E. (2016). Predicting asset returns in the BRICS: The role of macroeconomic and fundamental predictors. International Review of Economics & Finance, 41, 122-143.

Theil, H. (1966). Applied Economic Forecasting. North-Holland Publishing Co., Amsterdam.

Wang, J., and Wu, J.J. (2012). The Taylor rule and forecast intervals for exchange rates. Journal of Money, Credit and Banking, 44 (1), 103-144.

West, K.D. (1996). Asymptotic inference about predictive ability. Econometrica, 64, 1067–1084.

Wu, S.J., and Lee, W.M. (2015). Predicting severe simultaneous bear stock markets using macroeconomic variables as leading indicators. Finance Research Letters, 13, 196-204.




Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.