The Impact of Empirical Accuracy Studies On Time Series Analysis and Forecasting

Abstract

This paper examines a major area of statistics: the application of time series analysis to forecasting, particularly as it applies to the areas of business and economics. This area is unusual in the social sciences in that it permits objective, replicable and controlled experimentation through empirical studies using real-life data. In this paper we review the considerable empirical research carried out within the field over the last two decades and compare the major findings of such research with what may be expected if the accepted statistical paradigm held true. In doing so we note several anomalies which cannot be easily explained. Using citation analysis, we demonstrate that there has been little theoretical work that has taken such findings into account. The result has been little or no progress in re-defining the dominant paradigm in time series statistics and progressing the field in the direction of improved post-sample forecasting accuracy. We argue that whether the objective is post-sample forecasting accuracy or model validation, the strong assumption usually made in the field of time series forecasting of constancy (or worse, stationarity) must be re-examined. For application-oriented and empirically-based researchers the need for a theoretical framework in which to develop improved forecasting methods and establish effective selection criteria is a necessity. Various priority areas for research are described including robust modelling and the use of contextual information in model identification. The paper concludes with a challenge to theoretical time series statisticians and empirical researchers alike: working together can they learn from each other? If successful, their conclusions should advance the field to better serve those engaged in decision or policy making through the benefits from more accurate predictions. Equally important, forecasting competitions can provide researchers with an experimental test• bed that can direct the progress of their discipline and make it more useful and relevant for real life applications.