Conditionally dependent strategies for multiple-step-ahead prediction in local learning

[en] Computational intelligence approaches to multiple-step-ahead forecasting rely on either iterated one-step-ahead predictors or direct predictors. In both cases the predictions are obtained by means of multi-input single-output modeling techniques. This paper discusses the limitations of single-output approaches when the predictor is expected to return a long series of future values, and presents a multi-output approach to long term prediction. The motivation for this work is that, when predicting multiple steps ahead, the forecasted sequence should preserve the stochastic properties of the training series. However, this may not be the case, for instance in direct approaches where predictions for different horizons are produced independently. We discuss here a multi-output extension of conventional local modeling approaches, and present and compare three distinct criteria for performing conditionally dependent model selection. In order to assess the effectiveness of the different selection strategies, we carry out an extensive experimental session based on the 111 series in the NN5 competition. © 2010 International Institute of Forecasters.

Disciplines :

Computer science

Author, co-author :

Bontempi, Gianluca; Machine Learning Group, Département d'Informatique, Faculté des Sciences, ULB, Université Libre de Bruxelles, 1050 Bruxelles, Belgium

Ben taieb, Souhaib ; Université de Mons - UMONS > Faculté des Sciences > Service Big Data and Machine Learning

Language :

English

Title :

Conditionally dependent strategies for multiple-step-ahead prediction in local learning

Publication date :

July 2011

Journal title :

International Journal of Forecasting

ISSN :

0169-2070

eISSN :

1872-8200

Publisher :

Elsevier BV

Volume :

Issue :

Pages :

689 - 699

Peer reviewed :

Peer Reviewed verified by ORBi

Additional URL :

https://api.elsevier.com/content/article/PII:S0169207010001433?httpAccept=text/xml

Research unit :

S861 - Big Data and Machine Learning

Research institute :

Infortech
Complexys

Available on ORBi UMONS :

since 03 July 2023

Statistics

Number of views

24 (0 by UMONS)

Number of downloads

0 (0 by UMONS)

More statistics

Scopus citations^®

Scopus citations^®
without self-citations

OpenCitations

OpenAlex citations

Bibliography

Allen D.M. The relationship between variable and data augmentation and a method of prediction. Technometrics 1974, 16:125-127.
Anderson T.W. The statistical analysis of time series 1971, J. Wiley and Sons.
Andrawis R.R., Atiya A.F., El Shishiny H. Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition. International Journal of Forecasting 2011, 27(3):672-688.
Armstrong J., Collopy F. Error measures for generalizing about forecasting methods: empirical comparisons. International Journal of Forecasting 1992, 8:69-80.
Atkeson C.G., Moore A.W., Schaal S. Locally weighted learning. Artificial Intelligence Review 1997, 11(1-5):11-73.
Auestad B., Tjostheim D. Identification of nonlinear time series: first order characterization and order determination. Biometrika 1990, 77:669-687.
Ben Taieb, S., Bontempi, G., Sorjamaa, A., & Lendasse, A. (2009). Long-term prediction of time series by combining direct and MIMO strategies. In Proceedings of the 2009 IEEE international joint conference on neural networks (pp. 3054-3061).
Ben Taieb S., Sorjamaa A., Bontempi G. Multiple-output modelling for multi-step-ahead forecasting. Neurocomputing 2010, 73:1950-1957.
Birattari M., Bontempi G., Bersini H. Lazy learning meets the recursive least-squares algorithm. NIPS 1999, Vol. 11:375-381. MIT Press, Cambridge. M.S. Kearns, S.A. Solla, D.A. Cohn (Eds.).
Bontempi, G. (1999). Local learning techniques for modeling, prediction and control. Ph.D. thesis. IRIDIA-Université Libre de Bruxelles.
Bontempi, G. (2008). Long term time series prediction with multi-input multi-output local learning. In Proceedings of the 2nd European symposium on time series prediction. TSP. ESTSP08 (pp. 145-154).
Bontempi, G., Birattari, M., & Bersini, H. (1998). Lazy learning for iterated time series prediction. In J. A. K. Suykens, & J. Vandewalle (Eds.), Proceedings of the international workshop on advanced black-box techniques for nonlinear modeling. Katholieke Universiteit Leuven, Belgium (pp. 62-68).
Bontempi G., Birattari M., Bersini H. Lazy learning for modeling and control design. International Journal of Control 1999, 72(7-8):643-658.
Bontempi G., Birattari M., Bersini H. Local learning for iterated time-series prediction. Machine learning: proceedings of the sixteenth international conference 1999, 32-38. Morgan Kaufmann Publishers, San Francisco, CA. I. Bratko, S. Dzeroski (Eds.).
Chen R., Yang L., Hafner C. Nonparametric multistep-ahead prediction in time series analysis. Journal of the Royal Statistical Society, Series B 2004, 66:669-686.
Crone, S. (2008). Results of the NN5 time series forecasting competition. In WCCI 2008, IEEE world congress on computational intelligence.
Fan J., Gijbels I. Adaptive order polynomial fitting: bandwidth robustification and bias reduction. Journal of Computational and Graphical Statistics 1995, 4:213-227.
Fan J., Gijbels I. Local polynomial modelling and its applications 1996, Chapman and Hall.
Fan J., Yao Q. Nonlinear time series 2005, Springer.
Farmer J.D., Sidorowich J.J. Predicting chaotic time series. Physical Review Letters 1987, 8(59):845-848.
Gao J. Nonlinear time series: semiparametric and nonparametric methods 2007, Chapman and Hall.
Guo M., Bai Z., An H. Multi-step prediction for nonlinear autoregressive models based on empirical distributions. Statistica Sinica 1999, 9:559-570.
Härdle W., Vieu P. Kernel regression smoothing for time series. Journal of Time Series Analysis 1992, 13:209-232.
Jensen F.V. Bayesian networks and decision graphs 2001, Springer.
Lapedes, A., & Farber, R. (1987). Nonlinear signal processing using neural networks. Tech. rep. LA-UR-87-2662. Los Alamos National Laboratory. Los Alamos, NM.
Matias, J. M. (2005). Multi-output nonparametric regression. In C. Bento, A. Cardoso, & G. Dias (Eds.), Progress in artificial intelligence (pp. 288-292).
McNames, J. (1998). A nearest trajectory strategy for time series prediction. In Proceedings of the international workshop on advanced black-box techniques for nonlinear modeling (pp. 112-128).
McNames J., Suykens J., Vandewalle J. Winning contribution of the K.U. Leuven time-series prediction competition. International Journal of Bifurcation and Chaos 1999, 9:1485-1501.
Pemberton J. Exact least squares multi-step prediction from nonlinear autoregressive models. Journal of Time Series Analysis 1987, 8:443-448.
Ruppert D., Sheather S.J., Wand M.P. An effective bandwidth selector for local least squares regression. Journal of the American Statistical Association 1995, 90:1257-1270.
Sauer T. Time series prediction by using delay coordinate embedding. Time series prediction: forecasting the future and understanding the past 1994, 175-193. Addison Wesley, Harlow, UK. A.S. Weigend, N.A. Gershenfeld (Eds.).
Sorjamaa A., Hao J., Reyhani N., Ji Y., Lendasse A. Methodology for long-term prediction of time series. Neurocomputing 2007, 70(16-18):2861-2869.
Tashman L.J. Out-of-sample tests of forecasting accuracy: an analysis and review. International Journal of Forecasting 2000, 16:437-450.
Tong H. Threshold models in nonlinear time series analysis 1983, Springer Verlag, Berlin.
Verleysen M., Francois D. The curse of dimensionality in data mining and time series prediction. Lecture notes in computer science 2005, Vol. 3512. Springer. J. Cabestany, A. Prieto, F. Hernandez (Eds.).
Weigend A.S., Huberman B.A., Rumelhart D.E. Predicting sunspots and exchange rates with connectionist networks. Nonlinear modeling and forecasting 1992, 395-432. Addison-Wesley. M. Casdagli, S. Eubank (Eds.).
Williams R., Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1989, 1:270-280.
Zhang X., Hutchinson J. Simple architectures on fast machines: practical issues in nonlinear time series prediction. Time series prediction: forecasting the future and understanding the past 1994, 219-241. Addison Wesley. A.S. Weigend, N.A. Gershenfeld (Eds.).
Zhang G., Patuwo B.E., Hu M.Y. Forecasting with artificial neural networks: the state of the art. International Journal of Forecasting 1998, 14:35-62.