Advertisement

An optimized time series combined forecasting method based on neural networks

  • Kaiyi Zhao
  • Li Li
  • Saihua Cai
  • Ruizhi SunEmail author
Article
  • 27 Downloads

Abstract

In the field of time series forecasting, combining forecasts from multiple models significantly improves the forecasting precision as well as often produces better forecasts than each constituent model. The linear method is the main method in the current literatures for it is simpler and more efficient, and usually gives good results. However, the selection of the basic unit models and the determination of combination weights always bring difficulties to combined forecasting model. In addition, there is usually more time consuming in the combining model. To address these problems, this paper proposes an optimized time series combined forecasting method based on neural networks. The principle of the proposed method adheres to two primary aspects. (a) A multi-threaded grid search method is proposed to quickly determine the number of hidden layer nodes in a neural network for a scenario. (b) A method based on sample dynamic partitioning to explore the weight generation mode is proposed to determine the combined forecasting. Empirical results from six real-world time series show that the superiority of our approach in forecasting accuracies.

Keywords

Time series forecasting Linear combination Multi-threaded Neural networks 

Notes

References

  1. 1.
    Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environ Model 15(1):101–124CrossRefGoogle Scholar
  2. 2.
    Li H-Z, Guo S, Li C-J, Sun J-Q (2013) A hybrid annual power load forecasting model based on generalized regression neural network with fruit fly optimization algorithm. Knowl Based Syst 37:378–387CrossRefGoogle Scholar
  3. 3.
    Bodyanskiy Y, Popov S (2006) Neural network approach to forecasting of quasiperiodic financial time series. Eur J Oper Res 175(3):1357–1366MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Song H, Li G (2008) Tourism demand modelling and forecasting—a review of recent research. Tour Manag 29(2):203–220CrossRefGoogle Scholar
  5. 5.
    Chen C-F, Lai M-C, Yeh C-C (2012) Forecasting tourism demand based on empirical mode decomposition and neural network. Knowl Based Syst 26:281–287CrossRefGoogle Scholar
  6. 6.
    Lu X, Wang J, Cai Y, Zhao J (2015) Distributed HS-ARTMAP and its forecasting model for electricity load. Appl Soft Comput 32:13–22CrossRefGoogle Scholar
  7. 7.
    Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. Wiley, HobokenzbMATHGoogle Scholar
  8. 8.
    Pellegrini S, Ruiz E, Espasa A (2011) Prediction intervals in conditionally heteroscedastic time series with stochastic components. Int J Forecast 27(2):308–319CrossRefGoogle Scholar
  9. 9.
    Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Eural Process Lett 9(3):293–300CrossRefGoogle Scholar
  10. 10.
    Vapnik V (2013) The nature of statistical learning theory. Springer, BerlinzbMATHGoogle Scholar
  11. 11.
    Zhang GP (2007) A neural network ensemble method with jittered training data for time series forecasting. Inf Sci 177(23):5329–5346CrossRefGoogle Scholar
  12. 12.
    Bates JM, Granger CW (1969) The combination of forecasts. J Oper Res Soc 20(4):451–468CrossRefGoogle Scholar
  13. 13.
    Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Stat Sci 14(4):382–401MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Jose VRR, Winkler RL (2008) Simple robust averages of forecasts: some empirical results. Int J Forecast 24(1):163–169CrossRefGoogle Scholar
  15. 15.
    De Gooijer JG, Hyndman RJ (2006) 25 years of time series forecasting. Int J Forecast 22(3):443–473CrossRefGoogle Scholar
  16. 16.
    Armstrong JS (2001) Principles of forecasting: a handbook for researchers and practitioners, vol 30. Springer, BerlinCrossRefGoogle Scholar
  17. 17.
    Aksu C, Gunter SI (1992) An empirical analysis of the accuracy of SA, OLS, ERLS and NRLS combination forecasts. Int J Forecast 8(1):27–43CrossRefGoogle Scholar
  18. 18.
    Granger CW, Ramanathan R (1984) Improved methods of combining forecasts. J Forecast 3(2):197–204CrossRefGoogle Scholar
  19. 19.
    Lemke C, Gabrys B (2010) Meta-learning for time series forecasting and forecast combination. Neurocomputing 73(10–12):2006–2016CrossRefGoogle Scholar
  20. 20.
    Bunn DW (1975) A Bayesian approach to the linear combination of forecasts. J Oper Res Soc 26(2):325–329zbMATHCrossRefGoogle Scholar
  21. 21.
    Aiolfi M, Timmermann A (2006) Persistence in forecasting performance and conditional combination strategies. J Econom 135(1–2):31–53MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Andrawis RR, Atiya AF, El-Shishiny H (2011) Forecast combinations of computational intelligence and linear models for the NN5 time series forecasting competition. Int J Forecast 27(3):672–688CrossRefGoogle Scholar
  23. 23.
    Lin C-C, Lin C-L, Shyu JZ (2014) Hybrid multi-model forecasting system: a case study on display market. Knowl Based Syst 71:279–289CrossRefGoogle Scholar
  24. 24.
    Adhikari R (2015) A neural network based linear ensemble framework for time series forecasting. Neurocomputing 157:231–242CrossRefGoogle Scholar
  25. 25.
    Zhou Z-H, Wu J, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1–2):239–263MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Adhikari R (2015) A mutual association based nonlinear ensemble mechanism for time series forecasting. Appl Intell 43(2):233–250CrossRefGoogle Scholar
  27. 27.
    Rumelhart DE, McClelland JL (1986) Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge, MAGoogle Scholar
  28. 28.
    Shen W, Guo X, Wu C, Wu D (2011) Forecasting stock indices using radial basis function neural networks optimized by artificial fish swarm algorithm. Knowl Based Syst 24(3):378–385CrossRefGoogle Scholar
  29. 29.
    Wang S, Zhang N, Wu L, Wang Y (2016) Wind speed forecasting based on the hybrid ensemble empirical mode decomposition and GA-BP neural network method. Renew Energy 94:629–636CrossRefGoogle Scholar
  30. 30.
    Yu F, Xu X (2014) A short-term load forecasting model of natural gas based on optimized genetic algorithm and improved BP neural network. Appl Energy 134:102–113CrossRefGoogle Scholar
  31. 31.
    Wang D, Luo H, Grunder O, Lin Y, Guo H (2017) Multi-step ahead electricity price forecasting using a hybrid model based on two-layer decomposition technique and BP neural network optimized by firefly algorithm. Appl Energy 190:390–407CrossRefGoogle Scholar
  32. 32.
    Zhang GP (2003) Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing 50:159–175zbMATHCrossRefGoogle Scholar
  33. 33.
    Wang L, Zeng Y, Zhang J, Huang W, Bao Y (2006) The criticality of spare parts evaluating model using artificial neural network approach. In: International Conference on Computational Science. Springer, pp 728–735Google Scholar
  34. 34.
    Wang L, Zeng Y, Chen T (2015) Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst Appl 42(2):855–863CrossRefGoogle Scholar
  35. 35.
    Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRefGoogle Scholar
  36. 36.
    Gers FA, Schmidhuber J, Cummins F (1999) Learning to forget: continual prediction with LSTM. Neural Comput 12(10):2451–2471CrossRefGoogle Scholar
  37. 37.
    Zhao Z, Chen W, Wu X, Chen PC, Liu J (2017) LSTM network: a deep learning approach for short-term traffic forecast. IET Intel Transp Syst 11(2):68–75CrossRefGoogle Scholar
  38. 38.
    Zheng H, Yuan J, Chen L (2017) Short-term load forecasting using EMD-LSTM neural networks with a Xgboost algorithm for feature importance evaluation. Energies 10(8):1168CrossRefGoogle Scholar
  39. 39.
    Jaeger H (2001) The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn Ger Ger Natl Res Center Inf Technol GMD Tech Rep 148(34):13Google Scholar
  40. 40.
    Jaeger H, Haas H (2004) Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667):78–80CrossRefGoogle Scholar
  41. 41.
    Jaeger H (2007) Echo state network. Scholarpedia 2(9):2330CrossRefGoogle Scholar
  42. 42.
    Chatzis SP, Demiris Y (2011) Echo state Gaussian process. IEEE Trans Neural Netw 22(9):1435–1445CrossRefGoogle Scholar
  43. 43.
    Bianchi FM, Scardapane S, Uncini A, Rizzi A, Sadeghian A (2015) Prediction of telephone calls load using Echo State Network with exogenous variables. Neural Netw 71:204–213CrossRefGoogle Scholar
  44. 44.
    Chouikhi N, Ammar B, Rokbani N, Alimi AM (2017) PSO-based analysis of Echo State Network parameters for time series forecasting. Appl Soft Comput 55:211–225CrossRefGoogle Scholar
  45. 45.
    Shen L, Chen J, Zeng Z, Yang J, Jin J (2018) A novel echo state network for multivariate and nonlinear time series prediction. Appl Soft Comput 62:524–535CrossRefGoogle Scholar
  46. 46.
    Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078 (preprint)
  47. 47.
    Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555 (preprint)
  48. 48.
    Wang S, Zhao X, Li M, Wang H (2018) TRSWA-BP neural network for dynamic wind power forecasting based on entropy evaluation. Entropy 20(4):283CrossRefGoogle Scholar
  49. 49.
    Shao-Jiang L, Jia-Ying C, Zhi-Xue L (2018) A EMD-BP integrated model to forecast tourist number and applied to Jiuzhaigou. J Intell Fuzzy Syst 34(2):1045–1052CrossRefGoogle Scholar
  50. 50.
    Wentao X, Jianjun H, Aiguo S, Xinyu Z (2017) An improved forecast of ship sway based on ESN. In: 2017 4th International Conference on Information, Cybernetics and Computational Social Systems (ICCSS). IEEE, pp 422–425Google Scholar
  51. 51.
    Li J, Li Q (2015) Medium term electricity load forecasting based on CEEMDAN-permutation entropy and ESN with leaky integrator neurons. Electr Mach Control 8:012Google Scholar
  52. 52.
    Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing 323:203–213CrossRefGoogle Scholar
  53. 53.
    Baek Y, Kim HY (2018) ModAugNet: a new forecasting framework for stock market index value with an overfitting prevention LSTM module and a prediction LSTM module. Expert Syst Appl 113:457–480CrossRefGoogle Scholar
  54. 54.
    Tokgöz A, Ünal G (2018) A RNN based time series approach for forecasting turkish electricity load. In: 2018 26th Signal Processing and Communications Applications Conference (SIU). IEEE, pp 1–4Google Scholar
  55. 55.
    Zu X, Song R (2018) Short-term wind power prediction method based on wavelet packet decomposition and improved GRU. In: Journal of Physics: Conference Series, vol 2. IOP Publishing, p 022034Google Scholar
  56. 56.
    Freitas PS, Rodrigues AJ (2006) Model combination in neural-based forecasting. Eur J Oper Res 173(3):801–814MathSciNetzbMATHCrossRefGoogle Scholar
  57. 57.
    De Menezes LM, Bunn DW, Taylor JW (2000) Review of guidelines for the use of combined forecasts. Eur J Oper Res 120(1):190–204zbMATHCrossRefGoogle Scholar
  58. 58.
    Timmermann A (2006) Forecast combinations. Handb Econ Forecast 1:135–196CrossRefGoogle Scholar
  59. 59.
    Hyndman RJ (2019) Time series data library. https://datamarket.com/data/list/?q=provider:tsdl
  60. 60.
    Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980. Accessed 14 Nov 2018 (preprint)
  61. 61.
    Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International Conference on Machine Learning, pp 1139–1147Google Scholar
  62. 62.
    Dozat T (2016) Incorporating nesterov momentum into adam. Dostupnéz. http://cs229.stanford.edu/proj/054_report.pdf
  63. 63.
    Wang L, Wang Z, Qu H, Liu S (2018) Optimal forecast combination based on neural networks for time series forecasting. Appl Soft Comput 66:1–17CrossRefGoogle Scholar
  64. 64.
    Steel RGD, Torrie JH (1960) Principles and procedures of statistics: with special reference to the biological sciences. McGraw-Hill, New YorkzbMATHGoogle Scholar
  65. 65.
    Glantz SA, Slinker BK, Neilands TB (1990) Primer of applied regression and analysis of variance, vol 309. McGraw-Hill, New YorkGoogle Scholar
  66. 66.
    Draper NR, Smith H (2014) Fitting a straight line by least squares. WileyGoogle Scholar
  67. 67.
    Box GE, Jenkins GM (1970) Time series analysis forecasting and control. Wisconsin Univ Madison Dept of Statistics, MadisonzbMATHGoogle Scholar
  68. 68.
    Ziegel ER, Box GEP, Jenkins GM, Reinsel GC (1995) Time series analysis, forecasting, and control. Technometrics 37(2):238Google Scholar
  69. 69.
    Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14(1):35–62CrossRefGoogle Scholar
  70. 70.
    Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211CrossRefGoogle Scholar
  71. 71.
    Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366zbMATHCrossRefGoogle Scholar
  72. 72.
    Lim CP, Goh WY (2007) The application of an ensemble of boosted elman networks to time series prediction: a benchmark study. World Acad Sci Eng Technol Int J Electr Comput Energ Electron Commun Eng 1(10):1518–1525Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Information and Electrical EngineeringChina Agricultural UniversityBeijingChina
  2. 2.Scientific Research Base for Integrated Technologies of Precision Agriculture (Animal Husbandry)the Ministry of AgricultureBeijingChina

Personalised recommendations