Abstract
Macroeconomic forecasting is a very difficult task due to the lack of an accurate, convincing model of the economy. The most accurate models for economic forecasting, “black box” time series models, assume little about the structure of the economy. Constructing reliable time series models is challenging due to short data series, high noise levels, nonstationarities, and nonlinear effects. This chapter describes these challenges and presents some neural network solutions to them. Important issues include balancing the bias/variance tradeoff and the noise/nonstationarity tradeoff. A brief survey of methods includes hyperparameter selection (regularization parameter and training window length), input variable selection and pruning, network architecture selection and pruning, new smoothing regularizers, committee forecasts and model visualization. Separate sections present more in-depth descriptions of smoothing regularizers, architecture selection via the generalized prediction error (GPE) and nonlinear cross-validation (NCV), input selection via sensitivity based pruning (SBP), and model interpretation and visualization. Throughout, empirical results are presented for forecasting the U.S. Index of Industrial Production. These demonstrate that, relative to conventional linear time series and regression methods, superior performance can be obtained using state-of-the-art neural network models.
Previously published in: Orr, G.B. and Müller, K.-R. (Eds.): LNCS 1524, ISBN 978-3-540-65311-0 (1998).
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Akaike, H.: Statistical predictor identification. Ann. Inst. Statist. Math. 22, 203–217 (1970)
Ash, T.: Dynamic node creation in backpropagation neural networks. Connection Science 1(4), 365–375 (1989)
Barron, A.: Predicted squared error: a criterion for automatic model selection. In: Farlow, S. (ed.) Self–Organizing Methods in Modeling. Marcel Dekker, New York (1984)
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. on Neural Networks 5(4), 537–550 (1994)
Bonnlander, B.: Nonparametric selection of input variables for connectionist learning. Technical report, PhD Thesis. Department of Computer Science, University of Colorado (1996)
Clemen, R.T.: Combining forecasts: A review and annotated bibliography. International Journal of Forecasting (5), 559–583 (1989)
Craven, P., Wahba, G.: Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377–403 (1979)
Eubank, R.L.: Spline Smoothing and Nonparametric Regression. Marcel Dekker, Inc. (1988)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Computation 4(1), 1–58 (1992)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural network architectures. Neural Computation 7, 219–269 (1995)
Golub, G., Heath, H., Wahba, G.: Generalized cross validation as a method for choosing a good ridge parameter. Technometrics 21, 215–224 (1979)
Granger, C.W.J., Newbold, P.: Forecasting Economic Time Series, 2nd edn. Academic Press, San Diego (1986)
Granger, C.W.J., Terasvirta, T.: Modelling Nonlinear Economic Relationships. Oxford University Press (1993)
Hamilton, J.D.: Time Series Analysis. Princeton University Press (1994)
Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: Optimal brain surgeon. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 164–171. Morgan Kaufmann Publishers, San Mateo (1993)
Hastie, T.J., Tibshirani, R.J.: Generalized Additive Models. Monographs on Statistics and Applied Probability, vol. 43. Chapman and Hall (1990)
Hoerl, A.E., Kennard, R.W.: Ridge regression: applications to nonorthogonal problems. Technometrics 12, 69–82 (1970)
Hoerl, A.E., Kennard, R.W.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)
LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 2. Morgan Kaufmann Publishers (1990)
Levin, A.U., Leen, T.K., Moody, J.E.: Fast pruning using principal components. In: Cowan, J., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6. Morgan Kaufmann Publishers, San Francisco (1994)
Liao, Y., Moody, J.E.: A neural network visualization and sensitivity analysis toolkit. In: Amari, S.I., Xu, L., Chan, L.-W., King, I., Leung, K.-S. (eds.) Proceedings of the International Conference on Neural Information Processing, pp. 1069–1074. Springer-Verlag Singapore Pte. Ltd. (1996)
Litterman, R.B.: Forecasting with Bayesian vector autoregressions – five years of experience. Journal of Business and Economic Statistics 4(1), 25–38 (1986)
Moody, J.: Challenges of Economic Forecasting: Noise, Nonstationarity, and Nonlinearity. Invited talk presented at Machines that Learn., Snowbird Utah (April 1994)
Moody, J.: The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems. In: Moody, J.E., Hanson, S.J., Lippmann, R.P. (eds.) Advances in Neural Information Processing Systems, vol. 4, pp. 847–854. Morgan Kaufmann Publishers, San Mateo (1992)
Moody, J.: Prediction risk and neural network architecture selection. In: Cherkassky, V., Friedman, J.H., Wechsler, H. (eds.) From Statistics to Neural Networks: Theory and Pattern Recognition Applications. Springer (1994)
Moody, J.: Macroeconomic Forecasting: Challenges and Neural Network Solutions. In: Proceedings of the International Symposium on Artificial Neural Networks, Hsinchu, Taiwan (1995) (Invited keynote address)
Moody, J., Levin, A., Rehfuss, S.: Predicting the U.S. index of industrial production. Neural Network World 3(6), 791–794 (1993); Special Issue: Proceedings of Parallel Applications in Statistics and Economics 1993
Moody, J., Rehfuss, S., Saffell, M.: Macroeconomic forecasting with neural networks (1999) (manuscript in preparation)
Moody, J., Rögnvaldsson, T.: Smoothing regularizers for projective basis function networks. In: Proceedings of Advances in Neural Information Processing Systems, NIPS 1996, vol. 9. MIT Press, Cambridge (1997)
Moody, J., Utans, J.: Architecture selection strategies for neural networks: Application to corporate bond rating prediction. In: Refenes, A.N. (ed.) Neural Networks in the Captial Markets. John Wiley & Sons (1994)
Moody, J.E.: Note on generalization, regularization and architecture selection in nonlinear learning systems. In: Juang, B.H., Kung, S.Y., Kamm, C.A. (eds.) Neural Networks for Signal Processing, pp. 1–10. IEEE Signal Processing Society (1991)
Moody, J.E., Utans, J.: Principled architecture selection for neural networks: Application to corporate bond rating prediction. In: Moody, J.E., Hanson, S.J., Lippmann, R.P. (eds.) Advances in Neural Information Processing Systems, vol. 4, pp. 683–690. Morgan Kaufmann Publishers, San Mateo (1992)
Mozer, M.C., Smolensky, P.: Skeletonization: A technique for trimming the fat from a network via relevance assessment. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems, vol. 1. Morgan Kaufmann Publishers, San Mateo (1990)
Murata, N., Yoshizawa, S., Amari, S.: Network information criterion – determining the number of hidden units for an artificial neural network model. IEEE Transactions on Neural Networks 5(6), 865–872 (1994)
Natter, M., Haefke, C., Soni, T., Otruba, H.: Macroeconomic forecasting using neural networks. In: Neural Networks in the Capital Markets 1994 (1994)
Pi, H., Peterson, C.: Finding the embedding dimension and variable dependencies in time series. Neural Computation, 509–520 (1994)
Plaut, D., Nowlan, S., Hinton, G.: Experiments on learning by back propagation. Technical Report CMU-CS-86-126, Dept. of Computer Science, Carnegie-Mellon University, Pittsburgh, Pennsylvania (1986)
Rehfuss, S.: Macroeconomic forecasting with neural networks (1994) (unpublished simulations)
Rehkugler, H., Zimmermann, H.G. (eds.): Neuronale Netze in der Ökonomie. Verlag Vahlen (1994)
Swanson, N.R., White, H.: A model selection approach to real-time macroeconomic forecasting using linear models and artificial neural networks. Discussion paper, Department of Economics, Pennsylvania State University (1995)
Utans, J., Moody, J.: Selecting neural network architectures via the prediction risk: Application to corporate bond rating prediction. In: Proceedings of the First International Conference on Artificial Intelligence Applications on Wall Street. IEEE Computer Society Press, Los Alamitos (1991)
Utans, J., Moody, J., Rehfuss, S.: Selecting input variables via sensitivity analysis: Application to predicting the U.S. business cycle. In: Proceedings of Computational Intelligence in Financial Engineering. IEEE Press (1995)
Wahba, G.: Spline models for observational data. CBMS-NSF Regional Conference Series in Applied Mathematics (1990)
Winkler, R.L., Makridakis, S.: The combination of forecasts. Journal of Royal Statistical Society (146) (1983)
Wu, L., Moody, J.: A smoothing regularizer for feedforward and recurrent networks. Neural Computation 8(2) (1996)
Yang, H., Moody, J.: Input variable selection based on joing mutual information. Technical report, Department of Computer Science, Oregon Graduate Institute (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Moody, J. (2012). Forecasting the Economy with Neural Nets: A Survey of Challenges and Solutions. In: Montavon, G., Orr, G.B., Müller, KR. (eds) Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, vol 7700. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35289-8_22
Download citation
DOI: https://doi.org/10.1007/978-3-642-35289-8_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35288-1
Online ISBN: 978-3-642-35289-8
eBook Packages: Computer ScienceComputer Science (R0)