Skip to main content
SpringerLink
Log in
Book cover

Australasian Joint Conference on Artificial Intelligence

AI 2011: AI 2011: Advances in Artificial Intelligence pp 72–81Cite as

Penalized Least Squares for Smoothing Financial Time Series

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7106))

Abstract

Modeling of financial time series data by methods of artificial intelligence is difficult because of the extremely noisy nature of the data. A common and simple form of filter to reduce the noise originated in signal processing, the finite impulse response (FIR) filter. There are several of these noise reduction methods used throughout the financial instrument trading community. The major issue with these filters is the delay between the filtered data and the noisy data. This delay only increases as more noise reduction is desired. In the present marketplace, where investors are competing for quality and timely information, this delay can be a hindrance. This paper proposes a new FIR filter derived with the aim of maximizing the level of noise reduction and minimizing the delay. The model is modified from the old problem of time series graduation by penalized least squares. Comparison between five different methods has been done and experiment results have shown that our method is significantly superior to the alternatives in both delay and smoothness over short and middle range delay periods.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Boudraa, A.O., Cexus, J.C.: EMD-based signal filtering. IEEE Transactions on Instrumentation and Measurement 56(6), 2196–2202 (2007)

    Article  Google Scholar 

  2. Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association 90(432), 1200–1224 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  3. Donoho, D.L., Johnstone, I.M.: Minimax estimation via wavelet shrinkage. The Annals of Statistics 26(3), 879–921 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  4. Drakakis, K.: Empirical mode decomposition of financial data. International Mathematical Forum 3(25), 1191–1202 (2008)

    MathSciNet  MATH  Google Scholar 

  5. Ehlers, J.F.: Rocket science for traders: Digital signal processing applications. John Wiley & Sons, Inc., New York (2001)

    Google Scholar 

  6. Eilers, P.H.C.: A perfect smoother. Analytical Chemistry 75(14), 3631–3636 (2003)

    Article  Google Scholar 

  7. Ellis, C.A., Parbery, S.A.: Is smarter better? a comparison of adaptive, and simple moving average trading strategies. Research in International Business and Finance 19(3), 399–411 (2005)

    Article  Google Scholar 

  8. Farmer, D.J., Sidorowich, J.J.: Optimal shadowing and noise reduction. Physica D: Nonlinear Phenomena 47(3), 373–392 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gardner, J.E.S.: Exponential smoothing: The state of the art–part II. International Journal of Forecasting 22(4), 637–666 (2006)

    Article  Google Scholar 

  10. Godsill, S.J., Doucet, A., West, M.: Monte Carlo smoothing for nonlinear time series. Journal of the American Statistical Association 99(465), 156–168 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hale, D.: Recursive gaussian filters. Tech. rep., Center for Wave Phenomena (2006)

    Google Scholar 

  12. Hassani, H.: Singular spectrum analysis: Methodology and comparison. Journal of Data Sciences 5, 239–257 (2007), mPRA Paper

    Google Scholar 

  13. Hassani, H., Soofi, A.S., Zhigljavsky, A.A.: Predicting daily exchange rate with singular spectrum analysis. Nonlinear Analysis: Real World Applications 11(3), 2023–2034 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hodrick, R.J., Prescott, E.C.: Postwar u.s. business cycles: An empirical investigation. Journal of Money, Credit and Banking 29(1), 1–16 (1997)

    Article  Google Scholar 

  15. Hsu, L.-Y., Horng, S.-J., He, M., Fan, P., Kao, T.-W., Khan, M.K., Run, R.-S., Lai, J.-L., Chen, R.-J.: Mutual funds trading strategy based on particle swarm optimization. Expert Systems with Applications 38(6), 7582–7602 (2011)

    Article  Google Scholar 

  16. Hull, A.: Hull moving average HMA (2011), http://www.justdata.com.au/Journals/AlanHull/hull_ma.htm

  17. Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME Journal of Basic Engineering 82(Series D), 35–45 (1960)

    Article  Google Scholar 

  18. Kamen, E.W., Su, J.K.: Introduction to optimal estimation. Springer, London (1999)

    Book  MATH  Google Scholar 

  19. Karunasingha, D.S.K., Liong, S.Y.: Enhancement of chaotic time series prediction with real-time noise reduction. In: International Conference on Small Hydropower - Hydro Sri Lanka (2007)

    Google Scholar 

  20. Klaas, M., Briers, M., Freitas, N.d., Doucet, A., Maskell, S., Lang, D.: Fast particle smoothing: if I had a million particles. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 481–488 (2006)

    Google Scholar 

  21. Kostelich, E.J., Yorke, J.A.: Noise reduction in dynamical systems. Physical Review A 38(3), 1649 (1988)

    Article  MathSciNet  Google Scholar 

  22. Kostelich, E.J., Yorke, J.A.: Noise reduction: Finding the simplest dynamical system consistent with the data. Physica D: Nonlinear Phenomena 41(2), 183–196 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  23. Legoux, A.: ALMA Arnaud Legous Moving Average (2009), http://www.arnaudlegoux.com/wp-content/uploads/2011/03/ALMA-Arnaud-Legoux-Moving-Average.pdf (2011)

  24. Li, T., Li, Q., Zhu, S., Ogihara, M.: A survey on wavelet applications in data mining. SIGKDD Explor. Newsl. 4(2), 49–68 (2002), 772870

    Article  Google Scholar 

  25. Liu, Y., Liao, X.: Adaptive chaotic noise reduction method based on dual-lifting wavelet. Expert Systems with Applications 38(3), 1346–1355 (2011)

    Article  Google Scholar 

  26. Moore, J.B.: Discrete-time fixed-lag smoothing algorithms. Automatica 9(2), 163–173 (1973)

    Article  MATH  Google Scholar 

  27. Nikpour, M., Nadernejad, E., Ashtiani, H., Hassanpour, H.: Using pde’s for noise reduction in time series. International Journal of Computing and ICT Research 3(1), 42–48 (2009)

    Google Scholar 

  28. Russell, S.J., Norvig, P.: Artificial intelligence: A modern approach. Pretice Hall series in artificial intelligence. Prentice Hall, N.J (2010)

    MATH  Google Scholar 

  29. Sadasivan, P.K., Dutt, D.N.: SVD based technique for noise reduction in electroencephalographic signals. Signal Processing 55(2), 179–189 (1996)

    Article  MATH  Google Scholar 

  30. Shynk, J.J.: Frequency-domain and multirate adaptive filtering. IEEE Signal Processing Magazine 9(1), 14–37 (1992)

    Article  Google Scholar 

  31. Soofi, A.S., Cao, L.: Nonlinear forecasting of noisy financial data. In: Soofi, A.S., Cao, L. (eds.) Modeling and Forecasting Financial Data: Techniques of Nonlinear Dynamics, pp. 455–465. Kluwer Academic Publishers, Boston (2002)

    Chapter  Google Scholar 

  32. Vandewalle, N., Ausloos, M., Boveroux, P.: The moving averages demystified. Physica A: Statistical Mechanics and its Applications 269(1), 170–176 (1999)

    Article  Google Scholar 

  33. Whittaker, E.T.: On a new method of graduation. Proceedings of the Edinburgh Mathematical Society 41(-1), 63–75 (1923)

    Google Scholar 

  34. Zehtabian, A., Hassanpour, H.: A non-destructive approach for noise reduction in time domain. World Applied Sciences Journal 6(1), 53–63 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Mathematics, Charles Sturt University, Australia

    Adrian Letchford, Junbin Gao & Lihong Zheng

Authors
  1. Adrian Letchford
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junbin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lihong Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Engineering and Mathematical Sciences, La Trobe University, 3086, Melbourne, VIC, Australia

    Dianhui Wang

  2. School of Computer Science and Software Engineering, The University of Western Australia, 6009, Perth, WA, Australia

    Mark Reynolds

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Letchford, A., Gao, J., Zheng, L. (2011). Penalized Least Squares for Smoothing Financial Time Series. In: Wang, D., Reynolds, M. (eds) AI 2011: Advances in Artificial Intelligence. AI 2011. Lecture Notes in Computer Science(), vol 7106. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25832-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-25832-9_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-25831-2

  • Online ISBN: 978-3-642-25832-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation