Advertisement

Computational Statistics

, Volume 15, Issue 4, pp 463–484 | Cite as

Parallel MARS Algorithm Based on B-splines

  • Sergey Bakin
  • Markus Hegland
  • Michael R. Osborne
Article
  • 151 Downloads

Summary

We investigate one of the possible ways for improving Friedman’s Multivariate Adaptive Regression Splines (MARS) algorithm designed for flexible modelling of high-dimensional data. In our version of MARS called BMARS we use B-splines instead of truncated power basis functions. The fact that B-splines have compact support allows us to introduce the notion of a “scale” of a basis function. The algorithm starts building up models by using large-scale basis functions and switches over to a smaller scale after the fitting ability of the large scale splines has been exhausted. The process is repeated until the prespecified number of basis functions has been produced. In addition, we discuss a parallelisation of BMARS as well as an application of the algorithm to processing of a large commercial data set. The results demonstrate the computational efficiency of our algorithm and its ability to generate models competitive with those of the original MARS.

Keywords

MARS B-splines Data Mining Parallel Algorithms 

Notes

8 Acknowledgements

We are most grateful to Prof J.H. Friedman for suggesting the idea of the experiment involving the synthetic data set and to Dr B. Turlach for very fruitful discussions. Our thanks are also due to the anonymous referees for their constructive comments which greatly helped to improve the quality of this paper. The research of S. Bakin was supported by the Australian Government (Overseas Postgraduate Research Scholarship), by the Australian National University (ANU PhD Scholarship) and, also, by the Advanced Computational Systems CRC (ACSys), Australia.

References

  1. Breiman, L., Friedman, J.H., Olshen, R.A. & Stone, C.J. (1984), Classification and Regression Trees, Wadsworth, Belmont, California.zbMATHGoogle Scholar
  2. Chen, Z. (1990), Beyond additive models: interactions by smoothing spline methods, Technical Report SMS-009-90, The Australian National University.Google Scholar
  3. Cox, M.G. (1981), Practical spline approximation, Topics in Numerical Analysis, Lancaster, 79–112.Google Scholar
  4. Fayyad, U., Piatetsky-Shapiro, G. & Smyth, P. (1996), From Data Mining to Knowledge Discovery: An Overview, in ‘Advances in Knowledge Discovery and Data Mining’, pp. 1–36.Google Scholar
  5. Friedman, J.H. (1991), ‘Multivariate Adaptive Regression Splines’, The Annals of Statistics, 19(1), 1–141.MathSciNetCrossRefGoogle Scholar
  6. Friedman, J.H. (1981), Estimating functions of mixed ordinal and categorical variables, Technical Report 108, Stanford University.Google Scholar
  7. Friedman, J.H. & Stuetzle, W. (1981), ‘Projection Pursuit Regression’, Journal of the American Statistical Association, 76, 817–823.MathSciNetCrossRefGoogle Scholar
  8. Geist, A., Beguelin, A., Dongarra, J., Jiang, W., Manchek, R. & Sunderam, V. (1994), PVM: Parallel Virtual Machine, MIT Press.Google Scholar
  9. George, E.I. & McCulloch, R.E. (1993), ‘Variable selection via Gibbs sampling’, Journal of the American Statistical Association, 88, 881–889.CrossRefGoogle Scholar
  10. Luenberger, D.G. (1984), Linear and Nonlinear Programming, Reading, Massachusetts.zbMATHGoogle Scholar
  11. McCullagh, P. & Neider, J.A. (1983), Generalized Linear Models, Chapman and Hall.Google Scholar
  12. Miller, A.J. (1990), Subset Selection in Regression, Chapman and Hall.Google Scholar
  13. Stone, G. (1997), Analysis of Motor Vehicle Claims Data using Statistical Data Mining, CMIS Confidential Report CMIS-97/73, CSIRO, Australia.Google Scholar
  14. Wahba, G. (1990), Spline Models for Observational Data, SIAM, Philadelphia.CrossRefGoogle Scholar

Copyright information

© Physica-Verlag 2000

Authors and Affiliations

  • Sergey Bakin
    • 1
  • Markus Hegland
    • 2
  • Michael R. Osborne
    • 1
  1. 1.School of Mathematical SciencesThe Australian National UniversityCanberraAustralia
  2. 2.Computer Sciences LaboratoryThe Australian National UniversityCanberraAustralia

Personalised recommendations