Breaking Dimensions: Adaptive Scoring with Sparse Grids

  • Alexander Paprotny
  • Michael Thess
Part of the Applied and Numerical Harmonic Analysis book series (ANHA)


We introduce the concept of a sparse grid and show how this powerful approach to function space discretization may be employed to tackle high-dimensional machine learning problems of regression and classification. In particular, we address the issue of incremental computation of sparse grid regression coefficients so as to meet the requirements of realtime data mining. Conclusively, we present experimental results on real-world data sets.


Adaptive Multivariate Regression Spline Sparse Grid Combination Technique Refinement Level Hierarchical Basis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. [ADT95]
    Arge, E., Daelen, M., Tveito, A.: Approximation of scattered data using smooth grid functions. J. Comput. Appl. Math. 59, 191–205 (1995)CrossRefzbMATHMathSciNetGoogle Scholar
  2. [BGGK12]
    Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-lagrangian scheme for first order Hamilton-Jacobi Bellman equations, submitted to J. Sci. Comput., also available as INS Preprint No. 1207 (2012)Google Scholar
  3. [Bun92]
    Bungartz, H.-J: Dünne Gitter und deren Anwendung bei der adaptiven Lösung der dreidimensionalen Poisson-Gleichung (in German). Dissertation, TU München (1992)Google Scholar
  4. [BGRZ94]
    Bungartz, H.-J., Griebel, M., Röschke, D., Zenger, C.: Pointwise convergence of the combination technique for Laplace’s equation. East–west, J. Numer. Math. 2, 21–45 (1994)zbMATHMathSciNetGoogle Scholar
  5. [Diet98]
    Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1924 (1998)CrossRefGoogle Scholar
  6. [EPP00]
    Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and support vector machines. Adv. Comput. Math. 13, 1–50 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  7. [Fab9]
    Faber, G.: Über stetige Funktionen (in German). Math. Annal. 66, 81–94 (1909)CrossRefzbMATHMathSciNetGoogle Scholar
  8. [Gar06]
    Garcke J.: Regression with the optimized combination technique. In: ICML ’06: Proceedings of the 23th International Conference on Machine learning, pp. 321–328. ACM Press, New York (2006)Google Scholar
  9. [FM01]
    Fung, G., Mangasarian, O.L.: Proximity support vector machine classifiers. In: Provost, F., Srikant, R. (eds.) Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 77–86 (2001)Google Scholar
  10. [Gar11]
    Garcke, J.: A dimension adaptive sparse grid combination technique for machine learning. In: Read, W., Larson, J.W., Roberts, A.J. (eds.) Proceedings of the 13th Biennial Computational Techniques and Application Conference, CTAC-2006, vol. 48 of ANZIAM J., pp. C725–C740 (2007)Google Scholar
  11. [Gar12b]
    Garcke, J.: A dimension adaptive combination technique using localized adaptation criteria. In: Bock, H.G., Hoang, X.P., Rannacher, R., Schlöder, J.P. (eds.) Modeling, Simulation and Optimization of Complex Processes, pp. 115–125. Springer, Dordrecht (2012)CrossRefGoogle Scholar
  12. [GG01a]
    Garcke, J., Griebel, M.: Data Mining with sparse grids using simplicial basis functions. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2001)Google Scholar
  13. [GGT01]
    Garcke, J., Griebel, M., Thess, M.: Data mining with sparse grids. Computing 67(3), 225–253 (2001)CrossRefzbMATHMathSciNetGoogle Scholar
  14. [GG98]
    Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18, 209–232 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  15. [Gir98]
    Girosi, F.: An equivalence between sparse approximation and support vector machines. Neural Comput. 10, 1455–1480 (1998)CrossRefGoogle Scholar
  16. [GJP93]
    Girosi, F., Jones, M., Poggio, T.: Priors, stabilizers and basis functions: from regularization to radial, tensor and additive splines. AI Memo No. 1430. Artificial Intelligence Laboratory, MIT (1993)Google Scholar
  17. [GJP95]
    Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–265 (1995)CrossRefGoogle Scholar
  18. [Gri92]
    Griebel, M.: The combination technique for the sparse grid solution of PDEs on multiprocessor machines. Parallel Process. Lett. 2(1), 61–70 (1992)CrossRefMathSciNetGoogle Scholar
  19. [GSZ92]
    Griebel, M., Schneider, M., Zenger, C.: A combination technique for the solution of sparse grid problems. In: de Groen, P., Beauwens, R. (eds.) Iterative Methods in Linear Algebra, pp. 263–281. Elsevier, Amsterdam (1992)Google Scholar
  20. [Heg03]
    Hegland, M.: Adaptive sparse grids. In: Burrage, K., Sidje, R.B. (eds.) Proceedings of 10th Computational Techniques and Application Conference CTAC-2001, vol. 44, pp. C335–C353. (2003)Google Scholar
  21. [HL92]
    Hoschek, J., Lasser, D.: Grundlagen der geometrischen Datenverarbeitung. Teubner, Stuttgart (1992). Chapter 9 (in German)CrossRefzbMATHGoogle Scholar
  22. [Kuh60]
    Kuhn, H.W.: Some combinatorial lemmas in topology. IBM J. Res. Develop. 4, 518–524 (1960)CrossRefzbMATHGoogle Scholar
  23. [MM01]
    Mangasarian, O.L., Musicant, D.R.: Lagrangian Support Vector Machines, Journal of Machine Learning Research 1, 161-177 (2001)Google Scholar
  24. [Pfl10]
    Pflüger D.: Spatial adaptive sparse grids for high-dimensional problems. Dissertation, TU München (2010)Google Scholar
  25. [Rip94]
    Ripley, B.D.: Neural networks and related methods for classification. J. R. Stat. Soc. B 56(3), 409–456 (1994)zbMATHMathSciNetGoogle Scholar
  26. [Sal97]
    Salzberg, S.L.: On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min. Knowl Discov. 1, 317–327 (1997)CrossRefGoogle Scholar
  27. [Sin98]
    Singh, S.: 2d spiral pattern recognition with possible measures. Pattern Recognit. Lett. 19(2), 141–147 (1998)CrossRefzbMATHGoogle Scholar
  28. [Smo63]
    Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR. 148, 1042–1043 (1963). Russian, Engl. Transl.: Soviet Math. Dokl. 4, 240–243 (1963)Google Scholar
  29. [TA77]
    Tikhonov, A.N., Arsenin, V.A.: Solutions of Ill-Posed Problems. W.H. Winston, Washington, DC (1977)zbMATHGoogle Scholar
  30. [Wah90]
    Wahba, G.: Spline models for observational data, vol. 59. SIAM, Philadelphia (1990)Google Scholar
  31. [Wie88]
    Wieland, A.: Spiral data set. CMU Artificial Intelligence repository, 1988Google Scholar
  32. [Ys86]
    Yserentant, H.: On the multi-level splitting of finite element spaces. Numer. Math. 49, 379–412 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  33. [Zen91]
    Zenger, C.: Sparse grids. In: Hackbusch, W. (ed.) Parallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GAMM Seminar, Kiel, 1990, Vol. 31 of Notes on Num. Fluid Mech., pp. 241–251. Vieweg-Verlag (1991)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Alexander Paprotny
    • 1
  • Michael Thess
    • 2
  1. 1.Research and Developmentprudsys AGBerlinGermany
  2. 2.Research and Developmentprudsys AGChemnitzGermany

Personalised recommendations