Abstract
We introduce the concept of a sparse grid and show how this powerful approach to function space discretization may be employed to tackle high-dimensional machine learning problems of regression and classification. In particular, we address the issue of incremental computation of sparse grid regression coefficients so as to meet the requirements of realtime data mining. Conclusively, we present experimental results on real-world data sets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arge, E., Daelen, M., Tveito, A.: Approximation of scattered data using smooth grid functions. J. Comput. Appl. Math. 59, 191–205 (1995)
Bokanowski, O., Garcke, J., Griebel, M., Klompmaker, I.: An adaptive sparse grid semi-lagrangian scheme for first order Hamilton-Jacobi Bellman equations, submitted to J. Sci. Comput., also available as INS Preprint No. 1207 (2012)
Bungartz, H.-J: Dünne Gitter und deren Anwendung bei der adaptiven Lösung der dreidimensionalen Poisson-Gleichung (in German). Dissertation, TU München (1992)
Bungartz, H.-J., Griebel, M., Röschke, D., Zenger, C.: Pointwise convergence of the combination technique for Laplace’s equation. East–west, J. Numer. Math. 2, 21–45 (1994)
Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895–1924 (1998)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization networks and support vector machines. Adv. Comput. Math. 13, 1–50 (2000)
Faber, G.: Über stetige Funktionen (in German). Math. Annal. 66, 81–94 (1909)
Garcke J.: Regression with the optimized combination technique. In: ICML ’06: Proceedings of the 23th International Conference on Machine learning, pp. 321–328. ACM Press, New York (2006)
Fung, G., Mangasarian, O.L.: Proximity support vector machine classifiers. In: Provost, F., Srikant, R. (eds.) Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 77–86 (2001)
Garcke, J.: A dimension adaptive sparse grid combination technique for machine learning. In: Read, W., Larson, J.W., Roberts, A.J. (eds.) Proceedings of the 13th Biennial Computational Techniques and Application Conference, CTAC-2006, vol. 48 of ANZIAM J., pp. C725–C740 (2007)
Garcke, J.: A dimension adaptive combination technique using localized adaptation criteria. In: Bock, H.G., Hoang, X.P., Rannacher, R., Schlöder, J.P. (eds.) Modeling, Simulation and Optimization of Complex Processes, pp. 115–125. Springer, Dordrecht (2012)
Garcke, J., Griebel, M.: Data Mining with sparse grids using simplicial basis functions. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2001)
Garcke, J., Griebel, M., Thess, M.: Data mining with sparse grids. Computing 67(3), 225–253 (2001)
Gerstner, T., Griebel, M.: Numerical integration using sparse grids. Numer. Algorithms 18, 209–232 (1998)
Girosi, F.: An equivalence between sparse approximation and support vector machines. Neural Comput. 10, 1455–1480 (1998)
Girosi, F., Jones, M., Poggio, T.: Priors, stabilizers and basis functions: from regularization to radial, tensor and additive splines. AI Memo No. 1430. Artificial Intelligence Laboratory, MIT (1993)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–265 (1995)
Griebel, M.: The combination technique for the sparse grid solution of PDEs on multiprocessor machines. Parallel Process. Lett. 2(1), 61–70 (1992)
Griebel, M., Schneider, M., Zenger, C.: A combination technique for the solution of sparse grid problems. In: de Groen, P., Beauwens, R. (eds.) Iterative Methods in Linear Algebra, pp. 263–281. Elsevier, Amsterdam (1992)
Hegland, M.: Adaptive sparse grids. In: Burrage, K., Sidje, R.B. (eds.) Proceedings of 10th Computational Techniques and Application Conference CTAC-2001, vol. 44, pp. C335–C353. (2003)
Hoschek, J., Lasser, D.: Grundlagen der geometrischen Datenverarbeitung. Teubner, Stuttgart (1992). Chapter 9 (in German)
Kuhn, H.W.: Some combinatorial lemmas in topology. IBM J. Res. Develop. 4, 518–524 (1960)
Mangasarian, O.L., Musicant, D.R.: Lagrangian Support Vector Machines, Journal of Machine Learning Research 1, 161-177 (2001)
Pflüger D.: Spatial adaptive sparse grids for high-dimensional problems. Dissertation, TU München (2010)
Ripley, B.D.: Neural networks and related methods for classification. J. R. Stat. Soc. B 56(3), 409–456 (1994)
Salzberg, S.L.: On comparing classifiers: pitfalls to avoid and a recommended approach. Data Min. Knowl Discov. 1, 317–327 (1997)
Singh, S.: 2d spiral pattern recognition with possible measures. Pattern Recognit. Lett. 19(2), 141–147 (1998)
Smolyak, S.A.: Quadrature and interpolation formulas for tensor products of certain classes of functions. Dokl. Akad. Nauk SSSR. 148, 1042–1043 (1963). Russian, Engl. Transl.: Soviet Math. Dokl. 4, 240–243 (1963)
Tikhonov, A.N., Arsenin, V.A.: Solutions of Ill-Posed Problems. W.H. Winston, Washington, DC (1977)
Wahba, G.: Spline models for observational data, vol. 59. SIAM, Philadelphia (1990)
Wieland, A.: Spiral data set. CMU Artificial Intelligence repository, 1988
Yserentant, H.: On the multi-level splitting of finite element spaces. Numer. Math. 49, 379–412 (1986)
Zenger, C.: Sparse grids. In: Hackbusch, W. (ed.) Parallel Algorithms for Partial Differential Equations, Proceedings of the Sixth GAMM Seminar, Kiel, 1990, Vol. 31 of Notes on Num. Fluid Mech., pp. 241–251. Vieweg-Verlag (1991)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Paprotny, A., Thess, M. (2013). Breaking Dimensions: Adaptive Scoring with Sparse Grids. In: Realtime Data Mining. Applied and Numerical Harmonic Analysis. Birkhäuser, Cham. https://doi.org/10.1007/978-3-319-01321-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-01321-3_7
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-319-01320-6
Online ISBN: 978-3-319-01321-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)