Abstract
In this paper, we point out that hubness—some samples in a high-dimensional dataset emerge as hubs that are similar to many other samples—influences the performance of kernel regression. Because the dimension of feature spaces induced by kernels is usually very high, hubness occurs, giving rise to the problem of multicollinearity, which is known as a cause of instability of regression results. We propose hubness-reduced kernels for kernel regression as an extension of a previous approach for kNN classification that reduces spatial centrality to eliminate hubness.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Chatterjee, S., Hadi, A.S., Price, B.: Regression Analysis By Example. Wiley Series In Probability And Statistics. Wiley, New York (2000)
Gretton, A., Fukumizu, K., Teo, C., Song, L., Schölkopf, B., Smola, A.: A kernel statistical test of independence. Advances in Neural Information Processing Systems 20, 585–592 (2008)
Hara, K., Suzuki, I., Shimbo, M., Kobayashi, K., Fukumizu, K., Radovanović, M.: Localized centering: reducing hubness in large-sample data. In: AAAI (2015)
Montgomery, D.C., Peck, E.: Introduction to linear regression analysis. Wiley-Interscience Publication, John Wiley & sons, New York (1992)
Radovanović, M., Nanopoulos, A., Ivanović, M.: Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research 11, 2487–2531 (2010)
Schnitzer, D., Flexer, A., Schedl, M., Widmer, G.: Local and global scaling reduce hubs in space. Journal of Machine Learning Research 13(1), 2871–2902 (2012)
Suzuki, I., Hara, K., Shimbo, M., Matsumoto, Y., Saerens, M.: Investigating the effectiveness of laplacian-based kernels in hub reduction. In: AAAI (2012)
Suzuki, I., Hara, K., Shimbo, M., Saerens, M., Fukumizu, K.: Centering similarity measures to reduce hubs. In: EMNLP, pp. 613–623 (2013)
Tomasev, N., Radovanovic, M., Mladenic, D., Ivanovic, M.: The role of hubness in clustering high-dimensional data. IEEE Transactions on Knowledge and Data Engineering 26(3), 739–751 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Hara, K., Suzuki, I., Kobayashi, K., Fukumizu, K., Radovanović, M. (2015). Reducing Hubness for Kernel Regression. In: Amato, G., Connor, R., Falchi, F., Gennaro, C. (eds) Similarity Search and Applications. SISAP 2015. Lecture Notes in Computer Science(), vol 9371. Springer, Cham. https://doi.org/10.1007/978-3-319-25087-8_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-25087-8_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25086-1
Online ISBN: 978-3-319-25087-8
eBook Packages: Computer ScienceComputer Science (R0)