Abstract
The paper offers an algorithm (SNN-tree) that extends the binary tree search algorithm so that it can deal with distorted input vectors. Perceptrons are the tree nodes. The algorithm features an iterative solution search and stopping criterion. Unlike the SNN-tree algorithm, popular methods (LSH, k-d tree, BBF-tree, spill-tree) stop working as the dimensionality of the space grows (N > 1000). In this paper we managed to obtain an estimate of the upper bound on the error probability for SNN-tree algorithm. The proposed algorithm works much faster than exhaustive search (26 times faster at N = 10000).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Here, \( {\mathbf{X}} \) is a row-vector.
References
Kryzhanovsky, V., Malsagov, M., Tomas, J.A.C.: Hierarchical classifier: based on neural networks searching tree with iterative traversal and stop criterion. Opt. Mem. Neural Netw. (Inf. Opt.) 22(4), 217–223 (2013)
Friedman, J.H., Bentley, J.L., Finkel, R.A.: An algorithm for finding best matches in logarithmic expected time. ACM Trans. Math. Softw. 3, 209–226 (1977)
Liu, T., Moore, A.W., Gray, A, Yang, K.: An investigation of practical approximate nearest neighbor algorithms. In: Proceeding of Conference, Neural Information Processing Systems (2004)
Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of 30th STOC, pp. 604–613 (1998)
Beis, J.S., Lowe, D.G.: Shape indexing using approximate nearest-neighbour search in high-dimensional spaces. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1000–1006 (1997)
Kryzhanovsky, B., Kryzhanovskiy, V., Litinskii, L.: Machine learning in vector models of neural networks. In: Koronacki, J., Ras, Z.W., Wierzchon, S.T., et al. (eds.) Advances in Machine Learning II. Studies in Computational Intelligence, vol. SCI 263, pp. 427–443. Springer, Berlin (2010). (Dedicated to the memory of Professor Ryszard S. Michalski)
Kryzhanovsky, V., Malsagov, M., Zelavskaya, I., Tomas, J.A.C.: High-dimensional binary pattern classification by scalar neural network tree. In: Proceedings of International Conference on Artificial Neural Networks, pp. 169–177 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendix A
It is necessary to calculate the following probability:
Let scalar products \( {\mathbf{XX}}_{m}^{T} \) and \( {\mathbf{XX}}_{\mu }^{T} \) be independent random quantities, \( m \ne \mu \).
Now, it is necessary to calculate the probability that the product of each pattern by input vector is smaller than the threshold.
Scalar product \( {\mathbf{XX}}_{m}^{T} \) is a discrete quantity, which values lie in \( [ - N;N] \). Let k be the number of components with the opposite sign in vectors \( {\mathbf{X}} \) and \( {\mathbf{X}}_{m} \). Then its probability function is:
Random variable \( {\mathbf{XX}}_{m}^{T} \) is symmetrically distributed with zero mean, so
From A2 and A4 we can conclude that
Appendix B
Scalar product
consists of a large number of random quantities. Therefore, at big dimensions (N > 100) its distribution can be approximated by Gaussian law with the following probability moments:
Therefore, probability (A1) can be described by integral expression:
Using the following approximation
obtain the final estimation of probability (A1.1):
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kryzhanovsky, V., Malsagov, M. (2015). Distorted High-Dimensional Binary Patterns Search by Scalar Neural Network Tree. In: Khachay, M., Konstantinova, N., Panchenko, A., Ignatov, D., Labunets, V. (eds) Analysis of Images, Social Networks and Texts. AIST 2015. Communications in Computer and Information Science, vol 542. Springer, Cham. https://doi.org/10.1007/978-3-319-26123-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-26123-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26122-5
Online ISBN: 978-3-319-26123-2
eBook Packages: Computer ScienceComputer Science (R0)