Optical Memory and Neural Networks

, Volume 26, Issue 2, pp 129–136 | Cite as

Deep neural networks and maximum likelihood search for approximate nearest neighbor in video-based image recognition

  • A. V. Savchenko


We analyzed the way to increase computational efficiency of video-based image recognition methods with matching of high dimensional feature vectors extracted by deep convolutional neural networks. We proposed an algorithm for approximate nearest neighbor search. At the first step, for a given video frame the algorithm verifies a reference image obtained when recognizing the previous frame. After that the frame is compared with a few number of reference images. Each next examined reference image is chosen so that to maximize conditional probability density of distances to the reference instances tested at previous steps. To decrease the required memory space we beforehand calculate only distances from all the images to small number of instances (pivots). When experimenting with either face photos from Labeled Faces in the Wild and PubFig83 datasets or with video data from YouTube Faces we showed that our algorithm allows accelerating the recognition procedure by 1.4–4 times comparing with known approximate nearest neighbor methods.


video recognition image processing deep learning convolutional neural networks approximate nearest neighbor method classifier fusion face recognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bautista, C.M., Dy, C.A., Maýalac, M.I., Orbe, R.A., and Cordel, M., Convolutional neural network for vehicle detection in low resolution traffic videos, Proc. of the Region 10 Symposium (TENSYMP), IEEE, 2016, pp. 277–281.Google Scholar
  2. 2.
    Savchenko, A.V., Adaptive video image recognition system using a committee machine, Opt. Mem. Neural Networks (Inform. Opt.), 2012, vol. 21, no. 4, pp. 219–226.MathSciNetCrossRefGoogle Scholar
  3. 3.
    Wang, H., Wang, Y., and Cao, Y., Video-based face recognition: a survey, World Acad. Sci. Eng. Technol., 2009, vol. 60, pp. 293–302.Google Scholar
  4. 4.
    Huang, Z., Wang, R., Shan, S., and Chen, X., Projection metric learning on Grassmann manifold with application to video based face recognition. Proc. of Int. Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2015, pp. 140–149.Google Scholar
  5. 5.
    LeCun, Y., Bengio, Y., and Hinton, G., Deep learning, Nature, 2015, vol. 521, no. 7553, pp. 436–444.CrossRefGoogle Scholar
  6. 6.
    Parkhi, O.M., Vedaldi, A., and Zisserman, A., Deep face recognition, Proc. of the British Machine Vision, 2015, pp. 6–17.Google Scholar
  7. 7.
    Savchenko, A.V., Search Techniques in Intelligent Classification Systems, Switzerland: Springer International Publishing, 2016, p. 83.Google Scholar
  8. 8.
    Savchenko, A.V., Maximum-likelihood approximate nearest neighbor method in real-time Image recognition, Pattern Recognition, 2017, vol. 61, pp. 459–469.CrossRefGoogle Scholar
  9. 9.
    Viola, P. and Jones, M., Rapid object detection using a boosted cascade of simple features, Proc. of Int. Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2001, pp. 511–518.Google Scholar
  10. 10.
    Tresp, V., Committee machines, Handbook for Neural Network Signal Processing, CRC Press, 2001, pp. 135–151.Google Scholar
  11. 11.
    Lumini, A., Nanni, L., and Brahnam, S., Ensemble of texture descriptors and classifiers for face recognition, Appl. Comput. Inform., 2016.Google Scholar
  12. 12.
    Prince, S.J., Computer Vision: Models, Learning, and Reference, Cambridge University Press, 2012, p. 598.Google Scholar
  13. 13.
    Malkov, Y., Ponomarenko, A., Logvinov, A., and Krylov, V., Approximate nearest neighbor algorithm based on navigable small world graphs, Inform. Systems, 2014, vol. 45, pp. 61–68.CrossRefGoogle Scholar
  14. 14.
    Savchenko, A.V., Face recognition in real-time applications: Comparison of directed enumeration method and K-d trees, Proc. of Int. Conf. on Business Informatics Research (BIR), LNBIP, 2012, vol. 128, pp. 187–199.Google Scholar
  15. 15.
    Silpa-Anan, C. and Hartley, R., Optimised KD-trees for fast image descriptor matching, Proc. of Int. Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2008, pp. 1–8.Google Scholar
  16. 16.
    Savchenko, A.V., The maximal likelihood enumeration method for the problem of classifying piecewise regular objects, Autom. Remote Control, 2016, vol. 77, no. 3, pp. 443–450.CrossRefMATHGoogle Scholar
  17. 17.
    Burghouts, G., Smeulders, A., and Geusebroek, J.-M., The distribution family of similarity distances, Proc. of Int. Conf. on Advances in Neural Information Processing Systems (NIPS), 2008, pp. 201–208.Google Scholar
  18. 18.
    P’kalska, E. and Duin, R.P., Classifiers for dissimilarity-based pattern recognition, Proc. of the Int. Conf. on Pattern Recognition (CVPR), IEEE, 2000, pp. 12–16.Google Scholar
  19. 19.
    Gonzalez, E.C., Figueroa, K., and Navarro, G., Effective proximity retrieval by ordering permutations, IEEE Trans. Pattern Analysis Machine Intelligence, 2008, vol. 30, no. 9, pp. 1647–1658.CrossRefGoogle Scholar
  20. 20.
    Best-Rowden, L., Han, H., Otto, C., Klare, B.F., and Jain, A.K., Unconstrained face recognition: identifying a person of interest from a media collection, IEEE Trans. Inform. Forensics Security, 2014, vol. 9, pp. 2144–2157.CrossRefGoogle Scholar
  21. 21.
    Chellappa, R., Du, M., Turaga, P., and Zhou, S.K., Face tracking and recognition in video, in Handbook of Face Recognition, 2011, pp. 323–351.CrossRefGoogle Scholar
  22. 22.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., and Darrell, T., Caffe: Convolutional architecture for fast feature embedding, Proc. of the 22nd Int. Conf. on Multimedia. ACM, 2014, pp. 675–678.Google Scholar
  23. 23.
    Savchenko, A.V., Pattern recognition and increasing of the computational efficiency of a parallel realization of the probabilistic neural network with homogeneity testing, Opt. Mem. Neural Networks (Inform. Opt.), 2013, vol. 22, no. 3, pp. 184–192.CrossRefGoogle Scholar
  24. 24.
    Savchenko, A.V., Real-time image recognition with the parallel directed Enumeration method, Proc. of Int. Conf. on Vision Systems (ICVS), LNCS, 2013, vol. 7963, pp. 123–132.Google Scholar
  25. 25.
    Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H., and Hua, G., Labeled faces in the wild: A survey, in Advances in Face Detection and Facial Image Analysis, 2016, pp. 189–248.Google Scholar
  26. 26.
    Pinto, N., Stone, Z., Zickler, T., and Cox, D., Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on Facebook, Proc. of Int. Conf. on Computer Vision and Pattern Recognition (CVPR 2011 Workshops), IEEE, 2011, pp. 35–42.Google Scholar
  27. 27.
    Wolf, L., Hassner, T., and Maoz, I., Face recognition in unconstrained videos with matched background similarity, Proc. of Int. Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE, 2011, pp. 529–534.Google Scholar

Copyright information

© Allerton Press, Inc. 2017

Authors and Affiliations

  1. 1.National Research University Higher School of Economics, Laboratory of Algorithms and Technologies for Network AnalysisNizhny NovgorodRussia

Personalised recommendations