Advertisement

Descriptive Topological Spaces for Performing Visual Search

  • Jiajie Yu
  • Christopher J. HenryEmail author
Chapter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10810)

Abstract

This article presents an approach to performing the task of visual search in the context of descriptive topological spaces. The presented algorithm forms the basis of a descriptive visual search system (DVSS) that is based on the guided search model (GSM) that is motivated by human visual search. This model, in turn, consists of the bottom-up and top-down attention models and is implemented within the DVSS in three distinct stages. First, the bottom-up activation process is used to generate saliency maps and to identify salient objects. Second, perceptual objects, defined in the context of descriptive topological spaces, are identified and associated with feature vectors obtained from a VGG deep learning convolutional neural network. Lastly, the top-down activation process makes decisions on whether the object of interest is present in a given image through the use of descriptive patterns within the context of a descriptive topological space. The presented approach is tested with images from the ImageNet ILSVRC2012 and SIMPLIcity datasets. The contribution of this article is a descriptive pattern-based visual search algorithm.

Keywords

Human visual search Guided search model Bottom-up attention Top-down attention Salient objects Visual field Descriptive topological space Descriptive proximity Descriptive set intersection Convolutional neural network 

References

  1. 1.
    Yu, Y., Mann, G.K.I., Gosine, R.G.: A goal-directed visual perception system using object-based top-down attention. IEEE Trans. Auton. Ment. Dev. 4(1), 87–103 (2012)CrossRefGoogle Scholar
  2. 2.
    Duncan, J., Humphreys, G.W.: Visual search and stimulus similarity. Psychol. Rev. 96(3), 433 (1989)CrossRefGoogle Scholar
  3. 3.
    Wolfe, J.M.: Guided search 2.0 a revised model of visual search. Psychon. Bull. Rev. 1, 202–238 (1994)CrossRefGoogle Scholar
  4. 4.
    Peters, J.F., Naimpally, S.A.: Applications of near sets. Not. Am. Math. Soc. 59(4), 536–542 (2012)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Naimpally, S.A., Peters, J.F.: Topology with Applications Topological Spaces via Near and Far. World Scientific, Singapore (2013)CrossRefGoogle Scholar
  6. 6.
    Peters, J.F.: Topology of Digital Images. Visual Pattern Discovery in Proximity Spaces. Intelligent Systems Reference Library, vol. 63. Springer, Berlin (2014).  https://doi.org/10.1007/978-3-642-53845-2CrossRefzbMATHGoogle Scholar
  7. 7.
    Peters, J.: Computational Proximity: Excursions in the Topology of Digital Images. Intelligent Systems Reference Library. Springer, Berlin (2016).  https://doi.org/10.1007/978-3-319-30262-1CrossRefzbMATHGoogle Scholar
  8. 8.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2006)Google Scholar
  9. 9.
    Yu, J.: A descriptive topological framework for performing visual search, Masters thesis, University of Winnipeg (2017)Google Scholar
  10. 10.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  11. 11.
    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980)CrossRefGoogle Scholar
  12. 12.
    Dismone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995)CrossRefGoogle Scholar
  13. 13.
    Duncan, J., Humphreys, G., Ward, R.: Competitive brain activity in visual attention. Curr. Opin. Neurobiol. 7(2), 255–261 (1997)CrossRefGoogle Scholar
  14. 14.
    Fink, G.R., Dolan, R.J., Halligan, P.W., Marshall, J.C., Frith, C.D.: Space-base and object-based visual attention: shared and specific neural domains. Brain 120(11), 2013–2028 (1997)CrossRefGoogle Scholar
  15. 15.
    Peters, J.F.: Near sets. In: Henry, C.J. (ed.) Wikipedia, The Free Encyclopaedia (2015)Google Scholar
  16. 16.
    Sossinsky, A.B.: Tolerance space theory and some applications. Acta Applicandae Mathematicae: Int. Surv. J. Appl. Math. Math. Appl. 5(2), 137–167 (1986)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Poincaré, H.: Science and Hypothesis. The Mead Project, Brock University (1905). L. G. Ward’s translationGoogle Scholar
  18. 18.
    Benjamin Jr., L.T.: A Brief History of Modern Psychology. Blackwell Publishing, Malden (2007)Google Scholar
  19. 19.
    Hergenhahn, B.R.: An Introduction to the History of Psychology. Wadsworth Publishing, Belmont (2009)Google Scholar
  20. 20.
    Zeeman, E.C.: The topology of the brain and the visual perception. In: Fort, K.M. (ed.) Topoloy of 3-Manifolds and Selected Topics, pp. 240–256. Prentice Hall, New Jersey (1965)Google Scholar
  21. 21.
    Naimpally, S.A.: Near and far. A centennial tribute to Frigyes Riesz. Siberian Electron. Math. Rep. 6, A.1–A.10 (2009)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Naimpally, S.A., Warrack, B.D.: Proximity spaces. In: Cambridge Tract in Mathematics No. 59. Cambridge University Press, Cambridge (1970)Google Scholar
  23. 23.
    Pawlak, Z., Peters, J.F.: Jak Blisko (how near). Systemy Wspomagania Decyzji I, 57–109 (2002)Google Scholar
  24. 24.
    Mozzochi, C.J., Naimpally, S.A.: Uniformity and proximity. In: Allahabad Mathematical Society Lecture Note Series, vol. 2, p. 153 pp. The Allahabad Mathematical Society, Allahabad (2009)Google Scholar
  25. 25.
    Naimpally, S.A.: Proximity Approach to Problems in Topology and Analysis. Oldenburg Verlag, München (2009). ISBN 978-3-486-58917-7CrossRefGoogle Scholar
  26. 26.
    Hocking, J.G., Naimpally, S.A.: Nearness-a better approach to continuity and limits. In: Allahabad Mathematical Society Lecture Note Series, vol. 3, p. 153 pp. The Allahabad Mathematical Society (2009)Google Scholar
  27. 27.
    Peters, J.F.: Near sets. General theory about nearness of objects. Appl. Math. Sci. 1(53), 2609–2629 (2007)MathSciNetzbMATHGoogle Scholar
  28. 28.
    Peters, J.F.: Near sets. Special theory about nearness of objects. Fundamenta Informaticae 75(1–4), 407–433 (2007)MathSciNetzbMATHGoogle Scholar
  29. 29.
    Peters, J.F.: Tolerance near sets and image correspondence. Int. J. Bio-Inspired Comput. 1(4), 239–245 (2009)CrossRefGoogle Scholar
  30. 30.
    Peters, J.F.: Corrigenda and addenda: tolerance near sets and image correspondence. Int. J. Bio-Inspired Comput. 2(5), 310–318 (2010)CrossRefGoogle Scholar
  31. 31.
    İnan, E., Öztürk, M.A.: Near groups on nearness approximation spaces. Hacettepe J. Math. Stat. 41(4), 545–558 (2012)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Peters, J.F., İnan, E., Öztürk, M.A.: Spatial and descriptive isometries in proximity spaces. Gen. Math. Notes 21(2), 1–10 (2014)Google Scholar
  33. 33.
    Peters, J.F., Wasilewski, P.: Foundations of near sets. Inf. Sci. 179(18), 3091–3109 (2009)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Peters, J.F.: Classification of perceptual objects by means of features. Int. J. Inf. Technol. Intell. Comput. 3(2), 1–35 (2008)Google Scholar
  35. 35.
    Li, F., Karpathy, A.: CS231n: Convolutional Neural Networks for Visual Recognition, Course Lecture Notes, Standford University (2015)Google Scholar
  36. 36.
    Henry, C.J., Smith, G.: Proximity system: a description-based system for quantifying the nearness or apartness of visual rough sets. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets XVII. LNCS, vol. 8375, pp. 48–73. Springer, Heidelberg (2014).  https://doi.org/10.1007/978-3-642-54756-0_3CrossRefGoogle Scholar
  37. 37.
    Henry, C.J.: Near sets: theory and applications, Ph.D. thesis, University of Manitoba (2010)Google Scholar
  38. 38.
    Henry, C.J.: Metric free nearness measure using description-based neighbourhoods. Math. Comput. Sci. 7(1), 51–69 (2013)MathSciNetCrossRefGoogle Scholar
  39. 39.
    Peters, J.F.: Computational proximity. In: Computational Proximity. ISRL, vol. 102, pp. 1–62. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-30262-1_1Google Scholar
  40. 40.
    C̆ech, E.: Topological Spaces. Wiley, London (2014). fr seminar, Brno, 1936–1939; rev. ed. Z. Frolik, M. KatĕtovGoogle Scholar
  41. 41.
    Efremovic̆, V.A.: The geometry of proximity I (in Russian). Mat. Sb. (N.S.) 31(73)(1), 189–200 (1952)MathSciNetGoogle Scholar
  42. 42.
    Lodato, M.: On topologically induced generalized proximity relations, Ph.D. dissertation, Rutgers University (1962). supervisor: S. LeaderGoogle Scholar
  43. 43.
    Wallman, H.: Lattices and topological spaces. Ann. Math. 39(1), 112–126 (1938)MathSciNetCrossRefGoogle Scholar
  44. 44.
    Peters, J.F.: Local near sets. Pattern discovery in proximity spaces. Math. Comput. Sci. 7(1), 87–106 (2013)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Peters, J.F., Wasilewski, P.: Tolerance spaces: origins, theoretical aspects and applications. Inf. Sci. 195, 211–225 (2012)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Henry, C.J.: Perceptual indiscernibility, rough sets, descriptively near sets, and image analysis. In: Peters, J.F., Skowron, A. (eds.) Transactions on Rough Sets XV. LNCS, vol. 7255, pp. 41–121. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-31903-7_3CrossRefGoogle Scholar
  47. 47.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR, pp. 1–14 (2015). http://arxiv.org/abs/1409.1556
  48. 48.
    Karpathy, A.: CS231n: Convolutional Neural Networks for Visual Recognition. Stanford University, Stanford (2015)Google Scholar
  49. 49.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012)Google Scholar
  50. 50.
    Perone, C.: Deep learning - convolutional neural networks and feature extraction with python (2015). http://blog.christianperone.com/2015/08/convolutional-neural-networks-and-feature-extraction-with-python//
  51. 51.
    Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1349–1380 (2000)CrossRefGoogle Scholar
  52. 52.
    Yates-Baeza, R., Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press/Pearson Addison Wesley, New York (1999)Google Scholar
  53. 53.
    Deng, J.D.J., et al.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2–9 (2009)Google Scholar
  54. 54.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  55. 55.
    Wang, J.Z., Li, J., Wiederholdy, G.: SIMPLIcity: semantics-sensitive integrated matching for picture libraries. In: Laurini, R. (ed.) VISUAL 2000. LNCS, vol. 1929, pp. 360–371. Springer, Heidelberg (2000).  https://doi.org/10.1007/3-540-40053-2_32CrossRefGoogle Scholar
  56. 56.
    Jhanwar, N., Chaudhuri, S., Seetharaman, G., Zavidovique, B.: Content based image retrieval using motif cooccurrence matrix. Image Vis. Comput. 22(14), 1211–1220 (2004)CrossRefGoogle Scholar
  57. 57.
    Subrahmanyam, M., Jonathan Wu, Q.M., Maheshwari, R.P., Balasubramanian, R.: Modified color motif co-occurrence matrix for image indexing and retrieval. Comput. Electr. Eng. 39(3), 762–774 (2013)CrossRefGoogle Scholar
  58. 58.
    Vadivel, A., Sural, S., Majumdar, A.K.: An integrated color and intensity co-occurrence matrix. Pattern Recogn. Lett. 28(8), 974–983 (2007)CrossRefGoogle Scholar
  59. 59.
    Lin, C.-H., Chen, R.-T., Chan, Y.-K.: A smart content-based image retrieval system based on color and texture feature. Image Vis. Comput. 27(6), 658–665 (2009).  https://doi.org/10.1016/j.imavis.2008.07.004CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.University of WinnipegWinnipegCanada

Personalised recommendations