Abstract
As re-occurring compositions of visual data, visual patterns exist in complex spatial structures and diverse feature views of image and video data. Discovering visual patterns is of great interest to visual data analysis and recognition. Many methods have been proposed to address the problem of visual pattern discovery during the dozen years. In this chapter, we start with an overview of the visual pattern discovery problem and then discuss the major progress of spatial and feature co-occurrence pattern discovery.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Akata, Z., Thurau, C., Bauckhage, C., et al.: Non-negative matrix factorization in multimodality data for segmentation and label prediction. In: Proceedings of Computer Vision Winter Workshop (2011)
Bagon, S., Brostovski, O., Galun, M., Irani, M.: Detecting and sketching the common. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 33–40 (2010)
Blaschko, M., Lampert, C.: Correlational spectral clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2008)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Cai, X., Nie, F., Huang, H., Kamangar, F.: Heterogeneous image feature integration via multi-modal spectral clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1977–1984 (2011)
Cai, Z., Wang, L., Peng, X., Qiao, Y.: Multi-view super vector for action recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2014)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: binary robust independent elementary features. In: Proceedings of European Conference on Computer Vision, pp. 778–792 (2010)
Chaudhuri, K., Kakade, S.M., Livescu, K., Sridharan, K.: Multi-view clustering via canonical correlation analysis. In: Proceedings of International Conference on Machine Learning, pp. 129–136 (2009)
Cho, M., Shin, Y.M., Lee, K.M.: Unsupervised detection and segmentation of identical objects. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1617–1624 (2010)
Cong, Y., Yuan, J., Luo, J.: Towards scalable summarization of consumer videos via sparse dictionary selection. IEEE Trans. Multimed. 14(1), 66–75 (2012)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 886–893 (2005)
Diba, A., Pazandeh, A.M., Pirsiavash, H., Gool, L.V.: Deepcamp: deep convolutional action and attribute mid-level patterns. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3557–3565 (2016)
Elhamifar, E., Sapiro, G., Vidal, R.: See all by looking at a few: sparse modeling for finding representative objects. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1600–1607 (2012)
Eynard, D., Kovnatsky, A., Bronstein, M.M., Glashoff, K., Bronstein, A.M.: Multimodal manifold analysis by simultaneous diagonalization of laplacians. IEEE Trans. Pattern Anal. Mach. Intell. 37(12), 2505–2517 (2015)
Faktor, A., Irani, M.: “Clustering by composition”-unsupervised discovery of image categories. In: Proceedings of European Conference on Computer Vision, pp. 474–487 (2012)
Fang, Z., Cao, Z., Xiao, Y., Zhu, L., Yuan, J.: Adobe boxes: locating object proposals using object adobes. IEEE Trans. Image Process. 25(9), 4116–4128 (2016)
Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: CVPR Workshop on Generative-Model Based Vision, pp. 178–178 (2004)
Felzenszwalb, P., Girshick, R., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)
Fernando, B., Fromont, E., Tuytelaars, T.: Mining mid-level features for image classification. Int. J. Comput. Vis. 108(3), 186–203 (2014)
Fidler, S., Leonardis, A.: Towards scalable representations of object categories: learning a hierarchy of parts. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Gao, J., Hu, Y., Liu, J., Yang, R.: Unsupervised learning of high-order structural semantics from images. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2122–2129 (2009)
Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2268 (2011)
Grauman, K., Leibe, B.: Visual Object Recognition (Synthesis Lectures on Artificial Intelligence and Machine Learning). Morgan & Claypool Publishers, San Rafael, CA (2011)
Guo, X., Liu, D., Jou, B., Zhu, M., Cai, A., Chang, S.F.: Robust object co-detection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2013)
Guo, Y.: Convex subspace representation learning from multi-view data. In: Proceedings of AAAI Conference on Artificial Intelligence (2013)
Han, J., Cheng, H., Xin, D., Yan, X.: Frequent pattern mining: current status and future directions. Data Min. Knowl. Discov. 15(1), 55–86 (2007)
Heinly, J., Dunn, E., Frahm, J.M.: Comparative evaluation of binary features. In: Proceedings of European Conference on Computer Vision, pp. 759–773 (2012)
Hong, P., Huang, T.: Spatial pattern discovery by learning a probabilistic parametric model from multiple attributed relational graphs. Discret. Appl. Math. 139(1), 113–135 (2004)
Hsu, W., Dai, J., Lee, M.: Mining viewpoint patterns in image databases. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 553–558 (2003)
Huang, H.C., Chuang, Y.Y., Chen, C.S.: Affinity aggregation for spectral clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 773–780 (2012)
Jégou, H., Douze, M., Schmid, C., Pérez, P.: Aggregating local descriptors into a compact image representation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3304–3311 (2010)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. arXiv:1408.5093 (2014)
Jiang, Y., Liu, J., Li, Z., Li, P., Lu, H.: Co-regularized plsa for multi-view clustering. In: Proceedings of Asian Conference on Computer Vision, pp. 202–213 (2012)
Kim, S., Jin, X., Han, J.: Disiclass: discriminative frequent pattern-based image classification. In: KDD Workshop on Multimedia Data Mining, pp. 7:1–7:10 (2010)
Kobayashi, T.: Low-rank bilinear classification: efficient convex optimization and extensions. Int. J. Comput. Vis. 110(3), 308–327 (2014)
Kong, Y., Fu, Y.: Bilinear heterogeneous information machine for rgb-d action recognition. In: CVPR, pp. 1054–1062 (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Kumar, A., III, H.D.: A co-training approach for multi-view spectral clustering. In: Proceedings of International Conference on Machine Learning, pp. 393–400 (2011)
Kumar, A., Rai, P., III, H.D.: Co-regularized multi-view spectral clustering. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1413–1421 (2011)
Lange, T., Buhmann, J.M.: Fusion of similarity data in clustering. In: Proceedings of Advances in Neural Information Processing Systems (2005)
Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: Proceedings of IEEE International Conference on Computer Vision, vol. 2, pp. 1482–1489 (2005)
Li, C., Parikh, D., Chen, T.: Automatic discovery of groups of objects for scene understanding. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2012)
Li, Y., Liu, L., Shen, C., van den Hengel, A.: Mining mid-level visual patterns with deep cnn activations. Int. J. Comput. Vis. 121(3), 344–364 (2017)
Li, Y., Wang, S., Tian, Q., Ding, X.: A survey of recent advances in visual feature detection. Neurocomputing 149, 736–751 (2015)
Liu, D., Chen, T.: A topic-motion model for unsupervised video object discovery. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, Minnesota, USA (2007)
Liu, H., Yan, S.: Common visual pattern discovery via spatially coherent correspondences. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1609–1616 (2010)
Liu, J., Liu, Y.: Grasp recurring patterns from a single view. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2013)
Liu, J., Wang, C., Gao, J., Han, J.: Multi-view clustering via joint nonnegative matrix factorization. In: Proceedings of SIAM International Conference on Data Mining (2013)
Long, B., Philip, S.Y., Zhang, Z.M.: A general model for multiple view unsupervised learning. In: Proceedings of SIAM International Conference on Data Mining, pp. 822–833 (2008)
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Meng, J., Wang, H., Yuan, J., Tan, Y.P.: From keyframes to key objects: video summarization by representative object proposal selection. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1039–1048 (2016)
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1–2), 43–72 (2005)
Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: Proceedings of International Conference on Machine Learning, pp. 689–696 (2011)
Oramas, J.M., Tuytelaars, T.: Modeling visual compatibility through hierarchical mid-level elements. arXiv:1604.00036 (2016)
Perronnin, F., Dance, C.: Fisher kernels on visual vocabularies for image categorization. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Proceedings of European Conference on Computer Vision, pp. 143–156 (2010)
Philbin, J., Sivic, J., Zisserman, A.: Geometric latent dirichlet allocation on a matching graph for large-scale image datasets. Int. J. Comput. Vis. 95(2), 138–153 (2011)
Quack, T., Ferrari, V., Leibe, B., Van Gool, L.: Efficient mining of frequent and distinctive feature configurations. In: Proceedings of IEEE International Conference on Computer Vision (2007)
Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2009)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: Proceedings of IEEE International Conference on Computer Vision, pp. 2564–2571 (2011)
Russell, B., Freeman, W., Efros, A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1605–1614 (2006)
de Sa, V.R., Gallagher, P.W., Lewis, J.M., Malave, V.L.: Multi-view kernel construction. Mach. Learn. 79(1–2), 47–71 (2010)
Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv:1312.6229 (2013)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)
Sivic, J., Russell, B., Efros, A., Zisserman, A., Freeman, W.: Discovering objects and their location in images. In: Proceedings of IEEE International Conference on Computer Vision, pp. 370–377 (2005)
Sivic, J., Russell, B., Zisserman, A., Freeman, W., Efros, A.: Unsupervised discovery of visual object class hierarchies. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
Sivic, J., Zisserman, A.: Video data mining using configurations of viewpoint invariant regions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 488–495 (2004)
Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep boltzmann machines. J. Mach. Learn. Res. 15(1), 2949–2980 (2014)
Sun, M., hamme, H.V.: Image pattern discovery by using the spatial closeness of visual code words. In: Proceddings of IEEE International Conference on Image Processing, Brussels, Belgium, pp. 205–208 (2011)
Tang, J., Lewis, P.H.: Non-negative matrix factorisation for object class discovery and image auto-annotation. In: Proceedings of the International Conference on Content-based Image and Video Retrieval, Niagara Falls, Canada, pp. 105–112 (2008)
Thompson, D.W.: On Growth and Form. Cambridge University Press, Cambridge, UK (1961)
Todorovic, S., Ahuja, N.: Unsupervised category modeling, recognition, and segmentation in images. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2158–2174 (2008)
Tuytelaars, T., Lampert, C., Blaschko, M., Buntine, W.: Unsupervised object discovery: a comparison. Int. J. Comput. Vis. 88(2), 284–302 (2010)
Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Foundations and Trends® in Computer Graphics and Vision 3(3), 177–280 (2008)
Wang, B., Jiang, J., Wang, W., Zhou, Z.H., Tu, Z.: Unsupervised metric fusion by cross diffusion. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2997–3004 (2012)
Wang, G., Zhang, Y., Fei-Fei, L.: Using dependent regions for object categorization in a generative framework. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1597–1604 (2006)
Wang, H., Kawahara, Y., Weng, C., Yuan, J.: Representative selection with structured sparsity. Pattern Recognit. 63, 268–278 (2017)
Wang, H., Nie, F., Huang, H.: Multi-view clustering and feature learning via structured sparsity. In: Proceedings of International Conference on Machine Learning (2013)
Wang, H., Nie, F., Huang, H., Ding, C.: Heterogeneous visual features fusion via sparse multimodal machine. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2013)
Wang, H., Yuan, J., Tan, Y.: Combining feature context and spatial context for image pattern discovery. In: Proceedings of IEEE International Conference on Data Mining, pp. 764–773 (2011)
Wang, H., Yuan, J., Wu, Y.: Context-aware discovery of visual co-occurrence patterns. IEEE Trans. Image Process. 23(4), 1805–1819 (2014)
Wang, H., Zhao, G., Yuan, J.: Visual pattern discovery in image and video data: a brief survey. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 4(1), 24–37 (2014)
Wang, W., Arora, R., Livescu, K., Bilmes, J.A.: On deep multi-view representation learning: objectives and optimization. arXiv: 1602.01024 (2016)
Wang, X., Grimson, E.: Spatial latent dirichlet allocation. In: Proceedings of Advances in Neural Information Processing Systems (2008)
Wang, X., Qian, B., Ye, J., Davidson, I.: Multi-objective multi-view spectral clustering via pareto optimization. In: Proceedings of SIAM International Conference on Data Mining (2013)
Weng, C., Wang, H., Yuan, J., Jiang, X.: Discovering class-specific spatial layouts for scene recognition. IEEE Sig. Process. Lett. (2016)
Xu, C., Tao, D., Xu, C.: A survey on multi-view learning. arXiv:1304.5634 (2013)
Xu, C., Tao, D., Xu, C.: Large-margin multi-view information bottleneck. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1559–1572 (2014)
Xu, C., Tao, D., Xu, C.: Multi-view intact space learning. IEEE Trans. Pattern Anal. Mach. Intell. 37, 2531–2544 (2015)
Yang, J., Wang, Z., Lin, Z., Shu, X., Huang, T.: Bilevel sparse coding for coupled feature spaces. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2360–2367 (2012)
Yu, S., Tranchevent, L.C., Liu, X., Glanzel, W., Suykens, J.A., De Moor, B., Moreau, Y.: Optimized data fusion for kernel k-means clustering. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 1031–1039 (2012)
Yuan, J.: Discovering visual patterns in image and video data: concepts, algorithms, experiments. VDM Verlag Dr. Müller, Saarbrcken, Germany (2011)
Yuan, J., Wu, Y.: Spatial random partition for common visual pattern discovery. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1–8 (2007)
Yuan, J., Wu, Y.: Context-aware clustering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008)
Yuan, J., Wu, Y.: Mining visual collocation patterns via self-supervised subspace learning. IEEE Trans. Syst. Man Cybern. Part B Cybern. 42(2), 1–13 (2012)
Yuan, J., Zhao, G., Fu, Y., Li, Z., Katsaggelos, A., Wu, Y.: Discovering thematic objects in image collections and videos. IEEE Trans. Image Process. 21, 2207–2219 (2012)
Zhang, S., Yang, M., Cour, T., Yu, K., Metaxas, D.N.: Query specific fusion for image retrieval. In: Proceedings of European Conference on Computer Vision, pp. 660–673 (2012)
Zhang, S., Yang, M., Wang, X., Lin, Y., Tian, Q.: Semantic-aware co-indexing for image retrieval. In: Proceedings of IEEE International Conference on Computer Vision, pp. 1673–1680 (2013)
Zhang, Y., Jia, Z., Chen, T.: Image retrieval with geometry-preserving visual phrases. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 809–816 (2011)
Zhao, G., Yuan, J.: Discovering thematic patterns in videos via cohesive sub-graph mining. In: Proceedings of IEEE International Conference on Data Mining, pp. 1260–1265 (2011)
Zhao, G., Yuan, J., Hua, G.: Topical video object discovery from key frames by modeling word co-occurrence prior. IEEE Trans. Image Process. (2015)
Zhao, G., Yuan, J., Xu, J., Wu, Y.: Discovery of the thematic object in commercial videos. IEEE Multimed. Mag. 18(3), 56–65 (2011)
Zhao, J., Xie, X., Xu, X., Sun, S.: Multi-view learning overview: recent progress and new challenges. Inf. Fusion 38, 43–54 (2017)
Zheng, L., Wang, S., Liu, Z., Tian, Q.: Packing and padding: coupled multi-index for accurate image retrieval. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1947–1954 (2014)
Zhu, S., Guo, C., Wang, Y., Xu, Z.: What are textons? Int. J. Comput. Vis. 62(1), 121–143 (2005)
Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: Proceedings of European Conference on Computer Vision, pp. 391–405 (2014)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2017 The Author(s)
About this chapter
Cite this chapter
Wang, H., Weng, C., Yuan, J. (2017). Introduction. In: Visual Pattern Discovery and Recognition. SpringerBriefs in Computer Science. Springer, Singapore. https://doi.org/10.1007/978-981-10-4840-1_1
Download citation
DOI: https://doi.org/10.1007/978-981-10-4840-1_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-4839-5
Online ISBN: 978-981-10-4840-1
eBook Packages: Computer ScienceComputer Science (R0)