Skip to main content
SpringerLink
Log in

Automatic Visual Pattern Discovery via Cohesive Subgraph Mining

  • Chapter
  • First Online:
Mobile Cloud Visual Media Computing

  • 855 Accesses

Abstract

One category of videos usually contains the same thematic pattern, e.g., the spin action in skating videos. The discovery of the thematic pattern is essential to understand and summarize the video contents. This article addresses two critical issues in mining thematic video patterns: (1) automatic discovery of thematic patterns without any training or supervision information, and (2) accurate localization of the occurrences of all thematic patterns in videos. The major contributions are twofold. First, we formulate the thematic video pattern discovery as a cohesive subgraph selection problem by finding a subset of visual words that are spatio-temporally collocated. Then spatio-temporal branch-and-bound search can locate all instances accurately. Second, a novel method is proposed to efficiently find the cohesive subgraph of maximum overall mutual information scores. Our experimental results on challenging commercial and action videos show that our approach can discover different types of thematic patterns despite variations in scale, view-point, color, and lighting conditions, or partial occlusions. Our approach is also robust to the videos with cluttered and dynamic backgrounds.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003). doi:10.1162/jmlr.2003.3.4-5.993

    MATH  Google Scholar 

  2. Chen, C., Mangasarian, O.L.: A class of smoothing functions for nonlinear and mixed complementarity problems. Comput. Optim. Appl. 5, 97–138 (1996). doi:10.1007/BF00249052

    Article  MathSciNet  MATH  Google Scholar 

  3. Chum, O., Matas, J.: Large-scale discovery of spatially related images. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 371–377 (2010). doi:10.1109/TPAMI.2009.166

  4. Du, L., Buntine, W.L., Jin, H.: Sequential latent dirichlet allocation: Discover underlying topic structures within a document. In: Proceedings of the 2010 IEEE International Conference on Data Mining, ICDM ’10, pp. 148–157. IEEE Computer Society (2010). doi:10.1109/ICDM.2010.51

  5. Gao, J., Hu, Y., Liu, J., Yang, R.: Unsupervised learning of high-order structural semantics from images. In: IEEE 12th International Conference on Computer Vision, ICCV 2009, Kyoto, Japan, September 27–October 4, 2009, pp. 2122–2129 (2009). doi:10.1109/ICCV.2009.5459465

  6. Han, J., Cheng, H., Xin, D., Yan, X.: Frequent pattern mining: current status and future directions. Data Min. Knowl. Discov. 15, 55–86 (2007). doi:10.1007/s10618-006-0059-1. http://portal.acm.org/citation.cfm?id=1275092.1275097

  7. Hong, P., Huang, T.S.: Spatial pattern discovery by learning a probabilistic parametric model from multiple attributed relational graphs. Discrete Appl. Math. 139(1–3), 113–135 (2004). doi:10.1016/j.dam.2002.11.007

    Article  MathSciNet  MATH  Google Scholar 

  8. Kim, G., Xing, E.P.: Jointly aligning and segmenting multiple web photo streams for the inference of collective photo storylines. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 620–627. IEEE Computer Society, Washington, DC (2013). doi:10.1109/CVPR.2013.86

  9. Lampert, C.H., Blaschko, M.B., Hofmann, T.: Efficient subwindow search: a branch and bound framework for object localization. TPAMI 31(12), 2129–2142 (2009). doi:10.1109/TPAMI.2009.144

    Article  Google Scholar 

  10. Laptev, I.: On space-time interest points. Int. J. Comput. Vision 64(2–3), 107–123 (2005). doi:10.1007/s11263-005-1838-7

  11. Lee, Y.J., Grauman, K.: Shape discovery from unlabeled image collections. In: 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20–25 June 2009, Miami, Florida, USA, pp. 2254–2261 (2009). doi:10.1109/CVPRW.2009.5206698

  12. Li, Q., Wu, J., Tu, Z.: Harvesting mid-level visual concepts from large-scale internet images. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 851–858. IEEE Computer Society, Washington, DC (2013). doi:10.1109/CVPR.2013.115

  13. Liu, D., Hua, G., Chen, T.: A hierarchical visual model for video object summarization. TPAMI (2010). http://doi.ieeecomputersociety.org/10.1109/TPAMI.2010.31

  14. Liu, J., Liu, Y.: Grasp recurring patterns from a single view. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 2003–2010. IEEE Computer Society, Washington, DC (2013). doi:10.1109/CVPR.2013.261

  15. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004). doi:10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  16. Luo, Y., Zhao, G., Yuan, J.: Thematic saliency detection using spatial-temporal context. In: Proceedings of the 2013 IEEE International Conference on Computer Vision Workshops, ICCVW ’13, pp. 347–353. IEEE Computer Society, Washington, DC (2013). doi:10.1109/ICCVW.2013.53

  17. Ng, K.M.: A continuation approach for solving nonlinear optimization problems with discrete variables. Ph.d. Dissertation, Stanford University (2002)

    Google Scholar 

  18. Pavan, M., Pelillo, M.: Dominant sets and pairwise clustering. TPAMI 29, 167–172 (2007). doi:10.1109/TPAMI.2007.10

    Article  Google Scholar 

  19. Rodriguez, M.D., Ahmed, J., Shah, M.: Action MACH a spatio-temporal maximum average correlation height filter for action recognition. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008), 24–26 June 2008, Anchorage, Alaska (2008). doi:10.1109/CVPR.2008.4587727

  20. Rubinstein, M., Joulin, A., Kopf, J., Liu, C.: Unsupervised joint object discovery and segmentation in internet images. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 1939–1946. IEEE Computer Society, Washington, DC (2013). doi:10.1109/CVPR.2013.253

  21. Russell, B.C., Freeman, W.T., Efros, A.A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR ’06, vol. 2, pp. 1605–1614. IEEE Computer Society (2006). doi:|DOIurl10.1109/CVPR.2006.326

    Google Scholar 

  22. Shan, M.K., Wei, L.Y.: Algorithms for discovery of spatial co-orientation patterns from images. Expert Syst. Appl. 37, 5795–5802 (2010). doi:10.1016/j.eswa.2010.02.028

    Article  Google Scholar 

  23. Tang, K., Sukthankar, R., Yagnik, J., Fei-Fei, L.: Discriminative segment annotation in weakly labeled video. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’13, pp. 2483–2490. IEEE Computer Society, Washington, DC (2013). doi:10.1109/CVPR.2013.321

  24. Wang, D., Li, T., Ding, C.: Weighted feature subset non-negative matrix factorization and its applications to document understanding. In: ICDM10, pp. 541–550. IEEE Computer Society (2010). http://dx.doi.org/10.1109/ICDM.2010.47

  25. Wang, H., Zhao, G., Yuan, J.: Visual pattern discovery in image and video data: a brief survey. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 4(1), 24–37 (2014). doi:10.1002/widm.1110

    Google Scholar 

  26. Wang, L., Hua, G., Sukthankar, R., Xue, J., Zheng, N.: Video object discovery and co-segmentation with extremely weak supervision. In: Proceeding of the European Conference on Computer Vision (2014)

    Google Scholar 

  27. Todorovic, S., Ahuja, N.: Unsupervised category modeling, recognition, and segmentation in images. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2158–2174 (2008). doi:10.1109/TPAMI.2008.24

    Article  Google Scholar 

  28. Xie, Y., Yu, P.S.: Max-clique: A top-down graph-based approach to frequent pattern mining. In: ICDM10, pp. 1139–1144. IEEE Computer Society (2010). http://dx.doi.org/10.1109/ICDM.2010.73

  29. Xu, J., Yuan, J., Wu, Y.: Learning spatio-temporal dependency of local patches for complex motion segmentation. Comput. Vis. Image Underst. 115, 334–351 (2011). doi:10.1016/j.cviu.2010.11.010

    Article  Google Scholar 

  30. Yuan, J., Wu, Y., Yang, M.: From frequent itemsets to semantically meaningful visual patterns. ACM SIGKDD (2007). http://doi.acm.org/10.1145/1281192.1281284

  31. Yuan, J., Liu, Z., Wu, Y.: Discriminative video pattern search for efficient action detection. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1728–1743 (2011). doi:10.1109/TPAMI.2011.38

  32. Zhang, D., Javed, O., Shah, M.: Video object co-segmentation by regulated maximum weight cliques. In: Proceeding of the European Conference on Computer Vision (2014)

    Google Scholar 

  33. Zhao, G., Yuan, J.: Discovering thematic patterns in videos via cohesive sub-graph mining. In: Proceedings of the 2011 IEEE 11th International Conference on Data Mining, ICDM ’11, pp. 1260–1265. IEEE Computer Society, Washington, DC (2011). doi:10.1109/ICDM.2011.55

  34. Zhao, G., Yuan, J.: Mining and cropping common objects from images. In: Proceedings of the International Conference on Multimedia, MM ’10, pp. 975–978. ACM, New York (2010). doi:10.1145/1873951.1874127

Download references

Acknowledgments

This project is supported in part by MoE Tier-1 grant “Exploring Visual Relevance to Construct a Holistic Picture of Online News”.

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    Gangqiang Zhao & Junsong Yuan

Authors
  1. Gangqiang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junsong Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gangqiang Zhao .

Editor information

Editors and Affiliations

  1. Visual Computing Group, Microsoft Research Asia, Beijing, Beijing, China

    Gang Hua

  2. Alibaba Group, Hangzhou, Zhejiang, China

    Xian-Sheng Hua

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Zhao, G., Yuan, J. (2015). Automatic Visual Pattern Discovery via Cohesive Subgraph Mining. In: Hua, G., Hua, XS. (eds) Mobile Cloud Visual Media Computing. Springer, Cham. https://doi.org/10.1007/978-3-319-24702-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24702-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24700-7

  • Online ISBN: 978-3-319-24702-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation