Video Object Co-segmentation by Regulated Maximum Weight Cliques

  • Dong Zhang
  • Omar Javed
  • Mubarak Shah
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8695)


In this paper, we propose a novel approach for object co-segmentation in arbitrary videos by sampling, tracking and matching object proposals via a Regulated Maximum Weight Clique (RMWC) extraction scheme. The proposed approach is able to achieve good segmentation results by pruning away noisy segments in each video through selection of object proposal tracklets that are spatially salient and temporally consistent, and by iteratively extracting weighted groupings of objects with similar shape and appearance (with-in and across videos). The object regions obtained from the video sets are used to initialize per-pixel segmentation to get the final co-segmentation results. Our approach is general in the sense that it can handle multiple objects, temporary occlusions, and objects going in and out of view. Additionally, it makes no prior assumption on the commonality of objects in the video collection. The proposed method is evaluated on publicly available multi-class video object co-segmentation dataset and demonstrates improved performance compared to the state-of-the-art methods.


Video Segmentation Cosegmentation 


  1. 1.
    Alexe, B., Deselares, T., Ferrari, V.: Measuring the objectness of image windows. PAMI (2012)Google Scholar
  2. 2.
    Bron, C., Kerbosch, J.: Algorithm 457: finding all cliques of an undirected graph. Communications of the ACM 16(9), 575–577 (1973)CrossRefzbMATHGoogle Scholar
  3. 3.
    Chen, D.J., Chen, H.T., Chang, L.W.: Video object cosegmentation. In: ACM MM, pp. 805–808 (2012)Google Scholar
  4. 4.
    Chiu, W.C., Fritz, M.: Multi-class video co-segmentation with a generative multi-video model. In: CVPR (2013)Google Scholar
  5. 5.
    Endres, I., Hoiem, D.: Category independent object proposals. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 575–588. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Fairchild, M.D.: Color appearance models. John Wiley & Sons (2013)Google Scholar
  7. 7.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. PAMI 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  8. 8.
    Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: ICCV, pp. 670–677. IEEE (2009)Google Scholar
  9. 9.
    Galasso, F., Iwasaki, M., Nobori, K., Cipolla, R.: Spatio-temporal clustering of probabilistic region trajectories. In: ICCV, pp. 1738–1745. IEEE (2011)Google Scholar
  10. 10.
    Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph based video segmentation. In: CVPR (2010)Google Scholar
  11. 11.
    Jain, B., Obermayer, K.: Extending bron kerbosch for solving the maximum weight clique problem. arXiv preprint arXiv:1101.1266 (2011)Google Scholar
  12. 12.
    Joulin, A., Bach, F., Ponce, J.: Discriminative clustering for image co-segmentation. In: CVPR, pp. 1943–1950. IEEE (2010)Google Scholar
  13. 13.
    Joulin, A., Bach, F., Ponce, J.: Multi-class cosegmentation. In: CVPR, pp. 542–549. IEEE (2012)Google Scholar
  14. 14.
    Kim, G., Torralba, A.: Unsupervised detection of regions of interest using iterative link analysis. In: NIPS (2009)Google Scholar
  15. 15.
    Kumlander, D.: A new exact algorithm for the maximum-weight clique problem based on a heuristic vertex-coloring and a backtrack search. In: Proc. 5th Int. Conf. on Modelling, Computation and Optimization in Information Systems and Management Sciences, pp. 202–208 (2004)Google Scholar
  16. 16.
    Lee, Y., Kim, J., Grauman, K.: Key-segments for video object segmentation. In: ICCV, pp. 1995–2002. IEEE (2011)Google Scholar
  17. 17.
    Ma, T., Latecki, L.: Maximum weight cliques with mutex constraints for video object segmentation. In: CVPR, pp. 670–677. IEEE (2012)Google Scholar
  18. 18.
    Mukherjee, L., Singh, V., Peng, J.: Scale invariant cosegmentation for image groups. In: CVPR, pp. 1881–1888. IEEE (2011)Google Scholar
  19. 19.
    Ochs, P., Brox, T.: Object segmentation in video: a hierarchical variational approach for turning point trajectories into dense regions. In: ICCV, pp. 1583–1590. IEEE (2011)Google Scholar
  20. 20.
    Rother, C., Minka, T., Blake, A., Kolmogorov, V.: Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs. In: CVPR, pp. 993–1000. IEEE (2006)Google Scholar
  21. 21.
    Rubinstein, M., Joulin, A., Johannes, K., Liu, C.: Unsupervised joint object discovery and segmentation in internet images. In: CVPR (2013)Google Scholar
  22. 22.
    Rubio, J.C., Serrat, J., López, A.: Video co-segmentation. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part II. LNCS, vol. 7725, pp. 13–24. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  23. 23.
    Russell, B.C., Freeman, W.T., Efros, A.A., Sivic, J., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: CVPR, vol. 2, pp. 1605–1614. IEEE (2006)Google Scholar
  24. 24.
    Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering objects and their location in images. In: ICCV, vol. 1, pp. 370–377. IEEE (2005)Google Scholar
  25. 25.
    Tomita, E., Tanaka, A., Takahashi, H.: The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical Computer Science 363(1), 28–42 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Vicente, S., Rother, C., Kolmogorov, V.: Object cosegmentation. In: CVPR, pp. 2217–2224. IEEE (2011)Google Scholar
  27. 27.
    Xu, C., Xiong, C., Corso, J.J.: Streaming hierarchical video segmentation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 626–639. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
    Yuan, J., Zhao, G., Fu, Y., Li, Z., Katsaggelos, A.K., Wu, Y.: Discovering thematic objects in image collections and videos. IEEE Transactions on Image Processing 21(4), 2207–2219 (2012)CrossRefMathSciNetGoogle Scholar
  29. 29.
    Zhang, D., Javed, O., Shah, M.: Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: CVPR (2013)Google Scholar
  30. 30.
    Zhao, G., Yuan, J., Hua, G.: Topical video object discovery from key frames by modeling word co-occurrence prior. In: CVPR (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Dong Zhang
    • 1
  • Omar Javed
    • 2
  • Mubarak Shah
    • 1
  1. 1.Center for Research in Computer VisionUCFOrlandoUSA
  2. 2.SRI InternationalPrincetonUSA

Personalised recommendations