Unsupervised Multiple Object Cosegmentation via Ensemble MIML Learning

  • Weichen Yang
  • Zhengxing SunEmail author
  • Bo Li
  • Jiagao Hu
  • Kewei Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)


Multiple foreground cosegmentation (MFC) has being a new research topic recently in computer vision. This paper proposes a framework of unsupervised multiple object cosegmentation, which is composed of three components: unsupervised label generation, saliency pseudo-annotation and cosegmentation based on MIML learning. Based on object detection, unsupervised label generation is done in terms of the two-stage object clustering method, to obtain accurate consistent label between common objects without any user intervention. Then, the object label is propagated to the object saliency coming from saliency detection method, to finish saliency pseudo-annotation. This makes an unsupervised MFC problem as a supervised multi-instance multi-label (MIML) learning problem. Finally, an ensemble MIML framework is introduced to achieve image cosegmentation based on random feature selection. The experimental results on data sets ICoseg and FlickrMFC demonstrated the effectiveness of the proposed approach.


Multiple foreground cosegmentation Unsupervised label generation Saliency pseudo-annotation Cosegmentation based on MIML learning 



This work is supported by National High Technology Research and Development Program of China (No. 2007AA01Z334), National Natural Science Foundation of China (No. 61321491, 61272219), Innovation Fund of State Key Laboratory for Novel Software Technology (No. ZZKT2013A12, ZZKT2016A11), Program for New Century Excellent Talents in University of China (NCET-04-04605).


  1. 1.
    Kim, G., Xing, E.P.: On multiple foreground cosegmentation. In: IEEE CVPR, pp. 837–844 (2012)Google Scholar
  2. 2.
    Li, H., Meng, F.: Unsupervised multiclass region cosegmentation via ensemble clustering and energy minimization. IEEE Trans. Circ. Syst. Video Technol. 24(5), 789–801 (2014)CrossRefGoogle Scholar
  3. 3.
    Li, K., Zhang, J., Tao, W.: Unsupervised co-segmentation for indefinite number of common foreground objects. IEEE Trans. Image Process. 25(4), 1898–1909 (2016)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Meng, F., Li, H.: Constrained directed graph clustering and segmentation propagation for multiple foregrounds cosegmentation. IEEE Trans. Circ. Syst. Video Technol. 25(11), 1735–1748 (2015)CrossRefGoogle Scholar
  5. 5.
    Li, L., Fei, X.: Unsupervised multi-class co-segmentation via joint object detection and segmentation with energy minimization. In: MIPPR, pp. 9812–9814 (2015)Google Scholar
  6. 6.
    Chang, H.S., Wang, Y.C.F.: Optimizing the decomposition for multiple foreground cosegmentation. Comput. Vis. Image Underst. 141, 18–27 (2015)CrossRefGoogle Scholar
  7. 7.
    Rother, C., Minka, T.: Cosegmentation of image pairs by histogram matching-incorporating a global constraint into MRFs. In: IEEE CVPR, pp. 993–1000 (2006)Google Scholar
  8. 8.
    Mukherjee, L., Singh, V., Dyer, C.R.: Half-integrality based algorithms for cosegmentation of images. In: IEEE CVPR, pp. 2028–2035 (2009)Google Scholar
  9. 9.
    Kim, G., Xing, E.P., Fei-Fei, L., Kanade, T.: Distributed cosegmentation via submodular optimization on anisotropic diffusion. In: IEEE ICCV, pp. 169–176 (2011)Google Scholar
  10. 10.
    Wang, F., Huang, Q., Ovsjanikov, M., Guibas, L.J.: Unsupervised multi-class joint image segmentation. In: IEEE CVPR, pp. 3142–3149 (2014)Google Scholar
  11. 11.
    Ma, T., Jan Latecki, L.: Graph transduction learning with connectivity constraints with application to multiple foreground cosegmentation. In: IEEE CVPR, pp. 1955–1962(2013)Google Scholar
  12. 12.
    Zhu, H., Lu, J., Cai, J., Zheng, J., Thalmann, N.M.: Multiple foreground recognition and cosegmentation: an object-oriented CRF model with robust higher-order potentials. In: IEEE WACV, pp. 485–492 (2014)Google Scholar
  13. 13.
    Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 391–405. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10602-1_26 Google Scholar
  14. 14.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE CVPR, pp. 3431–3440 (2015)Google Scholar
  15. 15.
    Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: IEEE CVPR, pp. 2814–2821 (2014)Google Scholar
  17. 17.
    Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H., Hu, S.M.: Global contrast based salient region detection. IEEE TPAMI 37(3), 569–582 (2015)CrossRefGoogle Scholar
  18. 18.
    Achanta, R., Smith, S.A.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE TPAMI 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  19. 19.
    Zhou, Z.H., Zhang, M.L.: Multi-instance multi-label learning with application to scene classification. In: NIPS, pp. 1609–1616 (2006)Google Scholar
  20. 20.
    Briggs, F., Fern, X.Z., Raich, R.: Rank-loss support instance machines for MIML instance annotation. In: ACM SIGKDD, pp. 534–542 (2012)Google Scholar
  21. 21.
    Batra, D., Kowdle, A.: iCoseg: interactive co-segmentation with intelligent scribble guidance. In: IEEE CVPR, pp. 3169–3176 (2010)Google Scholar
  22. 22.
    Joulin, A., Bach, F., Ponce, J.: Discriminative clustering for image co-segmentation. In: IEEE CVPR, pp. 1943–1950 (2010)Google Scholar
  23. 23.
    Vicente, S., Rother, C., Kolmogorov, V.: Object cosegmentation. In: IEEE CVPR, pp. 2217–2224 (2011)Google Scholar
  24. 24.
    Rubinstein, M., Joulin, A., Koft, J., Liu, C.: Object co-segmentation based on shortest path algorithm and saliency model. IEEE Trans. Multimedia 14(5), 1429–1441 (2012)CrossRefGoogle Scholar
  25. 25.
    Rubinstein, M., Joulin, A., Kopf, J., Liu, C.: Unsupervised joint object discovery and segmentation in internet images. In: IEEE CVPR, pp. 1939–1946 (2013)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Weichen Yang
    • 1
  • Zhengxing Sun
    • 1
    Email author
  • Bo Li
    • 1
  • Jiagao Hu
    • 1
  • Kewei Yang
    • 1
  1. 1.State Key Laboratory for Novel Software TechnologyNanjing UniversityNanjingPeople’s Republic of China

Personalised recommendations