Common Object Discovery as Local Search for Maximum Weight Cliques in a Global Object Similarity Graph

  • Cong RaoEmail author
  • Yi Fan
  • Kaile Su
  • Longin Jan Latecki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11414)


In this paper, we consider the task of discovering the common objects in images. Initially, object candidates are generated in each image and an undirected weighted graph is constructed over all the candidates. Each candidate serves as a node in the graph while the weight of the edge describes the similarity between the corresponding pair of candidates. The problem is then expressed as a search for the Maximum Weight Clique (MWC) in this graph. The MWC corresponds to a set of object candidates sharing maximal mutual similarity, and each node in the MWC represents a discovered common object across the images. Since the problem of finding the MWC is NP-hard, most research of the MWC problem focuses on developing various heuristics for finding good cliques within a reasonable time limit. We utilize a recently very popular class of heuristics called local search methods. They search for the MWC directly in the discrete domain of the solution space. The proposed approach is evaluated on the PASCAL VOC image dataset and the YouTube-Objects video dataset, and it demonstrates superior performance over recent state-of-the-art approaches.


Common Object Discovery Visual Similarity Maximum Weight Clique Local search algorithm 



This work was supported in part by NSF grants IIS-1814745 and IIS-1302164. Yi Fan is supported by the Natural Science Foundation of China (Nos. U1711263, U1811264, 61463044, 61572234, 61672441, and 61603152) and the Shenzhen Basic Research Program (No. JCYJ20170818141325209). We also acknowledge donations of Titan X GPU cards by NVIDIA corporation.


  1. 1.
    Adamczewski, K., Suh, Y., Mu Lee, K.: Discrete tabu search for graph matching. In: ICCV, pp. 109–117 (2015)Google Scholar
  2. 2.
    Alidaee, B., Glover, F., Kochenberger, G., Wang, H.: Solving the maximum edge weight clique problem via unconstrained quadratic programming. Eur. J. Oper. Res. 181(2), 592–597 (2007)CrossRefGoogle Scholar
  3. 3.
    Bilen, H., Pedersoli, M., Tuytelaars, T.: Weakly supervised object detection with convex clustering. In: CVPR, pp. 1081–1089 (2015)Google Scholar
  4. 4.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: BMVC (2014)Google Scholar
  5. 5.
    Chen, W., Chen, X., Zhang, J., Huang, K.: Beyond triplet loss: a deep quadruplet network for person re-identification. arXiv (2017)Google Scholar
  6. 6.
    Cho, M., Kwak, S., Schmid, C., Ponce, J.: Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals. In: CVPR, pp. 1201–1210 (2015)Google Scholar
  7. 7.
    Cinbis, R.G., Verbeek, J., Schmid, C.: Weakly supervised object localization with multi-fold multiple instance learning. TPAMI 39, 189–203 (2016)CrossRefGoogle Scholar
  8. 8.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results (2007)Google Scholar
  9. 9.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results (2012)Google Scholar
  10. 10.
    Fan, Y., Li, C., Ma, Z., Wen, L., Sattar, A., Su, K.: Local search for maximum vertex weight clique on large sparse graphs with efficient data structures. In: Kang, B.H., Bai, Q. (eds.) AI 2016. LNCS (LNAI), vol. 9992, pp. 255–267. Springer, Cham (2016). Scholar
  11. 11.
    Fan, Y., Li, N., Li, C., Ma, Z., Latecki, L.J., Su, K.: Restart and random walk in local search for maximum vertex weight cliques. In: IJCAI (2017)Google Scholar
  12. 12.
    Gouveia, L., Martins, P.: Solving the maximum edge-weight clique problem in sparse graphs with compact formulations. EURO J. Comput. Optim. 3(1), 1–30 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Cham (2015). Scholar
  14. 14.
    Hosseinian, S., Fontes, D.B., Butenko, S.: A nonconvex quadratic optimization approach to the maximum edge weight clique problem. J. Glob. Optim. 72, 1–22 (2018)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Joulin, A., Tang, K., Fei-Fei, L.: Efficient image and video co-localization with Frank-Wolfe algorithm. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 253–268. Springer, Cham (2014). Scholar
  16. 16.
    Kalogeiton, V., Ferrari, V., Schmid, C.: Analysing domain shift factors between videos and images for object detection. TPAMI 38(11), 2327–2334 (2016)CrossRefGoogle Scholar
  17. 17.
    Kwak, S., Cho, M., Laptev, I., Ponce, J., Schmid, C.: Unsupervised object discovery and tracking in video collections. In: ICCV, pp. 3173–3181 (2015)Google Scholar
  18. 18.
    Li, Y., Liu, L., Shen, C., van den Hengel, A.: Image co-localization by mimicking a good detector’s confidence score distribution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 19–34. Springer, Cham (2016). Scholar
  19. 19.
    Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar
  20. 20.
    Ma, T., Latecki, L.J.: Maximum weight cliques with mutex constraints for video object segmentation. In: CVPR, pp. 670–677 (2012)Google Scholar
  21. 21.
    Mascia, F., Cilia, E., Brunato, M., Passerini, A.: Predicting structural and functional sites in proteins by searching for maximum-weight cliques. In: AAAI (2010)Google Scholar
  22. 22.
    Papazoglou, A., Ferrari, V.: Fast object segmentation in unconstrained video. In: ICCV, pp. 1777–1784 (2013)Google Scholar
  23. 23.
    Prest, A., Leistner, C., Civera, J., Schmid, C., Ferrari, V.: Learning object class detectors from weakly annotated video. In: CVPR, pp. 3282–3289 (2012)Google Scholar
  24. 24.
    Pullan, W.: Approximating the maximum vertex/edge weighted clique using local search. J. Heuristics 14(2), 117–134 (2008)CrossRefGoogle Scholar
  25. 25.
    Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. TPAMI 39(6), 1137–1149 (2017)CrossRefGoogle Scholar
  26. 26.
    Ren, W., Huang, K., Tao, D., Tan, T.: Weakly supervised large scale object localization with multiple instance learning and bag splitting. TPAMI 38(2), 405–416 (2016)CrossRefGoogle Scholar
  27. 27.
    Rochan, M., Wang, Y.: Weakly supervised localization of novel objects using appearance transfer. In: CVPR, pp. 4315–4324 (2015)Google Scholar
  28. 28.
    Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 430–443. Springer, Heidelberg (2006). Scholar
  29. 29.
    Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. IJCV 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv (2014)Google Scholar
  31. 31.
    Tang, K., Joulin, A., Li, L.J., Fei-Fei, L.: Co-localization in real-world images. In: CVPR, pp. 1464–1471 (2014)Google Scholar
  32. 32.
    Wang, C., Zhang, H., Yang, L., Cao, X., Xiong, H.: Multiple semantic matching on augmented n-partite graph for object co-segmentation. TIP 26(12), 5825–5839 (2017)MathSciNetGoogle Scholar
  33. 33.
    Wang, C., Ren, W., Huang, K., Tan, T.: Weakly supervised object localization with latent category learning. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 431–445. Springer, Cham (2014). Scholar
  34. 34.
    Wang, X., Zhu, Z., Yao, C., Bai, X.: Relaxed multiple-instance SVM with application to object discovery. In: ICCV, pp. 1224–1232 (2015)Google Scholar
  35. 35.
    Wang, Y., Cai, S., Yin, M.: Two efficient local search algorithms for maximum weight clique problem. In: AAAI, pp. 805–811 (2016)Google Scholar
  36. 36.
    Wei, X.S., et al.: Deep descriptor transforming for image co-localization. arXiv (2017)Google Scholar
  37. 37.
    Zhang, Y., Chen, X., Li, J., Wang, C., Xia, C.: Semantic object segmentation via detection in weakly labeled video. In: CVPR, pp. 3641–3649 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Cong Rao
    • 1
    Email author
  • Yi Fan
    • 2
    • 3
  • Kaile Su
    • 3
  • Longin Jan Latecki
    • 1
  1. 1.Temple UniversityPhiladelphiaUSA
  2. 2.Guangxi Key Lab of Trusted SoftwareGuilin University of Electronic TechnologyGuilinChina
  3. 3.Griffith UniversityBrisbaneAustralia

Personalised recommendations