U-PC: Unsupervised Planogram Compliance

  • Archan Ray
  • Nishant Kumar
  • Avishek Shaw
  • Dipti Prasad MukherjeeEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11214)


We present an end-to-end solution for recognizing merchandise displayed in the shelves of a supermarket. Given images of individual products, which are taken under ideal illumination for product marketing, the challenge is to find these products automatically in the images of the shelves. Note that the images of shelves are taken using hand-held camera under store level illumination. We provide a two-layer hypotheses generation and verification model. In the first layer, the model predicts a set of candidate merchandise at a specific location of the shelf while in the second layer, the hypothesis is verified by a novel graph theoretic approach. The performance of the proposed approach on two publicly available datasets is better than the competing approaches by at least 10%.


Planogram compliance Merchandise recognition 



This work is partially supported by TCS Limited. The authors would like to thank Mr. Bikash Santra for his help in preparing the manuscript.


  1. 1.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). Scholar
  2. 2.
    Cheng, D., Prasad, D.K., Brown, M.S.: Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. JOSA A 31(5), 1049–1058 (2014)CrossRefGoogle Scholar
  3. 3.
    Cormen, T.H., Leiserson, C.E., Rivest, R.L.: Introduction to Algorithms, 3rd edn. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  4. 4.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  5. 5.
    George, M., Floerkemeier, C.: Recognizing products: a per-exemplar multi-label image classification approach. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 440–455. Springer, Cham (2014). Scholar
  6. 6.
    George, M., Mircic, D., Soros, G., Floerkemeier, C., Mattern, F.: Fine-grained product class recognition for assisted shopping. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 154–162 (2015)Google Scholar
  7. 7.
    Gevers, T., Smeulders, A.W.: Color-based object recognition. Pattern Recognit. 32(3), 453–464 (1999)CrossRefGoogle Scholar
  8. 8.
    Goldberg, D.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Boston (1989)zbMATHGoogle Scholar
  9. 9.
    Jaderberg, M., Vedaldi, A., Zisserman, A.: Deep features for text spotting. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 512–528. Springer, Cham (2014). Scholar
  10. 10.
    Kim, J., Liu, C., Sha, F., Grauman, K.: Deformable spatial pyramid matching for fast dense correspondences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2307–2314 (2013)Google Scholar
  11. 11.
    MacKay, D.J.: Information-based objective functions for active data selection. Neural Comput. 4(4), 590–604 (1992)CrossRefGoogle Scholar
  12. 12.
    Merler, M., Galleguillos, C., Belongie, S.: Recognizing groceries in situ using in vitro training data. In: CVPR. IEEE Computer Society (2007)Google Scholar
  13. 13.
    Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  14. 14.
    Singh, S., Gupta, A., Efros, A.A.: Unsupervised discovery of mid-level discriminative patches. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, pp. 73–86. Springer, Heidelberg (2012). Scholar
  15. 15.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3360–3367. IEEE (2010)Google Scholar
  16. 16.
    Yao, B., Khosla, A., Fei-Fei, L.: Combining randomization and discrimination for fine-grained image categorization. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1577–1584. IEEE (2011)Google Scholar
  17. 17.
    Zhang, Y., Wang, L., Hartley, R., Li, H.: Where’s the weet-bix? In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4843, pp. 800–810. Springer, Heidelberg (2007). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Archan Ray
    • 1
  • Nishant Kumar
    • 2
  • Avishek Shaw
    • 3
  • Dipti Prasad Mukherjee
    • 4
    Email author
  1. 1.University of MassachusettsAmherstUSA
  2. 2.Singapore-MIT Alliance for Research and TechnologySingaporeSingapore
  3. 3.TCS LimitedMumbaiIndia
  4. 4.Indian Statistical InstituteKolkataIndia

Personalised recommendations