Color Inference from Semantic Labeling for Person Search in Videos

  • Jules SimonEmail author
  • Guillaume-Alexandre BilodeauEmail author
  • David Steele
  • Harshad Mahadik
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12131)


We propose an explainable model for classifying the color of pixels in images. We propose a method based on binary search trees and a large peer-labeled color name dataset, allowing us to synthesize the average human perception of colors. We test our method on the application of Person Search. In this context, persons are described from their semantic parts, such as hat, shirt, ... and person search consists in looking for people based on these descriptions. We label segments of pedestrians with their associated colors and evaluate our solution on datasets such as PCN and Colorful-Fashion. We show a precision as high as 83% as well as the model ability to generalize to multiple domains with no retraining.


Color classification Person search Semantic color labeling 



We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), [CRDPJ 528786 - 18], and the support of Arcturus network.


  1. 1.
    X11 Color Names. GitLabGoogle Scholar
  2. 2.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels. Technical report (2010)Google Scholar
  3. 3.
    Baslamisli, A.S., Le, H.A., Gevers, T.: CNN based learning using reflection and Retinex models for intrinsic image decomposition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018Google Scholar
  4. 4.
    Benavente, R., Vanrell, M., Baldrich, R.: Parametric fuzzy sets for automatic color naming. J. Opt. Soc. Am. A: 25(10), 2582 (2008)CrossRefGoogle Scholar
  5. 5.
    Berlin, B., Kay, P.: Basic Color Terms: Their Universality and Evolution. Center for the Study of Language and Information (1999)Google Scholar
  6. 6.
    Bianco, S., Cusano, C.: Quasi-unsupervised color constancy. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019Google Scholar
  7. 7.
    Billmeyer Jr., F.W.: Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd edn. Color Research & Application (1983)Google Scholar
  8. 8.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefGoogle Scholar
  9. 9.
    Cheng, Z., Li, X., Loy, C.C.: Pedestrian color naming via convolutional neural network. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10112, pp. 35–51. Springer, Cham (2017). Scholar
  10. 10.
    Jobson, D.J., Rahman, Z., Woodell, G.A.: Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6(3), 451–462 (1997)CrossRefGoogle Scholar
  11. 11.
    Kelly, K.L., Judd, D.B.: Inter-Society Color Council.: The ISCC-NBS Method of Designating Colors and a Dictionary of Color Names. National Bureau of Standards Circular, United States (1955)Google Scholar
  12. 12.
    Land, E.H., McCann, J.J.: Lightness and retinex theory. Josa 61(1), 1–11 (1971)CrossRefGoogle Scholar
  13. 13.
    Lin, Y., et al.: Improving person re-identification by attribute and identity learning. Pattern Recogn. 95, 151–161 (2019)CrossRefGoogle Scholar
  14. 14.
    Liu, S., et al.: Fashion parsing with weak color-category labels. IEEE Trans. Multimedia 16(1), 253–265 (2014)CrossRefGoogle Scholar
  15. 15.
    Liu, Y., Yuan, Z., Chen, B., Xue, J., Zheng, N.: Illumination robust color naming via label propagation. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 621–629. IEEE, Santiago, December 2015Google Scholar
  16. 16.
    Luo, P., Wang, X., Tang, X.: Pedestrian parsing via deep decompositional network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2648–2655 (2013)Google Scholar
  17. 17.
    Luo, Y., Zheng, Z., Zheng, L., Guan, T., Yu, J., Yang, Y.: Macro-micro adversarial network for human parsing, July 2018Google Scholar
  18. 18.
    Mojsilovic, A.: A computational model for color naming and describing color composition of images. IEEE Trans. Image Process. 14(5), 690–699 (2005)CrossRefGoogle Scholar
  19. 19.
    Munroe, R.: Color Survey Results. xkcd, May 2010.
  20. 20.
    Parthasarathy, S., Sankaran, P.: An automated multi Scale Retinex with Color Restoration for image enhancement. In: 2012 National Conference on Communications (NCC), pp. 1–5 (2012)Google Scholar
  21. 21.
    Petro, A.B., Sbert, C., Morel, J.M.: Multiscale Retinex. Image Processing On Line, pp. 71–88 (2014)Google Scholar
  22. 22.
    Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets, p. 12, May 2000.
  23. 23.
    Szeliski, R.: Computer Vision: Algorithms and Applications, p. 979, September 2010.
  24. 24.
    van de Weijer, J., Schmid, C., Verbeek, J., Larlus, D.: Learning color names for real-world applications. IEEE Trans. Image Process. 18(7), 1512–1523 (2009)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: Proceedings of the IEEE International Conference on Computer Vision (2015)Google Scholar
  26. 26.
    Zheng, Z., Zheng, L., Yang, Y.: Unlabeled samples generated by GAN improve the person re-identification baseline in vitro. arXiv preprint arXiv:1701.07717 (2017)

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.LITIV Lab.Polytechnique MontréalMontréalCanada
  2. 2.Arcturus NetworksEtobicokeCanada

Personalised recommendations