Advertisement

Person re-identification with features-based clustering and deep features

  • Muhammad Fayyaz
  • Mussarat Yasmin
  • Muhammad Sharif
  • Jamal Hussain ShahEmail author
  • Mudassar Raza
  • Tassawar Iqbal
Original Article
  • 45 Downloads

Abstract

Person re-identification (ReID) is an imperative area of pedestrian analysis and has practical applications in visual surveillance. In the person ReID, the robust feature representation is a key issue because of inconsistent visual appearances of a person. Also, an exhaustive gallery search is required to find the target image against each probe image. To answer such issues, this manuscript presents a framework named features-based clustering and deep features in person ReID. The proposed framework initially extracts three types of handcrafted features on the input images including shape, color, and texture for feature representation. To acquire optimal features, a feature fusion and selection technique is applied to these handcrafted features. Afterward, to optimize the gallery search, features-based clustering is performed for splitting the whole gallery into \(k\) consensus clusters. For relationship learning of gallery features and related labels of the chosen clusters, radial basis kernel is employed. Later on, cluster-wise, images are selected and provided to the deep convolution neural network model to obtain deep features. Then, a cluster-wise feature vector is obtained by fusing the deep and handcrafted features. It follows the feature matching process where multi-class support vector machine is applied to choose the related cluster. Finally, to find accurate matching pair from the classified cluster(s) instead of the whole gallery search, a cross-bin histogram-based distance similarity measure is used. The recognition rate at rank 1 is attained as 46.82%, 48.12%, and 40.67% on selected datasets VIPeR, CUHK01, and iLIDS-VID, respectively. It confirms the proposed framework outperforms the existing ReID approaches.

Keywords

Person ReID Entropy controlled features Features-based clustering Deep learning 

Notes

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest. Three publicly available datasets are used in this research work for validation of the proposed framework including VIPeR, CUHK01, and iLIDS-VID.

References

  1. 1.
    Vezzani R, Baltieri D, Cucchiara R (2013) People reidentification in surveillance and forensics: a survey. ACM Comput Surv (CSUR) 46:29CrossRefGoogle Scholar
  2. 2.
    Fang Y, Ding G, Yuan Y, Lin W, Liu H (2018) Robustness analysis of pedestrian detectors for surveillance. IEEE Access 6:28890CrossRefGoogle Scholar
  3. 3.
    Bedagkar-Gala A, Shah SK (2014) A survey of approaches and trends in person re-identification. Image Vis Comput 32:270–286CrossRefGoogle Scholar
  4. 4.
    An L, Chen X, Liu S, Lei Y, Yang S (2017) Integrating appearance features and soft biometrics for person re-identification. Multimed Tools Appl 76:12117–12131CrossRefGoogle Scholar
  5. 5.
    Fendri E, Frikha M, Hammami M (2017) Multi-level semantic appearance representation for person re-identification system. Pattern Recognit Lett 115:30–38CrossRefGoogle Scholar
  6. 6.
    Li S-M, Gao C, Zhu J-G, Li C-W (2018) Person reidentification using attribute-restricted projection metric learning. IEEE Trans Circuits Syst Video Technol 28:1765–1776CrossRefGoogle Scholar
  7. 7.
    Zhao C, Wang X, Wong WK, Zheng W, Yang J, Miao D (2017) Multiple metric learning based on bar-shape descriptor for person re-identification. Pattern Recognit 71:218–234CrossRefGoogle Scholar
  8. 8.
    Leng Q (2018) Co-metric learning for person re-identification. Adv Multimed 2018Google Scholar
  9. 9.
    Chahla C, Snoussi H, Abdallah F, Dornaika F (2017) Discriminant quaternion local binary pattern embedding for person re-identification through prototype formation and color categorization. Eng Appl Artif Intell 58:27–33CrossRefGoogle Scholar
  10. 10.
    Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3586–3593Google Scholar
  11. 11.
    Zhang Y, Li S (2011) Gabor-LBP based region covariance descriptor for person re-identification. In: 2011 sixth international conference on image and graphics (ICIG), pp 368–371Google Scholar
  12. 12.
    Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2197–2206Google Scholar
  13. 13.
    Lisanti G, Masi I, Bagdanov AD, Del Bimbo A (2015) Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intell 37:1629–1642CrossRefGoogle Scholar
  14. 14.
    Matsukawa T, Okabe T, Suzuki E, Sato Y (2016) Hierarchical gaussian descriptor for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1363–1372Google Scholar
  15. 15.
    Wang X, Zhao C, Miao D, Wei Z, Zhang R, Ye T (2016) Fusion of multiple channel features for person re-identification. Neurocomputing 213:125–136CrossRefGoogle Scholar
  16. 16.
    An L, Chen X, Yang S (2017) Multi-graph feature level fusion for person re-identification. Neurocomputing 259:39–45CrossRefGoogle Scholar
  17. 17.
    An L, Chen X, Yang S (2016) Person re-identification via hypergraph-based matching. Neurocomputing 182:247–254CrossRefGoogle Scholar
  18. 18.
    Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: Proceedings of the IEEE international conference on computer vision, pp 2528–2535Google Scholar
  19. 19.
    Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: 2010 IEEE conference on Computer vision and pattern recognition (CVPR), pp 2360–2367Google Scholar
  20. 20.
    Shen Y, Lin W, Yan J, Xu M, Wu J, Wang J (2015) Person re-identification with correspondence structure learning. In: Proceedings of the IEEE international conference on computer vision, pp 3200–3208Google Scholar
  21. 21.
    Lin W, Shen Y, Yan J, Xu M, Wu J, Wang J et al (2017) Learning correspondence structures for person re-identification. IEEE Trans Image Process 26:2438–2453MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Hirzer M, Roth PM, Köstinger M, Bischof H (2012) Relaxed pairwise learned metric for person re-identification. In: European conference on computer vision, pp 780–793CrossRefGoogle Scholar
  23. 23.
    PM Roth, M Hirzer, M Koestinger, C Beleznai, and H Bischof (2014) Mahalanobis distance learning for person re-identification. In: Person re-identification, Springer, pp 247–267zbMATHCrossRefGoogle Scholar
  24. 24.
    Kuo C-H, Khamis S, Shet V (2013) Person re-identification using semantic color names and rankboost. In: 2013 IEEE workshop on applications of computer vision (WACV), pp 281–287Google Scholar
  25. 25.
    Zhang L, Li K, Zhang Y, Qi Y, Yang L (2017) Adaptive image segmentation based on color clustering for person re-identification. Soft Comput 21:5729–5739CrossRefGoogle Scholar
  26. 26.
    Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit 48:2993–3003CrossRefGoogle Scholar
  27. 27.
    Sun L, Zhao C, Yan Z, Liu P, Duckett T, Stolkin R (2018) A novel weakly-supervised approach for RGB-d-based nuclear waste object detection and categorization. IEEE Sens J 19:3487–3500CrossRefGoogle Scholar
  28. 28.
    Xie Y, Zhang J, Xia Y, Fulham M, Zhang Y (2018) Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT. Inf Fusion 42:102–110CrossRefGoogle Scholar
  29. 29.
    Zhang Z, Si T (2018) Learning deep features from body and parts for person re-identification in camera networks. EURASIP J Wirel Commun Netw 2018:52CrossRefGoogle Scholar
  30. 30.
    Chen Y, Zhu X, Gong S (2018) Person re-identification by deep learning multi-scale representationsGoogle Scholar
  31. 31.
    Wu S, Chen Y-C, Li X, Wu A-C, You J-J, Zheng W-S (2016) An enhanced deep feature representation for person re-identification. In: 2016 IEEE winter conference on applications of computer vision (WACV), pp 1–8Google Scholar
  32. 32.
    Yang X, Chen P (2019) Person re-identification based on multi-scale convolutional network. Multimed Tools Appl, pp 1–15Google Scholar
  33. 33.
    Nie J, Huang L, Zhang W, Wei G, Wei Z (2019) Deep feature ranking for person re-identification. IEEE Access 7:15007–15017CrossRefGoogle Scholar
  34. 34.
    Ahmed E, Jones M, Marks TK (2015)An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3908–3916Google Scholar
  35. 35.
    Zhang Z, Huang M (2018) Learning local embedding deep features for person re-identification in camera networks. EURASIP J Wirel Commun Netw 2018:85CrossRefGoogle Scholar
  36. 36.
    Wu D, Zheng S-J, Bao W-Z, Zhang X-P, Yuan C-A, Huang D-S (2019) A novel deep model with multi-loss and efficient training for person re-identification. Neurocomputing 324:69–75CrossRefGoogle Scholar
  37. 37.
    Nanni L, Ghidoni S, Brahnam S (2017) Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognit 71:158–172CrossRefGoogle Scholar
  38. 38.
    Nanda A, Sa PK, Choudhury SK, Bakshi S, Majhi B (2017) A neuromorphic person re-identification framework for video surveillance. IEEE Access 5:6471–6482Google Scholar
  39. 39.
    Li T, Sun L, Han C, Guo J (2018) Salient region-based least-squares log-density gradient clustering for image-to-video person re-identification. IEEE Access 6:8638–8648CrossRefGoogle Scholar
  40. 40.
    Xin X, Wang J, Xie R, Zhou S, Huang W, Zheng N (2019) Semi-supervised person Re-Identification using multi-view clustering. Pattern Recognit 88:285–297CrossRefGoogle Scholar
  41. 41.
    Shah JH, Lin M, Chen Z (2016) Multi-camera handoff for person re-identification. Neurocomputing 191:238–248CrossRefGoogle Scholar
  42. 42.
    Chu H, Qi M, Liu H, Jiang J (2017) Local region partition for person re-identification. Multimed Tools Appl, pp 1–17Google Scholar
  43. 43.
    Nanda A, Sa PK, Chauhan DS, Majhi B (2019) A person re-identification framework by inlier-set group modeling for video surveillance. J Ambient Intell Humaniz Comput 10:13–25CrossRefGoogle Scholar
  44. 44.
    Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. Comput Vis–ECCV 2008, pp 262–275Google Scholar
  45. 45.
    Ye X, Zhou W-Y, Dong L-A (2019) Body part-based person re-identification integrating semantic attributes. Neural Process Lett 49:1111–1124CrossRefGoogle Scholar
  46. 46.
    Dai J, Zhang Y, Lu H, Wang H (2018) Cross-view semantic projection learning for person re-identification. Pattern Recognit 75:63–76CrossRefGoogle Scholar
  47. 47.
    Ye X, Zhou W-Y, Dong L-A (2018) Body part-based person re-identification integrating semantic attributes. Neural Process Lett 49:1–14CrossRefGoogle Scholar
  48. 48.
    Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. IEEE Trans Pattern Anal Mach Intell 35:1622–1634CrossRefGoogle Scholar
  49. 49.
    Yang Y, Yang J, Yan J, Liao S, Yi D, Li SZ (2014) Salient color names for person re-identification. In: European conference on computer vision, pp 536–551Google Scholar
  50. 50.
    Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: European conference on computer vision, pp 1–16Google Scholar
  51. 51.
    Chen Y-C, Zhu X, Zheng W-S, Lai J-H (2018) Person re-identification by camera correlation aware feature augmentation. IEEE Trans Pattern Anal Mach Intell 40:392–408CrossRefGoogle Scholar
  52. 52.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  53. 53.
    Girshick R, Donahue J, Darrell T, Malik J (2016) Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38:142–158CrossRefGoogle Scholar
  54. 54.
    Huang Y, Sheng H, Zheng Y, Xiong Z (2017) DeepDiff: learning deep difference features on human body parts for person re-identification. Neurocomputing 241:191–203CrossRefGoogle Scholar
  55. 55.
    Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3318–3325Google Scholar
  56. 56.
    Zheng W-S, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35:653–668CrossRefGoogle Scholar
  57. 57.
    Feng G, Liu W, Tao D, Zhou Y (2019) Hessian regularized distance metric learning for people re-identification. Neural Process Lett:1–14Google Scholar
  58. 58.
    Liu X, Ma X, Wang J, Wang H (2017) M3L: Multi-modality mining for metric learning in person re-Identification. Pattern Recognit 76:650CrossRefGoogle Scholar
  59. 59.
    Li W, Zhao R, Wang X (2012) Human reidentification with transferred metric learning. In: Asian conference on computer vision, pp 31–44Google Scholar
  60. 60.
    Ni T, Ding Z, Chen F, Wang H (2018) Relative distance metric leaning based on clustering centralization and projection vectors learning for person re-identification. IEEE Access 6:11405–11411CrossRefGoogle Scholar
  61. 61.
    Zhou Q, Zheng S, Ling H, Su H, Wu S (2017) Joint dictionary and metric learning for person re-identification. Pattern Recognit 72:196–206CrossRefGoogle Scholar
  62. 62.
    Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI), pp 322–329Google Scholar
  63. 63.
    Zheng W-S, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 649–656Google Scholar
  64. 64.
    Wang T, Gong S, Zhu X, Wang S (2016) Person re-identification by discriminative selection in video ranking. IEEE Trans Pattern Anal Mach Intell 38:2501–2514CrossRefGoogle Scholar
  65. 65.
    Tkalcic M, Tasic JF (2003) Colour spaces: perceptual, historical and applicational background, IEEE, vol 1Google Scholar
  66. 66.
    Hu A, Zhang R, Yin D, Zhan Y (2014) Image quality assessment using a SVD-based structural projection. Signal Process Image Commun 29:293–302CrossRefGoogle Scholar
  67. 67.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987zbMATHCrossRefGoogle Scholar
  68. 68.
    Verma M, Raman B, Murala S (2015) Local extrema co-occurrence pattern for color and texture image retrieval. Neurocomputing 165:255–269CrossRefGoogle Scholar
  69. 69.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, pp 886–893Google Scholar
  70. 70.
    Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, pp 428–441CrossRefGoogle Scholar
  71. 71.
    Zhang S, Wang X (2013) Human detection and object tracking based on Histograms of Oriented Gradients. In: 2013 ninth international conference on natural computation (ICNC), pp 1349–1353Google Scholar
  72. 72.
    Déniz O, Bueno G, Salido J, De la Torre F (2011) Face recognition using histograms of oriented gradients. Pattern Recognit Lett 32:1598–1603CrossRefGoogle Scholar
  73. 73.
    Dash M, Koot PW (2009) Feature selection for clustering. In: Encyclopedia of database systems, Springer, pp 1119–1125Google Scholar
  74. 74.
    Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. In: Advances in neural information processing systems, pp 1601–1608Google Scholar
  75. 75.
    Ben-David S, Pál D, Simon HU (2007) Stability of k-means clustering. In: International conference on computational learning theory, pp 20–34Google Scholar
  76. 76.
    Zhong C, Yue X, Zhang Z, Lei J (2015) A clustering ensemble: two-level-refined co-association matrix with path-based transformation. Pattern Recognit 48:2699–2709zbMATHCrossRefGoogle Scholar
  77. 77.
    Zhang YD, Chen S, Wang SH, Yang JF, Phillips P (2015) Magnetic resonance brain image classification based on weighted-type fractional Fourier transform and nonparallel support vector machine. Int J Imaging Syst Technol 25:317–327CrossRefGoogle Scholar
  78. 78.
    Chen L, Chen CP, Lu M (2011) A multiple-kernel fuzzy c-means algorithm for image segmentation. IEEE Trans Syst Man Cybern Part B (Cybern) 41:1263–1274CrossRefGoogle Scholar
  79. 79.
    Gan Y (2018) Facial expression recognition using convolutional neural network. In: Proceedings of the 2nd international conference on vision, image and signal processing, p 29Google Scholar
  80. 80.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  81. 81.
    Pele O, Werman M (2010) The quadratic-chi histogram distance family. In: European conference on computer vision, pp 749–762CrossRefGoogle Scholar
  82. 82.
    Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: European conference on computer vision, pp 688–703CrossRefGoogle Scholar
  83. 83.
    Geng Y, Hu H-M, Zeng G, Zheng J (2015) A person re-identification algorithm by exploiting region-based feature salience. J Vis Commun Image Represent 29:89–102CrossRefGoogle Scholar
  84. 84.
    Liong VE, Lu J, Ge Y (2015) Regularized local metric learning for person re-identification. Pattern Recognit Lett 68:288–296CrossRefGoogle Scholar
  85. 85.
    Bąk S, Carr P (2016) Person re-identification using deformable patch metric learning. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 1–9Google Scholar
  86. 86.
    An L, Chen X, Yang S, Li X (2017) Person re-identification by multi-hypergraph fusion. IEEE Trans Neural Netw Learn Syst 28:2763–2774MathSciNetCrossRefGoogle Scholar
  87. 87.
    An L, Kafai M, Yang S, Bhanu B (2016) Person reidentification with reference descriptor. IEEE Trans Circuits Syst Video Technol 26:776–787CrossRefGoogle Scholar
  88. 88.
    An L, Chen X, Yang S, Bhanu B (2016) Sparse representation matching for person re-identification. Inf Sci 355:74–89CrossRefGoogle Scholar
  89. 89.
    An L, Qin Z, Chen X, Yang S (2018) Multi-level common space learning for person re-identification. IEEE Trans Circuits Syst Video Technol 28:1777–1787CrossRefGoogle Scholar
  90. 90.
    Xie Y, Yu H, Gong X, Levine MD (2017) Adaptive Metric Learning and Probe-Specific Reranking for Person Reidentification. IEEE Signal Process Lett 24:853–857CrossRefGoogle Scholar
  91. 91.
    Li J, Ma AJ, Yuen PC (2018) Semi-supervised region metric learning for person re-identification. Int J Comput Vis 126:1–20CrossRefGoogle Scholar
  92. 92.
    Su C, Zhang S, Yang F, Zhang G, Tian Q, Gao W et al (2017) Attributes driven tracklet-to-tracklet person re-identification using latent prototypes space mapping. Pattern Recognit 66:4–15CrossRefGoogle Scholar
  93. 93.
    Cho Y-J, Yoon K-J (2016) Improving person re-identification via pose-aware multi-shot matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1354–1362Google Scholar
  94. 94.
    Karanam S, Li Y, Radke RJ (2015) Person re-identification with discriminatively trained viewpoint invariant dictionaries. In: Proceedings of the IEEE international conference on computer vision, pp 4516–4524Google Scholar
  95. 95.
    Li Y, Wu Z, Karanam S, Radke RJ (2015) Multi-shot human re-identification using adaptive fisher discriminant analysis. In: BMVC, p 2Google Scholar
  96. 96.
    Jiang Z, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35:2651–2664CrossRefGoogle Scholar
  97. 97.
    Zhao R, Ouyang W, Wang X (2014) Learning mid-level filters for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 144–151Google Scholar
  98. 98.
    Zheng L, Wang S, Tian L, He F, Liu Z, Tian Q (2015) Query-adaptive late fusion for image search and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750Google Scholar
  99. 99.
    An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Signal Process Lett 22:1103–1107CrossRefGoogle Scholar
  100. 100.
    Yuan C, Xu C, Wang T, Liu F, Zhao Z, Feng P et al (2018) Deep multi-instance learning for end-to-end person re-identification. Multimed Tools Appl 77:12437–12467CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceCOMSATS University Islamabad, Wah CampusWah CanttPakistan

Personalised recommendations