Skip to main content
SpringerLink
Log in

Person re-identification with features-based clustering and deep features

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Person re-identification (ReID) is an imperative area of pedestrian analysis and has practical applications in visual surveillance. In the person ReID, the robust feature representation is a key issue because of inconsistent visual appearances of a person. Also, an exhaustive gallery search is required to find the target image against each probe image. To answer such issues, this manuscript presents a framework named features-based clustering and deep features in person ReID. The proposed framework initially extracts three types of handcrafted features on the input images including shape, color, and texture for feature representation. To acquire optimal features, a feature fusion and selection technique is applied to these handcrafted features. Afterward, to optimize the gallery search, features-based clustering is performed for splitting the whole gallery into \(k\) consensus clusters. For relationship learning of gallery features and related labels of the chosen clusters, radial basis kernel is employed. Later on, cluster-wise, images are selected and provided to the deep convolution neural network model to obtain deep features. Then, a cluster-wise feature vector is obtained by fusing the deep and handcrafted features. It follows the feature matching process where multi-class support vector machine is applied to choose the related cluster. Finally, to find accurate matching pair from the classified cluster(s) instead of the whole gallery search, a cross-bin histogram-based distance similarity measure is used. The recognition rate at rank 1 is attained as 46.82%, 48.12%, and 40.67% on selected datasets VIPeR, CUHK01, and iLIDS-VID, respectively. It confirms the proposed framework outperforms the existing ReID approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Vezzani R, Baltieri D, Cucchiara R (2013) People reidentification in surveillance and forensics: a survey. ACM Comput Surv (CSUR) 46:29

    Google Scholar 

  2. Fang Y, Ding G, Yuan Y, Lin W, Liu H (2018) Robustness analysis of pedestrian detectors for surveillance. IEEE Access 6:28890

    Google Scholar 

  3. Bedagkar-Gala A, Shah SK (2014) A survey of approaches and trends in person re-identification. Image Vis Comput 32:270–286

    Google Scholar 

  4. An L, Chen X, Liu S, Lei Y, Yang S (2017) Integrating appearance features and soft biometrics for person re-identification. Multimed Tools Appl 76:12117–12131

    Google Scholar 

  5. Fendri E, Frikha M, Hammami M (2017) Multi-level semantic appearance representation for person re-identification system. Pattern Recognit Lett 115:30–38

    Google Scholar 

  6. Li S-M, Gao C, Zhu J-G, Li C-W (2018) Person reidentification using attribute-restricted projection metric learning. IEEE Trans Circuits Syst Video Technol 28:1765–1776

    Google Scholar 

  7. Zhao C, Wang X, Wong WK, Zheng W, Yang J, Miao D (2017) Multiple metric learning based on bar-shape descriptor for person re-identification. Pattern Recognit 71:218–234

    Google Scholar 

  8. Leng Q (2018) Co-metric learning for person re-identification. Adv Multimed 2018

  9. Chahla C, Snoussi H, Abdallah F, Dornaika F (2017) Discriminant quaternion local binary pattern embedding for person re-identification through prototype formation and color categorization. Eng Appl Artif Intell 58:27–33

    Google Scholar 

  10. Zhao R, Ouyang W, Wang X (2013) Unsupervised salience learning for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3586–3593

  11. Zhang Y, Li S (2011) Gabor-LBP based region covariance descriptor for person re-identification. In: 2011 sixth international conference on image and graphics (ICIG), pp 368–371

  12. Liao S, Hu Y, Zhu X, Li SZ (2015) Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2197–2206

  13. Lisanti G, Masi I, Bagdanov AD, Del Bimbo A (2015) Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intell 37:1629–1642

    Google Scholar 

  14. Matsukawa T, Okabe T, Suzuki E, Sato Y (2016) Hierarchical gaussian descriptor for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1363–1372

  15. Wang X, Zhao C, Miao D, Wei Z, Zhang R, Ye T (2016) Fusion of multiple channel features for person re-identification. Neurocomputing 213:125–136

    Google Scholar 

  16. An L, Chen X, Yang S (2017) Multi-graph feature level fusion for person re-identification. Neurocomputing 259:39–45

    Google Scholar 

  17. An L, Chen X, Yang S (2016) Person re-identification via hypergraph-based matching. Neurocomputing 182:247–254

    Google Scholar 

  18. Zhao R, Ouyang W, Wang X (2013) Person re-identification by salience matching. In: Proceedings of the IEEE international conference on computer vision, pp 2528–2535

  19. Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: 2010 IEEE conference on Computer vision and pattern recognition (CVPR), pp 2360–2367

  20. Shen Y, Lin W, Yan J, Xu M, Wu J, Wang J (2015) Person re-identification with correspondence structure learning. In: Proceedings of the IEEE international conference on computer vision, pp 3200–3208

  21. Lin W, Shen Y, Yan J, Xu M, Wu J, Wang J et al (2017) Learning correspondence structures for person re-identification. IEEE Trans Image Process 26:2438–2453

    MathSciNet  MATH  Google Scholar 

  22. Hirzer M, Roth PM, Köstinger M, Bischof H (2012) Relaxed pairwise learned metric for person re-identification. In: European conference on computer vision, pp 780–793

  23. PM Roth, M Hirzer, M Koestinger, C Beleznai, and H Bischof (2014) Mahalanobis distance learning for person re-identification. In: Person re-identification, Springer, pp 247–267

  24. Kuo C-H, Khamis S, Shet V (2013) Person re-identification using semantic color names and rankboost. In: 2013 IEEE workshop on applications of computer vision (WACV), pp 281–287

  25. Zhang L, Li K, Zhang Y, Qi Y, Yang L (2017) Adaptive image segmentation based on color clustering for person re-identification. Soft Comput 21:5729–5739

    Google Scholar 

  26. Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recognit 48:2993–3003

    Google Scholar 

  27. Sun L, Zhao C, Yan Z, Liu P, Duckett T, Stolkin R (2018) A novel weakly-supervised approach for RGB-d-based nuclear waste object detection and categorization. IEEE Sens J 19:3487–3500

    Google Scholar 

  28. Xie Y, Zhang J, Xia Y, Fulham M, Zhang Y (2018) Fusing texture, shape and deep model-learned information at decision level for automated classification of lung nodules on chest CT. Inf Fusion 42:102–110

    Google Scholar 

  29. Zhang Z, Si T (2018) Learning deep features from body and parts for person re-identification in camera networks. EURASIP J Wirel Commun Netw 2018:52

    Google Scholar 

  30. Chen Y, Zhu X, Gong S (2018) Person re-identification by deep learning multi-scale representations

  31. Wu S, Chen Y-C, Li X, Wu A-C, You J-J, Zheng W-S (2016) An enhanced deep feature representation for person re-identification. In: 2016 IEEE winter conference on applications of computer vision (WACV), pp 1–8

  32. Yang X, Chen P (2019) Person re-identification based on multi-scale convolutional network. Multimed Tools Appl, pp 1–15

  33. Nie J, Huang L, Zhang W, Wei G, Wei Z (2019) Deep feature ranking for person re-identification. IEEE Access 7:15007–15017

    Google Scholar 

  34. Ahmed E, Jones M, Marks TK (2015)An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3908–3916

  35. Zhang Z, Huang M (2018) Learning local embedding deep features for person re-identification in camera networks. EURASIP J Wirel Commun Netw 2018:85

    Google Scholar 

  36. Wu D, Zheng S-J, Bao W-Z, Zhang X-P, Yuan C-A, Huang D-S (2019) A novel deep model with multi-loss and efficient training for person re-identification. Neurocomputing 324:69–75

    Google Scholar 

  37. Nanni L, Ghidoni S, Brahnam S (2017) Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognit 71:158–172

    Google Scholar 

  38. Nanda A, Sa PK, Choudhury SK, Bakshi S, Majhi B (2017) A neuromorphic person re-identification framework for video surveillance. IEEE Access 5:6471–6482

    Google Scholar 

  39. Li T, Sun L, Han C, Guo J (2018) Salient region-based least-squares log-density gradient clustering for image-to-video person re-identification. IEEE Access 6:8638–8648

    Google Scholar 

  40. Xin X, Wang J, Xie R, Zhou S, Huang W, Zheng N (2019) Semi-supervised person Re-Identification using multi-view clustering. Pattern Recognit 88:285–297

    Google Scholar 

  41. Shah JH, Lin M, Chen Z (2016) Multi-camera handoff for person re-identification. Neurocomputing 191:238–248

    Google Scholar 

  42. Chu H, Qi M, Liu H, Jiang J (2017) Local region partition for person re-identification. Multimed Tools Appl, pp 1–17

  43. Nanda A, Sa PK, Chauhan DS, Majhi B (2019) A person re-identification framework by inlier-set group modeling for video surveillance. J Ambient Intell Humaniz Comput 10:13–25

    Google Scholar 

  44. Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. Comput Vis–ECCV 2008, pp 262–275

  45. Ye X, Zhou W-Y, Dong L-A (2019) Body part-based person re-identification integrating semantic attributes. Neural Process Lett 49:1111–1124

    Google Scholar 

  46. Dai J, Zhang Y, Lu H, Wang H (2018) Cross-view semantic projection learning for person re-identification. Pattern Recognit 75:63–76

    Google Scholar 

  47. Ye X, Zhou W-Y, Dong L-A (2018) Body part-based person re-identification integrating semantic attributes. Neural Process Lett 49:1–14

    Google Scholar 

  48. Kviatkovsky I, Adam A, Rivlin E (2013) Color invariants for person reidentification. IEEE Trans Pattern Anal Mach Intell 35:1622–1634

    Google Scholar 

  49. Yang Y, Yang J, Yan J, Liao S, Yi D, Li SZ (2014) Salient color names for person re-identification. In: European conference on computer vision, pp 536–551

  50. Xiong F, Gou M, Camps O, Sznaier M (2014) Person re-identification using kernel-based metric learning methods. In: European conference on computer vision, pp 1–16

  51. Chen Y-C, Zhu X, Zheng W-S, Lai J-H (2018) Person re-identification by camera correlation aware feature augmentation. IEEE Trans Pattern Anal Mach Intell 40:392–408

    Google Scholar 

  52. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  53. Girshick R, Donahue J, Darrell T, Malik J (2016) Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38:142–158

    Google Scholar 

  54. Huang Y, Sheng H, Zheng Y, Xiong Z (2017) DeepDiff: learning deep difference features on human body parts for person re-identification. Neurocomputing 241:191–203

    Google Scholar 

  55. Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3318–3325

  56. Zheng W-S, Gong S, Xiang T (2013) Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intell 35:653–668

    Google Scholar 

  57. Feng G, Liu W, Tao D, Zhou Y (2019) Hessian regularized distance metric learning for people re-identification. Neural Process Lett:1–14

  58. Liu X, Ma X, Wang J, Wang H (2017) M3L: Multi-modality mining for metric learning in person re-Identification. Pattern Recognit 76:650

    Google Scholar 

  59. Li W, Zhao R, Wang X (2012) Human reidentification with transferred metric learning. In: Asian conference on computer vision, pp 31–44

  60. Ni T, Ding Z, Chen F, Wang H (2018) Relative distance metric leaning based on clustering centralization and projection vectors learning for person re-identification. IEEE Access 6:11405–11411

    Google Scholar 

  61. Zhou Q, Zheng S, Ling H, Su H, Wu S (2017) Joint dictionary and metric learning for person re-identification. Pattern Recognit 72:196–206

    Google Scholar 

  62. Schwartz WR, Davis LS (2009) Learning discriminative appearance-based models using partial least squares. In: 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI), pp 322–329

  63. Zheng W-S, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 649–656

  64. Wang T, Gong S, Zhu X, Wang S (2016) Person re-identification by discriminative selection in video ranking. IEEE Trans Pattern Anal Mach Intell 38:2501–2514

    Google Scholar 

  65. Tkalcic M, Tasic JF (2003) Colour spaces: perceptual, historical and applicational background, IEEE, vol 1

  66. Hu A, Zhang R, Yin D, Zhan Y (2014) Image quality assessment using a SVD-based structural projection. Signal Process Image Commun 29:293–302

    Google Scholar 

  67. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24:971–987

    MATH  Google Scholar 

  68. Verma M, Raman B, Murala S (2015) Local extrema co-occurrence pattern for color and texture image retrieval. Neurocomputing 165:255–269

    Google Scholar 

  69. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, pp 886–893

  70. Dalal N, Triggs B, Schmid C (2006) Human detection using oriented histograms of flow and appearance. In: European conference on computer vision, pp 428–441

  71. Zhang S, Wang X (2013) Human detection and object tracking based on Histograms of Oriented Gradients. In: 2013 ninth international conference on natural computation (ICNC), pp 1349–1353

  72. Déniz O, Bueno G, Salido J, De la Torre F (2011) Face recognition using histograms of oriented gradients. Pattern Recognit Lett 32:1598–1603

    Google Scholar 

  73. Dash M, Koot PW (2009) Feature selection for clustering. In: Encyclopedia of database systems, Springer, pp 1119–1125

  74. Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. In: Advances in neural information processing systems, pp 1601–1608

  75. Ben-David S, Pál D, Simon HU (2007) Stability of k-means clustering. In: International conference on computational learning theory, pp 20–34

  76. Zhong C, Yue X, Zhang Z, Lei J (2015) A clustering ensemble: two-level-refined co-association matrix with path-based transformation. Pattern Recognit 48:2699–2709

    MATH  Google Scholar 

  77. Zhang YD, Chen S, Wang SH, Yang JF, Phillips P (2015) Magnetic resonance brain image classification based on weighted-type fractional Fourier transform and nonparallel support vector machine. Int J Imaging Syst Technol 25:317–327

    Google Scholar 

  78. Chen L, Chen CP, Lu M (2011) A multiple-kernel fuzzy c-means algorithm for image segmentation. IEEE Trans Syst Man Cybern Part B (Cybern) 41:1263–1274

    MathSciNet  Google Scholar 

  79. Gan Y (2018) Facial expression recognition using convolutional neural network. In: Proceedings of the 2nd international conference on vision, image and signal processing, p 29

  80. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al. (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  81. Pele O, Werman M (2010) The quadratic-chi histogram distance family. In: European conference on computer vision, pp 749–762

  82. Wang T, Gong S, Zhu X, Wang S (2014) Person re-identification by video ranking. In: European conference on computer vision, pp 688–703

  83. Geng Y, Hu H-M, Zeng G, Zheng J (2015) A person re-identification algorithm by exploiting region-based feature salience. J Vis Commun Image Represent 29:89–102

    Google Scholar 

  84. Liong VE, Lu J, Ge Y (2015) Regularized local metric learning for person re-identification. Pattern Recognit Lett 68:288–296

    Google Scholar 

  85. Bąk S, Carr P (2016) Person re-identification using deformable patch metric learning. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp 1–9

  86. An L, Chen X, Yang S, Li X (2017) Person re-identification by multi-hypergraph fusion. IEEE Trans Neural Netw Learn Syst 28:2763–2774

    MathSciNet  Google Scholar 

  87. An L, Kafai M, Yang S, Bhanu B (2016) Person reidentification with reference descriptor. IEEE Trans Circuits Syst Video Technol 26:776–787

    Google Scholar 

  88. An L, Chen X, Yang S, Bhanu B (2016) Sparse representation matching for person re-identification. Inf Sci 355:74–89

    Google Scholar 

  89. An L, Qin Z, Chen X, Yang S (2018) Multi-level common space learning for person re-identification. IEEE Trans Circuits Syst Video Technol 28:1777–1787

    Google Scholar 

  90. Xie Y, Yu H, Gong X, Levine MD (2017) Adaptive Metric Learning and Probe-Specific Reranking for Person Reidentification. IEEE Signal Process Lett 24:853–857

    Google Scholar 

  91. Li J, Ma AJ, Yuen PC (2018) Semi-supervised region metric learning for person re-identification. Int J Comput Vis 126:1–20

    Google Scholar 

  92. Su C, Zhang S, Yang F, Zhang G, Tian Q, Gao W et al (2017) Attributes driven tracklet-to-tracklet person re-identification using latent prototypes space mapping. Pattern Recognit 66:4–15

    Google Scholar 

  93. Cho Y-J, Yoon K-J (2016) Improving person re-identification via pose-aware multi-shot matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1354–1362

  94. Karanam S, Li Y, Radke RJ (2015) Person re-identification with discriminatively trained viewpoint invariant dictionaries. In: Proceedings of the IEEE international conference on computer vision, pp 4516–4524

  95. Li Y, Wu Z, Karanam S, Radke RJ (2015) Multi-shot human re-identification using adaptive fisher discriminant analysis. In: BMVC, p 2

  96. Jiang Z, Lin Z, Davis LS (2013) Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans Pattern Anal Mach Intell 35:2651–2664

    Google Scholar 

  97. Zhao R, Ouyang W, Wang X (2014) Learning mid-level filters for person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 144–151

  98. Zheng L, Wang S, Tian L, He F, Liu Z, Tian Q (2015) Query-adaptive late fusion for image search and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  99. An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Signal Process Lett 22:1103–1107

    Google Scholar 

  100. Yuan C, Xu C, Wang T, Liu F, Zhao Z, Feng P et al (2018) Deep multi-instance learning for end-to-end person re-identification. Multimed Tools Appl 77:12437–12467

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Wah Cantt, Pakistan

    Muhammad Fayyaz, Mussarat Yasmin, Muhammad Sharif, Jamal Hussain Shah, Mudassar Raza & Tassawar Iqbal

Authors
  1. Muhammad Fayyaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mussarat Yasmin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jamal Hussain Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mudassar Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tassawar Iqbal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jamal Hussain Shah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest. Three publicly available datasets are used in this research work for validation of the proposed framework including VIPeR, CUHK01, and iLIDS-VID.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fayyaz, M., Yasmin, M., Sharif, M. et al. Person re-identification with features-based clustering and deep features. Neural Comput & Applic 32, 10519–10540 (2020). https://doi.org/10.1007/s00521-019-04590-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04590-2

Keywords

Search

Navigation