Abstract
In the recent years, Closed Circuit Television (CCTV) is viewed as the basis for providing security. One of the most important aspects of CCTV surveillance systems security mechanism is to re-identify a person captured in one of the camera across different surveillance cameras. Re-identification has a major role in several applications like automated surveillance of universities, offices, malls, home and restricted environments like embassies or laboratories with strong security restrictions. Traditionally, identifying a person in a video was practiced under the set of same external conditions (like same illumination, viewpoint, back ground conditions etc.). But when it comes to automated re-identification in a CCTV surveillance system, several challenges emerge as the environment is uncontrolled and keeps varying, further the poses of the person and the angles of the cameras capturing the videos also incur additional challenge for the task considered. When a person disappears from one camera view for a period of time, he should be recognized in another view of camera at a different location when there are environmental disturbances like variation in illumination, crowded scene, partial occlusions, physical appearance variations, full occlusions, view point variations, background clutter, shadows and reflections, etc. In this chapter, the major focus is on the techniques of deep learning used to develop an end-to-end re-identification system highlighting the methods to handle the uncontrolled environment challenges mentioned. An end-to-end re-identification task consists of sequence of steps namely pedestrian detection, person tracking followed by person re-identification. Given a video sequence or an image as an input, firstly the humans are detected from the video sequence as a process of pedestrian detection. The person tracking within the camera is conducted, to find the different poses of the probe if needed. Then the re-identification process is conducted where the deep learning models are used to re-identify the person with the help of gallery set of videos and evaluates the similarities of gallery set and the person of interest by using deep learning metrics. The re-identification results end as a retrieval process where all similar images of the person of interest are retrieved. Several bench mark datasets considered in literature for re-identification system are VIPeR, ETHZ, PRID, CAVIAR, CUHK01, CUHK02, CUHK03, i-LIDS, RAiD, MARS, etc.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Bedagkar-Gala, A., Shah, S.K.: A survey of approaches and trends in person re-identification. Image Vis. Comput. 32(4), 270–286 (2014)
Zheng, L., Yang, Y., Hauptmann, A. G.: Person re-identification: Past, present and future (2016). arXiv preprint arXiv:1610.02984
Saghafi, M.A., Hussain, A., Zaman, H.B., Saad, M.H.M.: Review of person re-entification techniques. IET Comput. Vision 8(6), 455–474 (2014)
Zajdel, W., Zivkovic, Z., Krose, B.J.A.: Keeping track of humans: have I seen this person before? In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005. ICRA 2005, pp. 2081–2086. IEEE (2005)
Gheissari, N., Sebastian, T.B., Hartley, R.: Person reidentification using spatiotemporal appearance. In: Null, pp. 1528–1535. IEEE (2006)
Bazzani, L., Cristani, M., Perina, A., Farenzena, M., Murino, V.: Multiple-shot person re-identification by hpe signature. In: 20th International Conference on Pattern Recognition (ICPR), 2010, pp. 1413–1416. IEEE (2010)
Yi, D., Lei, Z., Liao, S., Li, S.Z.: Deep metric learning for person re-identification. In: Pattern Recognition (ICPR), 2014 22nd International Conference on, pp. 34–39. IEEE (2014)
Li, W., Zhao, R., Xiao, T., Wang, X.: Deepreid: deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 152–159 (2014)
Xu, Y., Ma, B., Huang, R., Lin, L.: Person search in a scene by jointly modeling people commonness and person uniqueness. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 937–940. ACM (2014)
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: ISCAS Vol. 2010, pp. 253–256 (2010)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Charu, C.A.: Neural Networks and Deep Learning: A Textbook. Springer (2019)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: European Conference on Computer Vision, pp. 818–833. Springer, Cham (2014)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint arXiv:1409.1556
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D.,… Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Dauphin, Y.N., de Vries, H., Chung, J., Bengio, Y.: RMSProp and equilibrated adaptive learning rates for non-convex optimization. CoRR arXiv:1502.04390 (2015)
He, K., Zhang, X., RenSchustera, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., … Berg, A.C.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016)
Graves, A.: Supervised sequence labelling. In: Supervised Sequence Labelling with Recurrent Neural Networks, pp. 5–13. Springer, Berlin, Heidelberg (2012)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. (1994)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Shanmugamani, R.: Deep Learning for Computer Vision: Expert techniques to train advanced neural networks using TensorFlow and Keras. Packt Publishing Ltd (2018)
Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks (2013). arXiv preprint arXiv:1302.4389
Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus) (2015). arXiv preprint arXiv:1511.07289
Farenzena, M., Bazzani, L., Perina, A., Murino, V., Cristani, M.: Person re-identification by symmetry-driven accumulation of local features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2360–2367. IEEE (2010)
Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: Null, pp. 1735–1742. IEEE (2006)
McLaughlin, N., Martinez del Rincon, J., Miller, P.: Recurrent convolutional network for video-based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1325–1334 (2016)
Chung, D., Tahboub, K., Delp, E.J.: A two stream siamese convolutional neural network for person re-identification. In: The IEEE International Conference on Computer Vision (ICCV) (2017)
Wu, L., Wang, Y., Li, X., Gao, J.: What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recogn. 76, 727–738 (2018)
Gray, D., Tao, H.: Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: European Conference on Computer Vision, pp. 262–275. Springer, Berlin, Heidelberg (2008)
Ess, A., Leibe, B., Van Gool, L.: Depth and appearance for mobile scene analysis. In: 11th International Conference on Computer Vision, 2007. ICCV 2007. IEEE, pp. 1–8. IEEE (2007)
Loy, C.C., Xiang, T., Gong, S.: Time-delayed correlation analysis for multi-camera activity understanding. Int. J. Comput. Vision 90(1), 106–129 (2010)
Zheng, W., Gong, S., Xiang., T.: Associating groups of people. In: BMVC (2009)
Baltieri, D., Vezzani, R., Cucchiara, R.: 3dpes: 3d people dataset for surveillance and forensics. In: Proceedings of the 2011 Joint ACM Workshop on Human Gesture and Behavior Understanding, pp. 59–64. ACM (2011)
Cheng, D.S., Cristani, M., Stoppa, M., Bazzani, L., Murino, V.: Custom pictorial structures for re-identification. In: Bmvc, Vol. 1, No. 2, p. 6 (2011)
Hirzer, M., Beleznai, C., Roth, P. M., Bischof, H.: Person re-identification by descriptive and discriminative classification. In: Scandinavian Conference on Image Analysis, pp. 91–102. Springer, Berlin, Heidelberg (2011)
Bialkowski, A., Denman, S., Sridharan, S., Fookes, C., Lucey, P.: A database for person re-identification in multi-camera surveillance networks. In: 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA), pp. 1–8. IEEE (2012)
Li, W., Zhao, R., Wang, X.: Human reidentification with transferred metric learning. In: Asian Conference on Computer Vision, pp. 31–44. Springer, Berlin, Heidelberg (2012)
Martinel, N., Micheloni, C.: Re-identify people in wide area camera network. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2012, pp. 31–36. IEEE (2012)
Li, W., Wang, X.: Locally aligned feature transforms across views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3594–3601 (2013)
Wang, T., Gong, S., Zhu, X., Wang, S.: Person re-identification by video ranking. In: European Conference on Computer Vision, pp. 688–703. Springer, Cham (2014)
Branch, H.O.S.D.: Imagery library for intelligent detection systems (i-lids). In: The Institution of Engineering and Technology Conference on Crime and Security, pp. 445–448 (2006)
Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)
Das, A., Chakraborty, A., Roy-Chowdhury, A.K.: Consistent re-identification in a camera network. In: European Conference on Computer Vision, pp. 330–345. Springer, Cham (2014)
Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1116–1124 (2015))
Ma, L., Liu, H., Hu, L., Wang, C., Sun, Q.: Orientation driven bag of appearances for person re-identification (2016). arXiv preprint arXiv:1605.02464
Zheng, L., Bie, Z., Sun, Y., Wang, J., Su, C., Wang, S., Tian, Q.: Mars: a video benchmark for large-scale person re-identification. In: European Conference on Computer Vision, pp. 868–884. Springer, Cham (2016)
Zheng, L., Zhang, H., Sun, S., Chandraker, M., Yang, Y., Tian, Q.: Person re-identification in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1367–1376(2017)
Xiao, T., Li, S., Wang, B., Lin, L., Wang, X.: End-to-end deep learning for person search (2016). arXiv preprint arXiv:1604.01850, 1(2)
Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp. 17–35. Springer, Cham (2016)
Camps, O., Gou, M., Hebble, T., Karanam, S., Lehmann, O., Li, Y.,… Xiong, F.: From the lab to the real world: Re-identification in an airport camera network. IEEE Trans. Circuits Syst. Video Technol. 27(3), 540–553 (2017)
Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer gan to bridge domain gap for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 79–88 (2018)
Zheng, M., Karanam, S., Radke, R.J.: RPIfield: a new dataset for temporally evaluating person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1893–1895 (2018)
Zhang, G., Kato, J., Wang, Y., Mase, K.: People re-identification using deep convolutional neural network. In: Computer Vision Theory and Applications (VISAPP), 2014 International Conference on Vol. 3, pp. 216–223. IEEE (2014)
Ahmed, E., Jones, M., Marks, T.K.: An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3908–3916 (2015)
Ding, S., Lin, L., Wang, G., Chao, H.: Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn. 48(10), 2993–3003 (2015)
Zhang, R., Lin, L., Zhang, R., Zuo, W., Zhang, L.: Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans. Image Process. 24(12), 4766–4779 (2015)
Shi, H., Zhu, X., Liao, S., Lei, Z., Yang, Y., Li, S.Z.: Constrained deep metric learning for person re-identification (2015). arXiv preprint arXiv:1511.07545
Iodice, S., Petrosino, A., Ullah, I.: Strict pyramidal deep architectures for person re-identification. In: International Workshop on Neural Networks, pp. 179–186. Springer, Cham (2015)
Cheng, D., Gong, Y., Zhou, S., Wang, J., Zheng, N.: Person re-identification by multi-channel parts-based cnn with improved triplet loss function. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1335–1344 (2016)
Chen, S.Z., Guo, C.C., Lai, J.H.: Deep ranking for person re-identification via joint representation learning. IEEE Trans. Image Process. 25(5), 2353–2367 (2016)
Wu, S., Chen, Y.C., Li, X., Wu, A.C., You, J.J., Zheng, W.S.: An enhanced deep feature representation for person re-identification. In: Applications of Computer Vision (WACV), 2016 IEEE Winter Conference on, pp. 1–8. IEEE (2016)
Xiao, T., Li, H., Ouyang, W., Wang, X.: Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1249–1258 (2016)
Wu, L., Shen, C., Hengel, A.V.D.: Personnet: Person re-identification with deep convolutional neural networks (2016). arXiv preprint arXiv:1601.07255
Li, S., Liu, X., Liu, W., Ma, H., Zhang, H.: A discriminative null space based deep learning approach for person re-identification. In: 4th International Conference on Cloud Computing and Intelligence Systems (CCIS), 2016, pp. 480–484. IEEE (2016)
Shi, H., Yang, Y., Zhu, X., Liao, S., Lei, Z., Zheng, W., Li, S.Z.: Embedding deep metric for person re-identification: A study against large variations. In: European Conference on Computer Vision, pp. 732–748. Springer, Cham (2016)
Varior, R.R., Shuai, B., Lu, J., Xu, D., Wang, G.: A siamese long short-term memory architecture for human re-identification. In: European Conference on Computer Vision, pp. 135–153. Springer, Cham (2016)
Wang, F., Zuo, W., Lin, L., Zhang, D., Zhang, L.: Joint learning of single-image and cross-image representations for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1288–1296 (2016)
Franco, A., Oliveira, L.: A coarse-to-fine deep learning for person re-identification. In: IEEE Winter Conference on Applications of Computer Vision (WACV), 2016, pp. 1–7. IEEE (2016)
McLaughlin, N., del Rincon, J.M., Miller, P.C.: Person reidentification using deep convnets with multitask learning. IEEE Trans. Circuits Syst. Video Techn. 27(3), 525–539 (2017)
Liu, J., Zha, Z.J., Tian, Q.I., Liu, D., Yao, T., Ling, Q., Mei, T.: Multi-scale triplet cnn for person re-identification. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 192–196. ACM (2016)
Wang, J., Wang, Z., Gao, C., Sang, N., Huang, R.: DeepList: learning deep features with adaptive Listwise constraint for person reidentification. IEEE Trans. Circuits Syst. Video Techn. 27(3), 513–524 (2017)
Liu, H., Feng, J., Qi, M., Jiang, J., Yan, S.: End-to-end comparative attention networks for person re-identification. IEEE Trans. Image Process. 26(7), 3492–3506 (2017)
Wu, L., Shen, C., van den Hengel, A.: Deep linear discriminant analysis on fisher networks: A hybrid architecture for person re-identification. Pattern Recogn. 65, 238–250 (2017)
Su, C., Li, J., Zhang, S., Xing, J., Gao, W., Tian, Q.: Pose-driven deep convolutional model for person re-identification. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3980–3989. IEEE (2017)
Franco, A., Oliveira, L.: Convolutional covariance features: Conception, integration and performance in person re-identification. Pattern Recogn. 61, 593–609 (2017)
Qian, X., Fu, Y., Jiang, Y.G., Xiang, T., Xue, X.: Multi-scale deep learning architectures for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5399–5408 (2017)
Zhu, J., Zeng, H., Liao, S., Lei, Z., Cai, C., Zheng, L.: Deep hybrid similarity learning for person re-identification. IEEE Trans. Circuits Syst. Video Technol. 28(11), 3183–3193 (2018)
Cheng, D., Gong, Y., Chang, X., Shi, W., Hauptmann, A., Zheng, N.: Deep Feature Learning via Structured Graph Laplacian Embedding for Person Re-Identification. Pattern Recogn. (2018)
Mao, C., Li, Y., Zhang, Z., Zhang, Y., Li, X.: Pyramid Person Matching Network for Person Re-identification(2018). arXiv preprint arXiv:1803.02547
Li, D., Chen, X., Zhang, Z., Huang, K.: Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 384–393 (2017)
Lin, J., Ren, L., Lu, J., Feng, J., Zhou, J.: Consistent-aware deep learning for person re-identification in a camera network. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 6 (2017)
Bai, X., Yang, M., Huang, T., Dou, Z., Yu, R., Xu, Y.: Deep-Person: Learning Discriminative Deep Features for Person Re-Identification (2017). arXiv preprint arXiv:1711.10658
Chang, Y. S., Wang, M.Y., He, L., Lu, W., Su, H., Gao, N., Yang, X.A.: Joint deep semantic embedding and metric learning for person re-identification. Pattern Recogn. Lett. (2018)
Chen, Y., Duffner, S., Stoian, A., Dufour, J.Y., Baskurt, A.: Deep and low-level feature-based attribute learning for person re-identification. Image Vis. Comput. 79, 25–34 (2018)
Tao, D., Guo, Y., Yu, B., Pang, J., Yu, Z.: Deep multi-view feature learning for person re-identification. IEEE Trans. Circuits Syst. Video Technol. 28(10), 2657–2666 (2018)
Wu, L., Wang, Y., Ge, Z., Hu, Q., Li, X.: Structured deep hashing with convolutional neural networks for fast person re-identification. Comput. Vis. Image Underst. 167, 63–73 (2018)
Su, C., Zhang, S., Xing, J., Gao, W., Tian, Q.: Multi-type attributes driven multi-camera person re-identification. Pattern Recogn. 75, 77–89 (2018)
Wang, J., Zhou, S., Wang, J., Hou, Q.: Deep ranking model by large adaptive margin learning for person re-identification. Pattern Recogn. 74, 241–252 (2018)
Wu, D., Zheng, S.J., Yuan, C.A., Huang, D.S.: A deep model with combined losses for person re-identification. Cogn. Syst. Res. 54, 74–82 (2019)
Zhang, Z., Si, T., Liu, S.: Integration convolutional neural network for person re-identification in camera networks. IEEE Access 6, 36887–36896 (2018)
Liu, Y., Song, N., Han, Y.: Multi-cue fusion: Discriminative enhancing for person re-identification. J. Vis. Commun. Image Represent. 58, 46–52 (2019)
Wang, F., Zhang, C., Chen, S., Ying, G., Lv, J.: Engineering Hand-designed and Deeply-learned features for person Re-identification. Pattern Recogn. Lett. (2018)
Yuan, C., Guo, J., Feng, P., Zhao, Z., Xu, C., Wang, T., … & Duan, K.: A jointly learned deep embedding for person re-identification. Neurocomputing 330, 127–137 (2019)
Xin, X., Wang, J., Xie, R., Zhou, S., Huang, W., Zheng, N.: Semi-supervised person Re-Identification using multi-view clustering. Pattern Recogn. 88, 285–297 (2019)
Wu, D., Zheng, S.J., Bao, W.Z., Zhang, X.P., Yuan, C.A., Huang, D.S.: A novel deep model with multi-loss and efficient training for person re-identification. Neurocomputing 324, 69–75 (2019)
Zhou, S., Ke, M., Luo, P.: Multi-camera transfer GAN for person re-identification. J. Vis. Commun. Image Represent. (2019)
Zhong, W., Jiang, L., Zhang, T., Ji, J., Xiong, H.: Combining multilevel feature extraction and multi-loss learning for person re-identification. Neurocomputing (2019)
Fumera, B.L.G., Roli, F.: Multi-stage ranking approach for fast person re-identification. IET Comput. Vis. 12(4), 513–519 (2018)
Acknowledgements
The authors thank VIT for providing ‘VIT SEED GRANT’ for carrying out this research work. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan Xp GPU used for this research on person Re-identification.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Narayanan, S.J., Perumal, B., Saman, S., Singh, A.P. (2020). Deep Learning for Person Re-identification in Surveillance Videos. In: Pedrycz, W., Chen, SM. (eds) Deep Learning: Algorithms and Applications. Studies in Computational Intelligence, vol 865. Springer, Cham. https://doi.org/10.1007/978-3-030-31760-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-31760-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31759-1
Online ISBN: 978-3-030-31760-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)