Two Stream Deep CNN-RNN Attentive Pooling Architecture for Video-Based Person Re-identification

  • W. AnsarEmail author
  • M. M. Fraz
  • M. Shahzad
  • I. Gohar
  • S. Javed
  • S. K. Jung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11401)


Person re-identification (re-ID), is the task of associating the relationship among the images of a person captured from different cameras with non-overlapping field of view. Fundamental and yet an open issue in re-ID is extraction of powerful features in low resolution surveillance videos. In order to solve this, a novel Two Stream Convolutional Recurrent model with Attentive pooling mechanism is presented for person re-ID in videos. Each stream of the model is a Siamese network which is aimed at extracting and matching most differentiated feature maps. Attentive pooling is used to select most informative video frames. The output of two streams is fused to formulate one combined feature map, which helps to deal with major challenges of re-ID e.g. pose and illumination variation, clutter background and occlusion. The proposed technique is evaluated on three challenging datasets: MARS, PRID-2011 and iLIDS-VID. Experimental evaluation shows that the proposed technique performs better than existing state-of-the-art supervised video based person re-ID models. The implementation is available at


Person re-identification Spatial stream Temporal stream 



This research was supported by development project of leading technology for future vehicle of the business of Daegu metropolitan city (No. 20180910). We are also thankful to NVIDIA Corporation for donating the TitanX GPU which is used in this research.


  1. 1.
    Karanam, S., et al.: A systematic evaluation and benchmark for person re-identification: features, metrics, and datasets. IEEE Trans. Pattern Anal. Mach. Intell. 41(3), 523–536 (2018)CrossRefGoogle Scholar
  2. 2.
    Perwaiz, N., Fraz, M.M., Shahzad, M.: Person re-identification using hybrid representation reinforced by metric learning. IEEE Access 6, 77334–77349 (2018)CrossRefGoogle Scholar
  3. 3.
    Boin, J.-B., Araujo, A., Girod, B.: Recurrent neural networks for person re-identification revisited. arXiv preprint arXiv:1804.03281 (2018)
  4. 4.
    Yu, Z., et al.: Three-stream convolutional networks for video-based person re-identification. arXiv preprint arXiv:1712.01652 (2017)
  5. 5.
    McLaughlin, N., Martinez del Rincon, J., Miller, P.: Recurrent convolutional network for video-based person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  6. 6.
    Hirzer, M., Beleznai, C., Roth, P.M., Bischof, H.: Person re-identification by descriptive and discriminative classification. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 91–102. Springer, Heidelberg (2011). Scholar
  7. 7.
    Zheng, L., et al.: MARS: a video benchmark for large-scale person re-identification. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 868–884. Springer, Cham (2016). Scholar
  8. 8.
    Wang, T., Gong, S., Zhu, X., Wang, S.: Person re-identification by video ranking. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 688–703. Springer, Cham (2014). Scholar
  9. 9.
    Yi, D., et al.: Deep metric learning for person re-identification. In: 22nd International Conference on Pattern Recognition (ICPR). IEEE (2014)Google Scholar
  10. 10.
    Mumtaz, S., et al.: Weighted hybrid features for person re-identification. In: 7th International Conference on Image Processing Theory Tools and Applications, Montreal (2017)Google Scholar
  11. 11.
    Mubariz, N., et al.: Optimization of person re-identification through visual descriptors. In: 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Funchal, Madeira, Portugal, pp. 348–355 (2018)Google Scholar
  12. 12.
    Khurram, I., Fraz, M.M., Shahzad, M.: Detailed sentence generation architecture for image semantics description. In: Bebis, G., et al. (eds.) ISVC 2018. LNCS, vol. 11241, pp. 423–432. Springer, Cham (2018). Scholar
  13. 13.
    Bashir, R.M.S., Shahzad, M., Fraz, M.M.: DUPL-VR: deep unsupervised progressive learning for vehicle re-identification. In: Bebis, G., et al. (eds.) ISVC 2018. LNCS, vol. 11241, pp. 286–295. Springer, Cham (2018). Scholar
  14. 14.
    Xu, S., et al.: Jointly attentive spatial-temporal pooling networks for video-based person re-identification. arXiv preprint arXiv:1708.02286 (2017)
  15. 15.
    Chung, D., Tahboub, K., Delp, E.J.: A two stream siamese convolutional neural network for person re-identification. In: The IEEE International Conference on Computer Vision (ICCV) (2017)Google Scholar
  16. 16.
    Liu, K., et al.: A spatio-temporal appearance representation for viceo-based pedestrian re-identification. In: Proceedings of the IEEE International Conference on Computer Vision (2015)Google Scholar
  17. 17.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision (1981)Google Scholar
  18. 18.
    Karanam, S., Li, Y., Radke, R.J.: Sparse re-id: block sparsity for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2015)Google Scholar
  19. 19.
    Ouyang, D., Zhang, Y., Shao, J.: Video-based person re-identification via spatio-temporal attentional and two-stream fusion convolutional networks. Pattern Recogn. Lett. 117, 153–160 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • W. Ansar
    • 1
    Email author
  • M. M. Fraz
    • 1
    • 2
    • 3
    • 5
  • M. Shahzad
    • 1
    • 5
  • I. Gohar
    • 1
  • S. Javed
    • 2
  • S. K. Jung
    • 4
  1. 1.School of Electrical Engineering and Computer ScienceNational University of Sciences and TechnologyIslamabadPakistan
  2. 2.Department of Computer ScienceUniversity of WarwickCoventryUK
  3. 3.The Alan Turing Institute, British LibraryLondonUK
  4. 4.Kyungpook National UniversityDaeguRepublic of Korea
  5. 5.Deep Learning LaboratoryNational Center of Artificial Intelligence (NCAI)IslamabadPakistan

Personalised recommendations