Skip to main content
SpringerLink
Log in

Identifying Customer Behaviour and Dwell Time Using Soft Biometrics

  • Chapter
Video Analytics for Business Intelligence

Part of the book series: Studies in Computational Intelligence ((SCI,volume 409))

Abstract

In a commercial environment, it is advantageous to know how long it takes customers to move between different regions, how long they spend in each region, and where they are likely to go as they move from one location to another. Presently, these measures can only be determined manually, or through the use of hardware tags (i.e. RFID). Soft biometrics are characteristics that can be used to describe, but not uniquely identify an individual. They include traits such as height, weight, gender, hair, skin and clothing colour. Unlike traditional biometrics, soft biometrics can be acquired by surveillance cameras at range without any user cooperation. While these traits cannot provide robust authentication, they can be used to provide identification at long range, and aid in object tracking and detection in disjoint camera networks. In this chapter we propose using colour, height and luggage soft biometrics to determine operational statistics relating to how people move through a space. A novel average soft biometric is used to locate people who look distinct, and these people are then detected at various locations within a disjoint camera network to gradually obtain operational statistics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ailisto, H., Vildjiounaite, E., Lindholm, M., Makela, S., Peltola, J.: Soft biometrics–combining body weight and fat measurements with fingerprint biometrics. Pattern Recognition Letters 27(5), 325–334 (2006)

    Article  Google Scholar 

  2. Bak, S., Corvee, E., Bremond, F., Thonnat, M.: Person re-identification using haar-based and dcd-based signature. In: 2nd Workshop on Activity Monitoring by Multi-Camera Surveillance Systems, AMMCSS 2010, in Conjunction with 7th IEEE International Conference on Advanced Video and Signal-Based Surveillance, AVSS, AVSS (2010)

    Google Scholar 

  3. Bak, S., Corvee, E., Bremond, F., Thonnat, M.: Person re-identification using spatial covariance regions of human body parts. In: 2010 Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 435–440 (2010)

    Google Scholar 

  4. Bazzani, L., Cristani, M., Perina, A., Farenzena, M., Murino, V.: Multiple-shot person re-identification by hpe signature. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 1413–1416 (2010)

    Google Scholar 

  5. Beymer, D.: Person counting using stereo. In: Workshop on Human Motion, vol. 0, p. 127. IEEE Computer Society, Los Alamitos (2000), http://doi.ieeecomputersociety.org/10.1109/HUMO.2000.897382

    Chapter  Google Scholar 

  6. Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Online multi-person tracking-by-detection from a single, uncalibrated camera. IEEE Transactions on Pattern Analysis and Machine Intelligence 99 (2010)

    Google Scholar 

  7. Chien, S.Y., Chan, W.K., Cherng, D.C., Chang, J.Y.: Human object tracking algorithm with human color structure descriptor for video surveillance systems. In: 2006 IEEE International Conference on Multimedia and Expo, pp. 2097–2100 (2006)

    Google Scholar 

  8. Collins, R., Gross, R., Shi, J.: Silhouette-based human identification from body shape and gait. In: Proceedings of IEEE Conference on Face and Gesture Recognition, pp. 351–356 (2002)

    Google Scholar 

  9. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Schmid, C., Soatto, S., Tomasi, C. (eds.) International Conference on Computer Vision & Pattern Recognition, INRIA Rhône-Alpes, ZIRST-655, av. de l’Europe, Montbonnot-38334, vol. 2, pp. 886–893 (2005)

    Google Scholar 

  10. Damen, D., Hogg, D.C.: Detecting Carried Objects in Short Video Sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 154–167. Springer, Heidelberg (2008), http://dx.doi.org/10.1007/978-3-540-88690-7_12

    Chapter  Google Scholar 

  11. Damen, D., Hogg, D.: Recognizing linked events: Searching the space of feasible explanations. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 927–934 (2009)

    Google Scholar 

  12. Dantcheva, A., Velardo, C., D’Angelo, A., Dugelay, J.L.: Bag of soft biometrics for person identification: New trends and challenges. Multimedia Tools and Applications 51(2), 739–777 (2011)

    Article  Google Scholar 

  13. Demirkus, M., Garg, K., Guler, S.: Automated person categorization for video surveillance using soft biometrics. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 7667, p. 22 (2010)

    Google Scholar 

  14. Demirkus, M., Toews, M., Clark, J.J., Arbel, T.: Gender classification from unconstrained video sequences. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 55–62 (2010)

    Google Scholar 

  15. Denman, S., Fookes, C., Bialkowski, A., Sridharan, S.: Soft-Biometrics: unconstrained authentication in a surveillance environment. In: Digital Image Computing: Techniques and Applications (DICTA), pp. 196–203 (2009)

    Google Scholar 

  16. Denman, S., Fookes, C., Sridharan, S.: Improved simultaneous computation of motion detection and optical flow for object tracking. In: Digital Image Computing: Techniques and Applications (DICTA), Melbourne, Australia (2009)

    Google Scholar 

  17. Denman, S., Fookes, C., Sridharan, S.: Group segmentation during object tracking using optical flow discontinuities. In: The 4th Pacific-Rim Symposium on Image and Video Technology, Singapore, pp. 270–275 (2010)

    Google Scholar 

  18. Denman, S., Fookes, C., Sridharan, S., Ryan, D.: Multi-modal object tracking using dynamic performance metrics. In: 7th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Boston, USA, pp. 286–293 (2010)

    Google Scholar 

  19. Farenzena, M., Bazzani, L., Perina, A., Murino, V., Cristani, M.: Person re-identification by symmetry-driven accumulation of local features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2360–2367 (2010)

    Google Scholar 

  20. Fleuret, F., Berclaz, J., Lengagne, R., Fua, P.A.F.P.: Multicamera people tracking with a probabilistic occupancy map. Transactions on Pattern Analysis and Machine Intelligence 30(2), 267–282, 0162-8828 (2008)

    Article  Google Scholar 

  21. Forssen, P.E.: Maximally stable colour regions for recognition and matching. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8 (June 2007), doi:10.1109/CVPR.2007.383120

    Google Scholar 

  22. Gheissari, N., Sebastian, T.B., Hartley, R.: Person reidentification using spatiotemporal appearance. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1528–1535 (2006)

    Google Scholar 

  23. Gray, D., Tao, H.: Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 262–275. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  24. Gray, D., Brennan, S., Tao, H.: Evaluating appearance models for recognition, acquisition and tracking. In: PETS (2007)

    Google Scholar 

  25. Hahnel, M., Klunder, D., Kraiss, K.: Color and texture features for person recognition. In: IEEE International Joint Conference on Neural Networks, Budapest, Hungary, p. 652 (2004)

    Google Scholar 

  26. Han, J., Bhanu, B.: Individual recognition using gait energy image. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 316–322 (2006)

    Article  Google Scholar 

  27. Haritaoglu, I., Flickner, M.: Attentive billboards: towards to video based customer behavior understanding. In: Proceedings of Sixth IEEE Workshop on Applications of Computer Vision (WACV 2002), pp. 127–131 (2002), doi:10.1109/ACV.2002.1182169

    Google Scholar 

  28. Haritaoglu, I., Cutler, R., Harwood, D., Davis, L.S.: Backpack: Detection of people carrying objects using silhouettes. In: IEEE International Conference on Computer Vision, vol. 1, p. 102 (1999), doi: doi.ieeecomputersociety.org/10.1109/ICCV.1999.791204

    Google Scholar 

  29. Haritaoglu, I., Harwood, D., Davis, L.: W4: real-time surveillance of people and their activities. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8), 809–830 (2000)

    Article  Google Scholar 

  30. Haritaoglu, I., Beymer, D., Flickner, M.: Ghost3D: detecting body posture and parts using stereo. In: Proceedings of Workshop on Motion and Video Computing, pp. 175–180 (2002)

    Google Scholar 

  31. Hu, M., Hu, W., Tan, T.: Tracking people through occlusions. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, pp. 724–727 (2004)

    Google Scholar 

  32. Hu, W., Hu, M., Zhou, X., Tan, T., Lou, J., Maybank, S.: Principal axis-based correspondence between multiple cameras for people tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(4), 663–671 (2006), doi:10.1109/TPAMI.2006.80

    Article  Google Scholar 

  33. Jain, A.K., Dass, S.C., Nandakumar, K.: Soft biometric traits for personal recognition systems. In: International Conference on Biometric Authentication, Hong Kong, pp. 731–738 (2004)

    Google Scholar 

  34. Jojic, N., Frey, B., Kannan, A.: Epitomic analysis of appearance and shape. In: Proceedings of Ninth IEEE International Conference on Computer Vision, vol. 1, pp. 34–41 (2003), doi:10.1109/ICCV.2003.1238311

    Google Scholar 

  35. Kim, B.-S., Lee, G.-G., Yoon, J.-Y., Kim, J.-J., Kim, W.-Y.: A Method of Counting Pedestrians in Crowded Scenes. In: Huang, D.-S., Wunsch II, D.C., Levine, D.S., Jo, K.-H. (eds.) ICIC 2008. LNCS (LNAI), vol. 5227, pp. 1117–1126. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  36. Krahnstoever, N., Rittscher, J., Tu, P., Chean, K., Tomlinson, T.: Activity recognition using visual tracking and RFID. In: IEEE Workshop on Applications of Computer Vision and the IEEE Workshop on Motion and Video Computing, vol. 1, pp. 494–500. IEEE Computer Society, Los Alamitos (2005), doi: doi.ieeecomputersociety.org/10.1109/ACVMOT.2005.17

    Google Scholar 

  37. Lazebnik, S., Schmid, C., Ponce, J.: A sparse texture representation using local affine regions. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 1265–1278 (2005), doi: doi.ieeecomputersociety.org/10.1109/TPAMI.2005.151

    Google Scholar 

  38. Lempitsky, V., Zisserman, A.: Learning to count objects in images. In: Advances in Neural Information Processing Systems (2010)

    Google Scholar 

  39. Lin, Z., Davis, L.S.: Learning pairwise dissimilarity profiles for appearance recognition in visual surveillance. In: ISVC, pp. 23–34 (2008)

    Google Scholar 

  40. Liu, X., Krahnstoever, N., Yu, T., Tu, P.: What are customers looking at? (2007)

    Google Scholar 

  41. Lu, J., Tan, Y.P.: Gait-based human age estimation. In: 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 1718–1721 (2010)

    Google Scholar 

  42. Lu, X., Jain, A.K.: Ethnicity identification from face images. In: Proc. SPIE, vol. 5404, pp. 114–123 (2004)

    Google Scholar 

  43. Macrae, C.N., Bodenhausen, G.V.: Social cognition: Thinking categorically about others. Annual Review of Psychology 51(1), 93–120 (2000)

    Article  Google Scholar 

  44. Marcialis, G.L., Roli, F., Muntoni, D.: Group-specific face verification using soft biometrics. Journal of Visual Languages and Computing 20(2), 101–109 (2009)

    Article  Google Scholar 

  45. Nakajima, C., Pontil, M., Heisele, B., Poggio, T.: Full-body person recognition system. Pattern Recognition 36(9), 1997–2006 (2003)

    Article  MATH  Google Scholar 

  46. Niinuma, K., Unsang, P., Jain, A.K.: Soft biometric traits for continuous user authentication. IEEE Transactions on Information Forensics and Security 5(4), 771–780 (2010)

    Article  Google Scholar 

  47. Park, U., Jain, A.: Face matching and retrieval using soft biometrics. IEEE Transactions on Information Forensics and Security 5(3), 406–415 (2010), doi:10.1109/TIFS.2010.2049842

    Article  Google Scholar 

  48. Park, U., Jain, A., Kitahara, I., Kogure, K., Hagita, N.: Vise: Visual search engine using multiple networked cameras. In: 18th International Conference on Pattern Recognition, ICPR 2006, vol. 3, pp. 1204–1207 (2006), doi:10.1109/ICPR.2006.1176

    Google Scholar 

  49. Popa, M., Rothkrantz, L., Yang, Z., Wiggers, P., Braspenning, R., Shan, C.: Analysis of shopping behavior based on surveillance system. In: 2010 IEEE International Conference on Systems Man and Cybernetics (SMC), pp. 2512–2519. IEEE (2010)

    Google Scholar 

  50. Popa, M., Rothkrantz, L., Wiggers, P.: Products appreciation by facial expressions analysis. In: Proceedings of the 11th International Conference on Computer Systems and Technologies and Workshop for PhD Students in Computing on International Conference on Computer Systems and Technologies, pp. 293–298. ACM (2010)

    Google Scholar 

  51. Prosser, B., Zheng, W.-S., Gong, S., Xiang, T.: Person re-identification by support vector ranking. In: Proceedings of the British Machine Vision Conference, pp. 1–21. BMVA Press (2010), doi:10.5244/C.24.21

    Google Scholar 

  52. Qi, Y., Huang, G.-C., Wang, Y.-H.: Carrying object detection and tracking based on body main axis. In: International Conference on Wavelet Analysis and Pattern Recognition, ICWAPR 2007, vol. 3, pp. 1237–1240 (2007), doi:10.1109/ICWAPR.2007.4421623

    Google Scholar 

  53. Ran, Y., Rosenbush, G., Zheng, Q.: Computational approaches for real-time extraction of soft biometrics. In: IEEE Int. Conf. on Pattern Recognition, pp. 1–4 (2008)

    Google Scholar 

  54. Rodriguez, M., Ali, S., Kanade, T.: Tracking in unstructured crowded scenes. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1389–1396 (2009)

    Google Scholar 

  55. Ryan, D., Denman, S., Fookes, C., Sridharan, S.: Crowd counting using group tracking and local features. In: 7th IEEE International Conference on Advanced Video and Signal-Based Surveillance, Boston, USA, pp. 218–224 (2010)

    Google Scholar 

  56. Samangooei, S., Guo, B., Nixon, M.: The use of semantic human description as a soft biometric. In: 2nd IEEE International Conference on Biometrics: Theory, Applications and Systems, BTAS 2008, pp. 1–7 (2008), doi:10.1109/BTAS.2008.4699354

    Google Scholar 

  57. Schwartz, W.R., Davis, L.S.: Learning discriminative appearance-based models using partial least squares. In: XXII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI), pp. 322–329 (2009)

    Google Scholar 

  58. Senior, A., Brown, L., Hampapur, A., Shu, C., Zhai, Y., Feris, R., Tian, Y., Borger, S., Carlson, C.: Video analytics for retail. In: IEEE Conference on Advanced Video and Signal Based Surveillance, AVSS 2007, pp. 423–428 (2007), doi:10.1109/AVSS.2007.4425348

    Google Scholar 

  59. Shan, C., Gong, S., McOwan, P.W.: Learning gender from human gaits and faces. In: IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS 2007), pp. 505–510 (2007), doi:10.1109/AVSS.2007.4425362

    Google Scholar 

  60. Stauffer, C.: Estimating tracking sources and sinks. In: Event Mining Workshop, Madison, WI (2003)

    Google Scholar 

  61. Tsai, R.Y.: An efficient and accurate camera calibration technique for 3d machine vision. In: IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL, pp. 364–374 (1986)

    Google Scholar 

  62. Vaquero, D.A., Feris, R.S., Tran, D., Brown, L., Hampapur, A., Turk, M.: Attribute-based people search in surveillance environments. In: 2009 Workshop on Applications of Computer Vision (WACV), pp. 1–8 (2009), doi:10.1109/WACV.2009.5403131

    Google Scholar 

  63. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR (2001)

    Google Scholar 

  64. Wang, L., Tan, T., Hu, W., Ning, H.: Automatic gait recognition based on statistical shape analysis. IEEE Transactions on Image Processing 12(9), 1120–1131 (2003)

    Article  MathSciNet  Google Scholar 

  65. Wayman, J.L.: Large-scale civilian biometric Systems-Issues and feasibility. In: Proceedings of Card Tech/Secur Tech ID, vol. 732 (1997)

    Google Scholar 

  66. Xiang-tao, C., Zhi-hui, F., Hui, W., Zhe-qing, L.: Automatic gait recognition using kernel principal component analysis. In: 2010 International Conference on ICBECS, pp. 1–4 (2010)

    Google Scholar 

  67. Xiaogang, W., Doretto, G., Sebastian, T., Rittscher, J., Tu, P.: Shape and appearance context modeling. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8 (2007)

    Google Scholar 

  68. Xu, J., Denman, S., Fookes, C., Sridaharan, S.: Activity modelling in crowded environments: A soft-decision approach. In: International Conference on Digital Image Computing: Techniques and Applications, Noosa Heads, Queensland, Australia (2011)

    Google Scholar 

  69. Yu, S., Tan, T., Huang, K., Jia, K., Wu, X.: A study on Gait-Based gender classification. IEEE Transactions on Image Processing 18(8), 1905–1910 (2009), doi:10.1109/TIP.2009.2020535

    Article  MathSciNet  Google Scholar 

  70. Zhang, E., Zhao, Y., Xiong, W.: Active energy image plus 2dlpp for gait recognition. Signal Processing 90(7), 2295–2302 (2010)

    Article  MATH  Google Scholar 

  71. Zheng, W.-S., Gong, S., Xiang, T.: Associating groups of people. In: BMVC 2009 (2009)

    Google Scholar 

  72. Zheng, W.-S., Gong, S., Xiang, T.: Person re-identification by probabilistic relative distance comparison. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 649–656 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Image and Video Laboratory, Queensland University of Technology, Brisbane, Australia

    Simon Denman, Alina Bialkowski, Clinton Fookes & Sridha Sridharan

Authors
  1. Simon Denman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alina Bialkowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clinton Fookes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sridha Sridharan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon Denman .

Editor information

Editors and Affiliations

  1. Philips Research, High Tech Campus 36, Eindhoven, 5656 AE, Netherlands

    Caifeng Shan

  2. Mitsubishi Electric Research Laboratorie, Broadway 201, Cambridge, 02139, USA

    Fatih Porikli

  3. Dept. Computer Science, Queen Mary University of London, Mile End Rd., London, E1 4NS, United Kingdom

    Tao Xiang

  4. Dept. Computer Science, Queen Mary University of London, Mile End Rd., London, E1 4NS, United Kingdom

    Shaogang Gong

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Berlin Heidelberg

About this chapter

Cite this chapter

Denman, S., Bialkowski, A., Fookes, C., Sridharan, S. (2012). Identifying Customer Behaviour and Dwell Time Using Soft Biometrics. In: Shan, C., Porikli, F., Xiang, T., Gong, S. (eds) Video Analytics for Business Intelligence. Studies in Computational Intelligence, vol 409. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28598-1_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-28598-1_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-28597-4

  • Online ISBN: 978-3-642-28598-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation