Detecting Hands in Egocentric Videos: Towards Action Recognition

  • Alejandro CartasEmail author
  • Mariella Dimiccoli
  • Petia Radeva
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10672)


Recently, there has been a growing interest in analyzing human daily activities from data collected by wearable cameras. Since the hands are involved in a vast set of daily tasks, detecting hands in egocentric images is an important step towards the recognition of a variety of egocentric actions. However, besides extreme illumination changes in egocentric images, hand detection is not a trivial task because of the intrinsic large variability of hand appearance. We propose a hand detector that exploits skin modeling for fast hand proposal generation and Convolutional Neural Networks for hand recognition. We tested our method on UNIGE-HANDS dataset and we showed that the proposed approach achieves competitive hand detection results.


Ego-centric vision First person vision Hand-detection 



A.C. was supported by a doctoral fellowship from the Mexican Council of Science and Technology (CONACYT) (grant-no. 366596). This work was partially founded by TIN2015-66951-C2, SGR 1219, CERCA, ICREA Academia’2014 and 20141510 (Marató TV3). M.D. is grateful to the NVIDIA donation program for its support with a GPU card.


  1. 1.
    Bolaños, M., Dimiccoli, M., Radeva, P.: Toward storytelling from visual lifelogging: an overview. IEEE Trans. Hum.-Mach. Syst. 47, 77–90 (2017)Google Scholar
  2. 2.
    Karaman, S., Benois-Pineau, J., Mégret, R., Dovgalecs, V., Dartigues, J.F., Gaëstel, Y.: Human daily activities indexing in videos from wearable cameras for monitoring of patients with dementia diseases. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 4113–4116. IEEE (2010)Google Scholar
  3. 3.
    Zariffa, J., Popovic, M.R.: Hand contour detection in wearable camera video using an adaptive histogram region of interest. J. Neuroeng. Rehabil. 10, 1–10 (2013)CrossRefGoogle Scholar
  4. 4.
    Rogez, G., Supancic, J.S., Ramanan, D.: Understanding everyday hands in action from RGB-D images. In: ICCV 2015 - IEEE International Conference on Computer Vision, Santiago, Chile (2015)Google Scholar
  5. 5.
    Cartas, A., Marín, J., Radeva, P., Dimiccoli, M.: Recognizing activities of daily living from egocentric images. In: Alexandre, L.A., Salvador Sánchez, J., Rodrigues, J.M.F. (eds.) IbPRIA 2017. LNCS, vol. 10255, pp. 87–95. Springer, Cham (2017). CrossRefGoogle Scholar
  6. 6.
    Nguyen, T.H.C., Nebel, J.C., Florez-Revuelta, F., et al.: Recognition of activities of daily living with egocentric vision: a review. Sensors 16, 72 (2016)CrossRefGoogle Scholar
  7. 7.
    Li, C., Kitani, K.M.: Pixel-level hand detection in egocentric videos. In: Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3570–3577. IEEE (2013)Google Scholar
  8. 8.
    Fathi, A., Ren, X., Rehg, J.M.: Learning to recognize objects in egocentric activities. In: 2011 IEEE Conference On Computer Vision and Pattern Recognition (CVPR), pp. 3281–3288. IEEE (2011)Google Scholar
  9. 9.
    Serra, G., Camurri, M., Baraldi, L., Benedetti, M., Cucchiara, R.: Hand segmentation for gesture recognition in EGO-vision. In: Proceedings of the 3rd ACM International Workshop on Interactive Multimedia on Mobile & #38; Portable Devices. IMMPD 2013, pp. 31–36. ACM, New York (2013)Google Scholar
  10. 10.
    Li, C., Kitani, K.: Model recommendation with virtual probes for egocentric hand detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2624–2631 (2013)Google Scholar
  11. 11.
    Betancourt, A., Lopez, M., Regazzoni, C.S., Rauterberg, M.: A sequential classifier for hand detection in the framework of egocentric vision. In: Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, vol. 1. IEEE Computer Society (2014)Google Scholar
  12. 12.
    Betancourt, A., Morerio, P., Barakova, E.I., Marcenaro, L., Rauterberg, M., Regazzoni, C.S.: A dynamic approach and a new dataset for hand-detection in first person vision. In: Azzopardi, G., Petkov, N. (eds.) CAIP 2015. LNCS, vol. 9256, pp. 274–287. Springer, Cham (2015). CrossRefGoogle Scholar
  13. 13.
    Bambach, S., Lee, S., Crandall, D., Yu, C.: Lending a hand: detecting hands and recognizing activities in complex egocentric interactions. In: 2015 IEEE International Conference on Computer Vision (ICCV). IEEE (2015)Google Scholar
  14. 14.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
  15. 15.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115, 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Everingham, M., Eslami, S.M.A., Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111, 98–136 (2014)CrossRefGoogle Scholar
  17. 17.
    Eshed Ohn-Bar, S.M., Mogelmose, A., Trivedi, M.M.: Vision for intelligent vehicles and applications (VIVA) workshop and challenge. Workshop Chall. 13, 17–30 (2015)Google Scholar
  18. 18.
    Mittal, A., Zisserman, A., Torr, P.H.S.: Hand detection using multiple proposals. In: British Machine Vision Conference (2011)Google Scholar
  19. 19.
    Ng, H.W., Winkler, S.: A data-driven approach to cleaning large face datasets. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 343–347 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Alejandro Cartas
    • 1
    Email author
  • Mariella Dimiccoli
    • 1
    • 2
  • Petia Radeva
    • 1
    • 2
  1. 1.University of BarcelonaBarcelonaSpain
  2. 2.Computer Vision CentreCerdanyola del Valls, BarcelonaSpain

Personalised recommendations