HAR-Net: Fusing Deep Representation and Hand-Crafted Features for Human Activity Recognition

  • Mingtao Dong
  • Jindong HanEmail author
  • Yuan He
  • Xiaojun Jing
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 550)


Wearable computing and context awareness are the focuses of study in the field of artificial intelligence recently. One of the most appealing as well as challenging applications is the Human Activity Recognition (HAR) utilizing smart phones. Conventional HAR based on Support Vector Machine relies on manually extracted features. This approach is time and energy consuming in prediction due to the partial view toward which features to be extracted by human. With the rise of deep learning, artificial intelligence has been making progress toward being a mature technology. This paper proposes a new approach based on deep learning called HAR-Net to address the HAR issue. The study used the data collected by gyroscopes and acceleration sensors in android smart phones. The HAR-Net fusing the hand-crafted features and high-level features extracted from convolutional neural network to make prediction. The performance of the proposed method was proved to be higher than the original MC-SVM approach. The experimental results on the UCI dataset demonstrate that fusing the two kinds of features can make up for the shortage of traditional feature engineering and deep learning techniques.


Human Activity Recognition Inception convolutional neural network Hand-crafted features 


  1. 1.
    Hernandez, J., Riobo, I., Rozga, A., Abowd, G.D., Picard, R.W.: Using electrodermal activity to recognize ease of engagement in children during social interactions. In: ACM International Joint Conference on Pervasive & Ubiquitous Computing, vol. 48, pp. 307–317. ACM, Geneva (2014)Google Scholar
  2. 2.
    Kjærgaard, M.B., Wirz, M., Roggen, D.: Detecting pedestrian flocks by fusion of multi-modal sensors in mobile phones. In: ACM Conference on Ubiquitous Computing, pp. 240–249. ACM, Geneva (2012)Google Scholar
  3. 3.
    Chengqing, Z.: Statistical Natural Language Processing. Tsinghua University Press, Beijing (2008)Google Scholar
  4. 4.
    Xinqing, S.: A Brief Treatise on Computational Electromagnetics. Press of University of Science and Technology of China, Beijing (2004)Google Scholar
  5. 5.
    Murata, S., Suzuki, M., Fujinami, K.: A wearable projector-based gait assistance system and its application for elderly people. In: ACM International Joint Conference on Pervasive & Ubiquitous Computing, pp. 143–152. ACM, Geneva (2013)Google Scholar
  6. 6.
    Fan, M., Gravem, D., Dan, M.C., Patterson, D.J.: Augmenting gesture recognition with Erlang-Cox models to identify neurological disorders in premature babies. In: International Conference on Ubiquitous Computing, pp. 411–420. ACM Geneva (2012)Google Scholar
  7. 7.
    Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L.: Energy efficient smartphone-based activity recognition using fixed-point arithmetic. J. Univ. Comput. Sci. 19(9), 1295–1314 (2013)Google Scholar
  8. 8.
    Ng, Y.H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., Toderici, G.: Beyond short snippets: deep networks for video classification. Comput. Vis. Pattern Recogn. 16(4), 4694–4702 (2015)Google Scholar
  9. 9.
    Plötz, T., Hammerla, N.Y., Olivier, P.: Feature learning for activity recognition in ubiquitous computing. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI), pp. 1729–1734. IJCAI, Barcelona, 16–22 (2011)Google Scholar
  10. 10.
    Jiang, W., Yin, Z.: Human activity recognition using wearable sensors by deep convolutional neural networks. In: Proceedings of the 23rd ACM International Conference on Multimedia, pp. 1307–1310. ACM, Geneva (2015)Google Scholar
  11. 11.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  12. 12.
    Zeng, M., Le, T.N., Yu, B., Mengshoel, O.J., Zhu, J., Wu, P.: Convolutional neural networks for human activity recognition using mobile sensors. In: International Conference on Mobile Computing, Applications and Services, pp. 197–205. IEEE, New York (2015)Google Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980. (2014)
  14. 14.
    Anguita, D., Ghio, A., Oneto, L., Parra, X., Reyes-Ortiz, J.L.: A Public domain dataset for human activity recognition using smartphones. In: 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, pp. 24–26. ESANN, Bruges (2013)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Mingtao Dong
    • 1
  • Jindong Han
    • 2
    Email author
  • Yuan He
    • 2
  • Xiaojun Jing
    • 2
  1. 1.The Second High School Attached to Beijing Normal UniversityBeijingChina
  2. 2.Beijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations