Atypical Sample Regularizer Autoencoder for Cross-Domain Human Activity Recognition


The sensor-based human activity recognition (HAR) using machine learning requires a sufficiently large amount of annotated data to realize an accurate classification model. This requirement stimulates the advancement of the transfer learning research area that minimizes the use of labeled data by transferring knowledge from the existing activity recognition domain. Existing approaches transform the data into a common subspace between domains which theoretically loses information, to begin with. Besides, they are based on the linear projection which is bound to linearity assumption and its limitations. Some recent works have already incorporated nonlinearity to find a latent representation that minimizes domain discrepancy based on an autoencoder that includes statistical distance minimization. However, such approach discovers latent representation for both domains at once, which causes sub-optimal representation because both domains compensate each other’s reconstruction error during the training. We propose an autoencoder-based approach on domain adaptation for sensor-based HAR. The proposed approach learns a latent representation which minimizes the discrepancy between domains by reducing statistical distance. Instead of learning representation of both domains simultaneously, our method is a two-phase approach which first learns the representation for the domain of interest independently to ensure its optimality. Subsequently, the useful information from the existing domain is transferred. We test our approach on the publicly available sensor-based HAR datasets, using cross-domain setup. The experimental result shows that our approach significantly outperforms the existing ones.

This is a preview of subscription content, access via your institution.

Fig. 1


  1. 1.

  2. 2.


  1. Al-Stouhi S, Reddy CK (2011) Adaptive boosting for transfer learning using dynamic updates. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, pp 60–75.

  2. Aramendi, A. A., Weakley, A., Goenaga, A. A., Schmitter-Edgecombe, M., & Cook, D. J. (2018). Automatic assessment of functional health decline in older adults based on smart home data. Journal of Biomedical Informatics, 81, 119–130.

    Article  Google Scholar 

  3. Bagaveyev S, Cook DJ (2014) Designing and evaluating active learning methods for activity recognition. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, ACM, pp 469–478.

  4. Barshan, B., & Yüksek, M. C. (2014). Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. The Computer Journal, 57(11), 1649–1667.

    Article  Google Scholar 

  5. Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. Proceedings of the 2006 conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 120–128.

  6. Chavarriaga, R., Sagha, H., Calatroni, A., Digumarti, S. T., Tröster, G., Millán, J. R., & Roggen, D. (2013). The opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters, 34(15), 2033–2042.

    Article  Google Scholar 

  7. Chen M, Xu Z, Weinberger K, Sha F (2012) Marginalized denoising autoencoders for domain adaptation. arXiv preprint arXiv:12064683.

  8. Dai W (2007). Boosting for transfer learning. ICML’07: Proc. of the 24th Int. Conf, on Machine learning, 2007, pp 193–200.

  9. Fernando B, Habrard A, Sebban M, Tuytelaars T (2014). Subspace alignment for domain adaptation. arXiv preprint arXiv:14095241.

  10. Gong B, Shi Y, Sha F, Grauman K (2012). Geodesic flow kernel for unsupervised domain adaptation. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE, pp 2066–2073.

  11. Jahn A, Kroll D, Lau SL, David K (2017). An activity history based approach for recognizing the mode of transportation. In: 2017 IEEE Conference on Open Systems (ICOS), IEEE, pp 21–25.

  12. Kim, J., Chi, S., & Seo, J. (2018). Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks. Automation in Construction, 87, 297–308.

    Article  Google Scholar 

  13. Kingma DP, Ba J (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980.

  14. Long, M., Wang, J., Sun, J., & Philip, S. Y. (2015). Domain invariant transfer kernel learning. IEEE Transactions on Knowledge and Data Engineering, 27(6), 1519–1532.

    Article  Google Scholar 

  15. Meditskos, G., Plans, P. M., Stavropoulos, T. G., Benois-Pineau, J., Buso, V., & Kompatsiaris, I. (2018). Multi-modal activity recognition from egocentric vision, semantic enrichment and lifelogging applications for the care of dementia. Journal of Visual Communication and Image Representation, 51, 169–190.

    Article  Google Scholar 

  16. Pan SJ, Kwok JT, Yang Q, et al. (2008). Transfer learning via dimensionality reduction. AAAI, vol 8, pp 677–682.

  17. Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011). Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 22(2), 199–210.

    Article  Google Scholar 

  18. Ponce, H., Martínez-Villaseñor, M., & Miralles-Pechuán, L. (2016). A novel wearable sensor-based human activity recognition approach using artificial hydrocarbon networks. Sensors, 16(7), 1033.

    Article  Google Scholar 

  19. Reiss A, Stricker D (2012) Introducing a new benchmarked dataset for activity monitoring. 2012 16th International Symposium on Wearable Computers, IEEE, pp 108–109.

  20. Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., & Tröster, G. (2008). Wearable activity tracking in car manufacturing. IEEE Pervasive Computing, 2, 42–50.

    Article  Google Scholar 

  21. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. Thirtieth AAAI Conference on Artificial Intelligence.

  22. Sun B, Feng J, Saenko K (2017) Correlation alignment for unsupervised domain adaptation. Domain Adaptation in Computer Vision Applications, Springer, pp 153–171.

  23. Wang J, Chen Y, Hu L, Peng X, Philip SY (2018a). Stratified transfer learning for cross-domain activity recognition. 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom), IEEE, pp 1–10.

  24. Wang S, Zeng Y, Liu Q, Zhu C, Zhu E, Yin J (2018b). Detecting abnormality without knowing normality: A two-stage approach for unsupervised video abnormal event detection. In: 2018 ACM Multimedia Conference on Multimedia Conference, ACM, pp 636–644.

  25. Wannenburg, J., & Malekian, R. (2017). Physical activity recognition from smartphone accelerometer data for user context awareness sensing. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(12), 3142–3149.

    Article  Google Scholar 

  26. Yao Y, Doretto G (2010). Boosting for transfer learning with multiple sources. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, pp 1855–1862.

  27. Zhang Y, Tian G, Zhang S, Li C (2019). A knowledge-based approach for multiagent collaboration in smart home: From activity recognition to guidance service. IEEE Transactions on Instrumentation and Measurement.

  28. Zhuang, F., Cheng, X., Luo, P., Pan, S. J., & He, Q. (2018). Supervised representation learning with double encoding-layer autoencoder for transfer learning. ACM Transactions on Intelligent Systems and Technology (TIST), 9(2), 16.

    Google Scholar 

Download references


This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2019S1A5A2A03049067) and (2018R1D1A1B07047241).

Author information



Corresponding author

Correspondence to Bernardo Nugroho Yahya.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Human Participants

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Bernardo Nugroho Yahya and Seok-Lyong Lee contributed equally to this article.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Prabono, A.G., Yahya, B.N. & Lee, SL. Atypical Sample Regularizer Autoencoder for Cross-Domain Human Activity Recognition. Inf Syst Front 23, 71–80 (2021).

Download citation


  • Transfer learning
  • Autoencoder
  • Human activity recognition
  • Machine learning