Skip to main content
SpringerLink
Log in

Methods for Determination of Psychophysiological Condition of User Within Smart Environment Based on Complex Analysis of Heterogeneous Data

  • Conference paper
  • First Online:
Proceedings of 14th International Conference on Electromechanics and Robotics “Zavalishin's Readings”

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 154))

Abstract

Smart Environments (SE)’ implementation provides great possibilities and services for users. But also such systems can be potentially harmful in terms of destructive behavior that can be met in this kind of informational space. In this case, it is important to prevent such information from users, especially from children. To solve this problem, it is necessary to analyze all available information in such systems: video (images), text, and audio. In this paper, the review of state-of-the-art methods of data analysis from different modalities is presented. Also the way of transfer learning technique implementation of heterogeneous data analysis is proposed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Usova, E.B.: Psychology of Deviant Behavior. Training and Metodology Complex. Publishing house of the MIU, Minsk (2010)

    Google Scholar 

  2. Zhukovskiy, Y., Koteleva, N.: Method of Data storing, collection and aggregation for definition of life-cycle resources of electromechanical equipment. In: IOP Conference Series: Earth and Environmental Science, vol. 87, no. 3, pp. 032057. IOP Publishing (2017)

    Google Scholar 

  3. Zhukovskiy, Y., Koteleva, N.: Development of augmented reality system for servicing electromechanical equipment. In Journal of Physics: Conference Series, vol. 1015, no. 4, pp. 042068. IOP Publishing (2018)

    Google Scholar 

  4. Zhukovskiy, Y., Korolev, N., Koteleva, N.: About increasing informativity of diagnostic system of asynchronous electric motor by extracting additional information from values of consumed current parameter. In Journal of Physics: Conference Series, vol. 1015, no. 3, pp. 032158. IOP Publishing (2018)

    Google Scholar 

  5. IqbalQuraishi, M., Pal Choudhury, J., De, M., Chakraborty, P.: A framework for the recognition of human emotion using soft computing models. Int. J. Comput. Appl. 40(17), 50–55 (2012)

    Google Scholar 

  6. Yang, Z., Wu, W., Li, H., Zhou, Y.: Consensus in smart computing systems. In: 2017 IEEE International Conference on Smart Computing (SMARTCOMP), pp. 1–8. IEEE (2017)

    Google Scholar 

  7. Verma, S., Doshi, J.: Correlation between text book usage and academic performance of student in higher education using ‘R’. In: Proceedings of International Conference on Communication and Networks, pp. 11–18. Springer, Singapore (2017)

    Google Scholar 

  8. Moussa, M., Hmila, M., Douik, A.: A novel face recognition approach based on genetic algorithm optimization. Stud. Inform. Control. 27(1), 127–134 (2018)

    Article  Google Scholar 

  9. Prieto, L.A.B., Komínkova-Oplatková, Z.: A performance comparison of two emotion-recognition implementations using OpenCV and Cognitive Services API. In: MATEC Web of Conferences, vol. 125, pp. 02067. EDP Sciences (2017)

    Google Scholar 

  10. Suryani, D., Ekaputra, V., Chowanda, A.: Multi-modal asian conversation mobile video dataset for recognition task. Int. J. Electr. Comput. Eng. (IJECE) 8(5) (2018)

    Article  Google Scholar 

  11. Lekdioui, K., Ruichek, Y., Messoussi, R., Chaabi, Y., Touahni, R.: Facial expression recognition using face-regions. In: 2017 International Conference on Advanced Technologies for Signal and Image Processing (ATSIP), pp. 1–6. IEEE (2017)

    Google Scholar 

  12. Li, J., et al.: Target Detection method before tracking based on particle filter. In: International Congress of Information and Communication Technology (ICICT), vol. 107, pp. 135–141(2017)

    Google Scholar 

  13. Yang, H., Ciftci, U., Yin, L.: Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2168–2177 (2018)

    Google Scholar 

  14. Bardhan, Y., Fulzele, T.A., Ranjan, P., Upadhyay, S., Bharate, V.D.: Emotion Recognition using Image Processing. Int. J. Trend Sci. Res. Dev. (IJTSRD) 3(2), 1523–1526 (2018)

    Google Scholar 

  15. Krestinskaya, O., James, A.P.: Facial emotion recognition using min-max similarity classifier. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 752–758 (2017)

    Google Scholar 

  16. Glazkova, A.V.: An Approach to text classification based on age groups of addressees. SPIIRAS Proc. 3, 51–69 (2017)

    Article  Google Scholar 

  17. Ventirozos, F.K., Varlamis, I., Tsatsaronis, G.: Detecting aggressive behavior in dis-cussion threads using text mining. In: International Conference on Computational Linguistics and Intelligent Text Processing, pp. 420–431. Springer, Cham. (2017)

    Chapter  Google Scholar 

  18. Medhat, W., Hassan, A., Korashy, H.: Sentiment analysis algorithms and applications: A survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014)

    Article  Google Scholar 

  19. Chatzakou, D., Kourtellis, N., Blackburn, J., De Cristofaro, E., Stringhini, G., Vakali, A.: Mean birds: detecting aggression and bullying on twitter. In: Proceedings of the 2017 ACM on web science conference, pp. 13–22. ACM (2017)

    Google Scholar 

  20. Van Hee, C., Jacobs, G., Emmery, C., Desmet, B., Lefever, E., Verhoeven, B., Hoste, V.: Automatic detection of cyberbullying in social media text. PLoS ONE 13(10), e020379 (2018)

    Google Scholar 

  21. Tommasel, A., Rodriguez, J.M., Godoy, D.: Textual aggression detection through deep learning. In: Proceedings of the First Workshop on Trolling, Aggression and Cyberbully-ing (TRAC-2018), pp. 177–187 (2018)

    Google Scholar 

  22. Golem, V., Karan, M., Šnajder, J.: Combining shallow and deep learning for aggressive text detection. In; Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 188–198 (2018)

    Google Scholar 

  23. Reyes, A., Rosso, P.: Making objective decisions from subjective data: Detecting irony in customer reviews. Decis. Support Syst. 53(4), 754–760 (2012)

    Article  Google Scholar 

  24. Mejova, Y., Srinivasan, P.: Exploring feature definition and selection for sentiment classifiers. In: Fifth international AAAI conference on weblogs and social media (2011)

    Google Scholar 

  25. Escalante, H.J., Villatoro-Tello, E., Garza, S.E., López-Monroy, A.P., Montes-y-Gómez, M., Villaseñor-Pineda, L.: Early detection of deception and aggressiveness using profile-based representations. Expert Syst. Appl. 89, 99–111 (2017)

    Article  Google Scholar 

  26. Plutchik, R.: A general psychoevolutionary theory of emotion. In: Theories of emotion pp. 3–33. Academic press (1980)

    Google Scholar 

  27. Zhmurov, D.V.: Nasilie (agressiya) i literatura, 2nd edn. ASP-print, Irkutsk (2005)

    Google Scholar 

  28. Apresyan, V.Yu.: Impliczitnaya agressiya v yazyke. In: Kompyuternaya lingvistika i intellektualnye tekhnologii: Tr. Mezhdunar. konferenczii Dialog conference, pp. 32–35 (2003)

    Google Scholar 

  29. Jo, H., Kim, S.M., Ryu, J.: What we really want to find by Sentiment Analysis: the Relationship between computational models and psychological State. arXiv preprint arXiv:1704.03407 (2017)

  30. Smirnov, I.V., Shelmanov, A.O., Kuzneczova, E.S., Khramoin, I.V.: Semantiko-sintaksicheskiy analiz estestvennykh yazykov. Iskusstvennyj intellekt i prinyatie reshenij 1(43) (2013)

    Google Scholar 

  31. Kipyatkova, I.S., Karpov, A.A.: Variants of deep artificial neural networks for speech recognition systems. SPIIRAS Proc. 6, 80–103 (2016)

    Article  Google Scholar 

  32. Verkholyak, O.V., Kaya, H., et al.: Modeling short-term and long-term dependencies of the speech signal for paralinguistic emotion classification. SPIIRAS Proc. 18, 30–56 (2019)

    Article  Google Scholar 

  33. Sidorov, K.V., Filatova, N.N.: Analiz priznakov emoczional`no-okrashennoj rechi. Vestnik TvGTU 180(20), 26–32 (2012)

    Google Scholar 

  34. Simonchik, K.K.: Razrabotka programmno-apparatnogo kompleksa dlya oczenki psikhofiziologicheskogo (psikhoe`moczional`nogo) sostoyaniya diktora. Bibliogr, SPb (2013)

    Google Scholar 

Download references

Acknowledgements

This research is supported by the Russian Foundation for Basic Research (project No. 18-29-22061_мк).

Author information

Authors and Affiliations

  1. Laboratory of Big Data of Socio-Cyberphysical Systems, SPIIRAS, 14-Th Linia, VI, no. 39, 199178, St. Petersburg, Russia

    Dmitrii Levonevskii

  2. Laboratory of Autonomous Robotic Systems, SPIIRAS, 14-Th Linia, VI, no. 39, 199178, St. Petersburg, Russia

    Olga Shumskaya & Dmitrii Malov

  3. Laboratory of Speech and Multimodal Interfaces, SPIIRAS, 14-Th Linia, VI, no. 39, 199178, St. Petersburg, Russia

    Alena Velichko

  4. Laboratory of Computer-Information Systems and Software Engineering, SPIIRAS, 14-Th Linia, VI, no. 39, 199178, St. Petersburg, Russia

    Mikhael Uzdiaev

Authors
  1. Dmitrii Levonevskii
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olga Shumskaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alena Velichko
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mikhael Uzdiaev
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dmitrii Malov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dmitrii Malov .

Editor information

Editors and Affiliations

  1. St. Petersburg Institute for Informatics and Automation of the Russian Academy of Sciences, St. Petersburg, Russia

    Andrey Ronzhin

  2. Saint Petersburg State University of Aerospace Instrumentation, St. Petersburg, Russia

    Vladislav Shishlakov

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Levonevskii, D., Shumskaya, O., Velichko, A., Uzdiaev, M., Malov, D. (2020). Methods for Determination of Psychophysiological Condition of User Within Smart Environment Based on Complex Analysis of Heterogeneous Data. In: Ronzhin, A., Shishlakov, V. (eds) Proceedings of 14th International Conference on Electromechanics and Robotics “Zavalishin's Readings”. Smart Innovation, Systems and Technologies, vol 154. Springer, Singapore. https://doi.org/10.1007/978-981-13-9267-2_42

Download citation

Publish with us

Policies and ethics

Search

Navigation