Structure Function Based Transform Features for Behavior-Oriented Social Media Image Classification

  • Divya Krishnani
  • Palaiahnakote Shivakumara
  • Tong Lu
  • Umapada Pal
  • Raghavendra RamachandraEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12046)


Social media has become an essential part of people to reflect their day to day activities including emotions, feelings, threatening and so on. This paper presents a new method for the automatic classification of behavior-oriented images like Bullying, Threatening, Neuroticism-Depression, Neuroticism-Sarcastic, Psychopath and Extraversion of a person from social media images. The proposed method first finds facial key points for extracting features based on a face detection algorithm. Then the proposed method labels face regions as foreground and other than face region as background to define context between foreground and background information. To extract context, the proposed method explores Structural Function based Transform (SFBT) features, which study variations on pixel values. To increase discriminating power of the context features, the proposed method performs clustering to integrate the strength of the features. The extracted features are then fed to Support Vector Machines (SVM) for classification. Experimental results on a dataset of six classes show that the proposed method outperforms the existing methods in terms of confusion matrix and classification rate.


Social media images Face detection Structural features Person behavior Classification 


  1. 1.
    Xue, F., Sun, J., Liu, X., Liu, T., Lu, Q.: Social multi-modal event analysis via knowledge-based weighted topic model. J. Vis. Commun. Image Represent. 59, 1–8 (2019)CrossRefGoogle Scholar
  2. 2.
    Roy, S., et al.: Rough fuzzy based scene categorization for text detection and recognition in video. Pattern Recogn. 80, 64–82 (2018)CrossRefGoogle Scholar
  3. 3.
    Nogueria, K., Penatti, O.A.B., Santos, J.A.D.: Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recogn. 61, 539–556 (2017)CrossRefGoogle Scholar
  4. 4.
    Mabrouk, A.B., Zagrouba, E.: Abnormal behavior recognition for intelligent video surveillance systems: a review. Expert Syst. Appl. 91, 480–491 (2018)CrossRefGoogle Scholar
  5. 5.
    Liu, L., Pietro, D.P., Samani, Z.R., Moghadadam, M.E., Ungar, L.: Analyzing personality through social media profile picture choice. In: Proceedings of ICWSM (2016)Google Scholar
  6. 6.
    Majumder, A., Behera, L., Subramanian, V.K.: Automatic facial expression recognition system using deep network-based data fusion. IEEE Trans. Cybern. 48, 103–114 (2018)CrossRefGoogle Scholar
  7. 7.
    Perveen, N., Roy, D., Mohan, C.K.: Spontaneous expression recognition using universal attribute model. IEEE Trans. Image Process. 27, 5575–5584 (2018)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Wang, S., Pan, B., Chen, H., Ji, Q.: Thermal augmented expression recognition. IEEE Trans. Cybern. 48, 2203–2214 (2018)CrossRefGoogle Scholar
  9. 9.
    Prasad, N., Unnikrishnan, K., Jayakrishnan, R.: Fraud detection by facial expression analysis using intel realsense and augmented reality. In: Proceedings of ICICCS, pp. 919–923 (2018)Google Scholar
  10. 10.
    Arru, G., Mazumdar, P., Battisti, F.: Exploiting visual behavior for autism spectrum disorder identification. In: Proceedings of ICMEM, pp. 637–640 (2019)Google Scholar
  11. 11.
    Beeck, K.V., Engeland, K.V., Vennekens, J., Goedeme, T.: Abnormal behavior detection in LWIR surveillance of railway platforms. In: Proceedings of AVSS (2017)Google Scholar
  12. 12.
    Cosar, S., Donatiello, G., Bogorny, V., Garate, C., Otavio, L., Bremond, F.: Toward abnormal trajectory and event detection in video surveillance. IEEE Trans. CSVT 27, 683–695 (2017)Google Scholar
  13. 13.
    Liu, C.W., Chen, H.T., Lo, K.H., Wang, C.J., Chuang, J.H.: Accelerating vanishing point based line sampling scheme for real time people localization. IEEE Trans. CSVT 27, 409–420 (2017)Google Scholar
  14. 14.
    Tiwari, C., Hanmandlu, M., Vasikarla, S.: Suspicious face detection based on eye and other facial features movement monitoring. In: Proceedings of AIPR, pp. 1–8 (2015)Google Scholar
  15. 15.
    Hsu, S.C., Chuang, C.H., Huang, C.L., Teng, P.R., Lin, M.J.: A video based abnormal human behavior detection for psychiatric patient monitoring. In: Proceedings of IWAIT, pp. 1–4 (2018)Google Scholar
  16. 16.
    Grover, J., Hanmandlu, M.: The fusion of multispectral palmprints using the information set based features and classifier. Eng. Appl. Artif. Intell. 67, 111–125 (2018)CrossRefGoogle Scholar
  17. 17.
    Farzindar, A., Inkpen, D.: Natural Language Processing for Social Media. Synthesis Lectures on Human Language Techniques. Morgan and Claypool Publishers, San Rafael (2015)CrossRefGoogle Scholar
  18. 18.
    Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multi-task cascaded convolutional networks. IEEE Signal Process. Lett. 23, 1499–1503 (2016)CrossRefGoogle Scholar
  19. 19.
    Xu, X., Wang, Y., Chen, S.: Medical image fusion using discrete fractional wavelet transform. Biomed. Signal Process. Control 27, 103–111 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Divya Krishnani
    • 1
  • Palaiahnakote Shivakumara
    • 2
  • Tong Lu
    • 3
  • Umapada Pal
    • 4
  • Raghavendra Ramachandra
    • 5
    Email author
  1. 1.International Institute of Information Technology, Naya RaipurNaya RaipurIndia
  2. 2.Faculty of Computer Science and Information TechnologyUniversity of MalayaKuala LumpurMalaysia
  3. 3.National Key Lab for Novel Software TechnologyNanjing UniversityNanjingChina
  4. 4.Computer Vision and Pattern Recognition UnitIndian Statistical InstituteKolkataIndia
  5. 5.Faculty of Information Technology and Electrical EngineeringNorwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations