Multiagent Social Influence Detection Based on Facial Emotion Recognition

  • Pankaj MishraEmail author
  • Rafik Hadfi
  • Takayuki Ito
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 616)


There has been an increasing interest in information diffusion within social networks and the usage of multiagent systems for knowledge discovery. In this paper, we build a multiagent system that can track the social correlations within a group of people based on video data. Our information diffusion system targets small groups of people, possibly composed of office workers, meeting attendees, etc. Adopting a multiagent architecture to study the influential correlations in a social network is an adequate choice since it maintains the scalability and robustness of the system. The correlation amongst the nodes of the social network is built on the basis of facial emotions. We evaluated the method in a social network with scripted discussions. Our results show that the emotion propagation was effectively reflected in the predicted social influence correlation.


Multiagent system Facial feature extraction Knowledge discovery Emotion diffusion Social network Social correlation 



This work has been partially supported by the project “Large-scale Consensus Support System based on Agents Technology” in the research area “Intelligent Information Processing Systems Creating Co-Experience Knowledge and Wisdom with Human-Machine Harmonious Collaboration” of JST CREST projects.


  1. 1.
    Anagnostopoulos, A., Kumar, R., Mahdian, M.: Influence and correlation in social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 7–15. ACM (2008)Google Scholar
  2. 2.
    Ashraf, A.B., Lucey, S., Cohn, J.F., Chen, T., Ambadar, Z., Prkachin, K.M., Solomon, P.E.: The painful face-pain expression recognition using active appearance models. Image Vision Comput. 27(12), 1788–1796 (2009)CrossRefGoogle Scholar
  3. 3.
    Bellifemine, F., Bergenti, F., Caire, G., Poggi, A.: Jadea java agent development framework. In: Multi-Agent Programming, pp. 125–147. Springer, New York (2005)Google Scholar
  4. 4.
    Chen, C.W., Ugarte, R.C., Wu, C., Aghajan, H.: Discovering social interactions in real work environments. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 933–938. IEEE (2011)Google Scholar
  5. 5.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 6, 681–685 (2001)CrossRefGoogle Scholar
  6. 6.
    Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression using the Facial Action Coding System (FACS). Oxford University Press, New York (1997)Google Scholar
  7. 7.
    Finin, T., Labrou, Y., Mayfield, J.: KQML as an agent communication language. In: Software Agents. MIT Press, Cambridge (1995)Google Scholar
  8. 8.
    Ghamen, K., Caplier, A.: Positive and negative expressions classification using the belief theory. Int. J. Tomogr. Stat. 17(S11), 72–87 (2011)Google Scholar
  9. 9.
    Happy, S., Routray, A.: Robust facial expression classification using shape and appearance features. In: 2015 Eighth International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–5. IEEE (2015)Google Scholar
  10. 10.
    Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Hsu, C.W., Chang, C.C., Lin, C.J., et al.: A practical guide to support vector classification (2003)Google Scholar
  12. 12.
    Jensen, O.H.: Implementing the viola-jones face detection algorithm (2008)Google Scholar
  13. 13.
    Li, X., Wang, L., Sung, E.: Adaboost with SVM-based component classifiers. Eng. Appl. Artif. Intell. 21(5), 785–795 (2008)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157. IEEE (1999)Google Scholar
  15. 15.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101. IEEE (2010)Google Scholar
  16. 16.
    McPherson, M., Smith-Lovin, L., Cook, J.M.: Birds of a feather: homophily in social networks. Annu. Rev. Sociol. 27, 415–444 (2001)CrossRefGoogle Scholar
  17. 17.
    Alabort-i Medina, J., Antonakos, E., Booth, J., Snape, P., Zafeiriou, S.: Menpo: A comprehensive platform for parametric image alignment and visual deformable models. In: Proceedings of the ACM International Conference on Multimedia, MM 2014, pp. 679–682. ACM, New York (2014).
  18. 18.
    Ruiz, A., Van de Weijer, J., Binefa, X.: From emotions to action units with hidden and semi-hidden-task learning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3703–3711 (2015)Google Scholar
  19. 19.
    Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: 2013 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 397–403. IEEE (2013)Google Scholar
  20. 20.
    Simon, T., Nguyen, M.H., De La Torre, F., Cohn, J.F.: Action unit detection with segment-based SVMS. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2737–2744. IEEE (2010)Google Scholar
  21. 21.
    Stone, P., Veloso, M.: Multiagent systems: a survey from a machine learning perspective. Auton. Robots 8(3), 345–383 (2000)CrossRefGoogle Scholar
  22. 22.
    Sun, J., Tang, J.: A survey of models and algorithms for social influence analysis. In: Aggarwal, C.C. (ed.) Social Network Data Analytics, pp. 177–214. Springer, New York (2011)CrossRefGoogle Scholar
  23. 23.
    Tang, J., Sun, J., Wang, C., Yang, Z.: Social influence analysis in large-scale networks. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 807–816. ACM (2009)Google Scholar
  24. 24.
    Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Scherer, K.: The first facial expression recognition and analysis challenge. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), pp. 921–926. IEEE (2011)Google Scholar
  25. 25.
    Zhang, D.: Probabilistic graphical models for human interaction analysis. Technical report IDIAP (2006)Google Scholar
  26. 26.
    Zhu, Y., De la Torre, F., Cohn, J.F., Zhang, Y.J.: Dynamic cascades with bidirectional bootstrapping for action unit detection in spontaneous facial behavior. IEEE Trans. Affect. Comput. 2(2), 79–91 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringNagoya Institute of TechnologyNagoyaJapan

Personalised recommendations