Psychological Studies

, Volume 63, Issue 3, pp 266–275 | Cite as

Toward Computational Model of Emotion from Individual Difference in Perceiving Facial Expressions

  • Shakuntala GuptaEmail author
  • Ramakrishna Biswal
  • Supratim Gupta
Research in Progress


We propose a computational model for identifying emotional state of a facial expression from appraisal scores given by human observers utilizing their differences in perception. The appraisal model of human emotion is adopted as the basis of this evaluation process with appraisal variables as output. We investigated the performance for both categorical and continuous representation of the variables appraised by human observers. Analysis of the data exhibits higher degree of agreement between estimated Indian ratings and the available reference when these are rated through continuous domain. We also observed that emotional state with negative valence are influential in the perception of hybrid emotional state like ‘Surprise’, only when appraisal variables are labeled through categories of emotions. Thus, the proposed method has implications in developing software to detect emotion using appraisal variables in continuous domain, perceived from facial expression of an agent (or human subject). Further, this model can be customized to include cultural variability in recognizing emotions.


Appraisal variable Computational model Emotion perception Facial expressions Kernel density estimation 



The authors would like to thank the subjects participated in the experiments. This research work is self funded.


  1. Baldassarri, S., & Cerezo, E. (2012). Maxine: Embodied conversational agents for multimodal emotional communication. In N. Mukai (Ed.), Computer Graphics (pp. 195–212). Croatia: Intech.Google Scholar
  2. Barrett, L. F. (2011). Constructing emotion. Psychological Topics, 20(3), 359–380.Google Scholar
  3. Barrett, L. F., Mesquita, B., & Gendron, M. (2011). Context in emotion perception. Current Directions in Psychological Science, 20(5), 286–290.CrossRefGoogle Scholar
  4. Blais, C., Jack, R. E., Scheepers, C., Fiset, D., & Caldara, R. (2008). Culture shapes how we look at faces. PLoS ONE, 3(8), 1–8.CrossRefGoogle Scholar
  5. Bowman, A. W., & Azzalini, A. (1997). Applied smoothing techniques for data analysis. Oxford: Clarendon Press.Google Scholar
  6. Broekens, J., DeGroot, D., & Kosters, W. A. (2008). Formal models of appraisal: Theory, specification, and computational model. Cognitive Systems Research, 9(3), 173–197.CrossRefGoogle Scholar
  7. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.CrossRefGoogle Scholar
  8. Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220.CrossRefPubMedGoogle Scholar
  9. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001). Emotion recognition in human–computer interaction. IEEE Signal Processing Magazine, 18(1), 32–80.CrossRefGoogle Scholar
  10. Dailey, M. N., Lyons, M. J., Ishi, H., Joyce, C., Gyoba, J., & Cottrell, G. W. (2010). Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion, American Psychological Association, 10(6), 874–893.Google Scholar
  11. Ekman, P. (1972). Universals and cultural differences in facial expressions of emotions. In Cole, J. (Ed.), Nebraska Symposium on Motivation (pp. 207–282). Lincoln, NB: University of Nebraska Press.Google Scholar
  12. Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion Review, 3(4), 364–370.CrossRefGoogle Scholar
  13. Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128(2), 203–235.CrossRefPubMedGoogle Scholar
  14. Elfenbein, H. A., & Ambady, N. (2003). Universals and cultural differences in recognizing emotions. Current Directions in Psychological Science, 12(5), 159–164.CrossRefGoogle Scholar
  15. Elfenbein, H. A., Beaupré, M. G., Lévesque, M., & Hess, U. (2007). Toward a dialect theory: Cultural differences in the expression and recognition of posed facial expressions. Emotion, 7(1), 131–146.CrossRefPubMedGoogle Scholar
  16. Elfenbein, H . A., & Eisenkraft, N. (2010). The relation between displaying and perceiving nonverbal cues of affect: A meta-analysis to solve an old mystery. Journal of Personality and Social Psychology, 98(2), 301–318.CrossRefPubMedGoogle Scholar
  17. Ellsworth, P. C. (2013). Appraisal theory: Old and new questions. Emotion Review, 5(2), 125–131.CrossRefGoogle Scholar
  18. Engelmann, J. B., & Pogosyan, M. (2013). Emotion perception across culture: The role of cognitive mechanisms. Frontiers in Psychology, 4, 118.
  19. Gendron, M., Lindquist, K. A., Barsalou, L., & Barrett, L. F. (2012). Emotion words shape emotion percepts. Emotion, 12(2), 314–325.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Jack, R. E., Garrod, O. G. B., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. PNAS USA, 109(19), 7241–7244.CrossRefPubMedGoogle Scholar
  21. Kvalseth, T. O. (1989). Note on cohen’s kappa. Psychological Reports, 65(1), 223–226.CrossRefGoogle Scholar
  22. Lin, J., Spraragen, M., & Zyda, M. (2012). Computational models of emotion and cognition. Advances in Cognitive Systems, 2(1), 59–76.Google Scholar
  23. Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010). The extended Cohn–Kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, (pp. 94–101). San Francisco, CA: IEEE.Google Scholar
  24. Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (1998). Coding facial expressions with gabor wavelets. In Third IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, pp. 1–6.Google Scholar
  25. Marsella, S., & Gratch, J. (2014). Computationally modeling human emotion. Communications of the ACM, 57(12), 56–67.CrossRefGoogle Scholar
  26. Martinez, A., & Du, S. (2012). A model of the perception of facial expressions of emotion by humans: Research overview and perspectives. Machine Learning Research, 13(1), 1589–1608.Google Scholar
  27. Mesquita, B., Vissers, N., & De Leersnyder, J. (2015). Culture and amotion. In J. Wright & J. Berry (Eds.), International encyclopedia of social and behavioral sciences (pp. 542–549). New York: Elsevier.Google Scholar
  28. Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  29. Phelps, E. A., Ling, S., & Carrasco, M. (2006). Emotion facilitates perception and potentiates the perceptual benefit of attention. Psychological Sciences, 17(4), 292–299.CrossRefGoogle Scholar
  30. Roberson, D., Damjanovic, L., & Pilling, M. (2007). Categorical perception of facial expressions: Evidence for a “category adjustment” model. Memory & Cognition, 35(7), 1814–1829.CrossRefGoogle Scholar
  31. Roberson, D., Davidoff, J., & Braisby, N. (1999). Similarity and categorisation: Neuropsychological evidence for a dissociation in explicit categorisation tasks. Cognition, 71(1), 1–42.CrossRefPubMedGoogle Scholar
  32. Robinson, D. L. (2009). Brain function, mental experience and personality. The Netherlands Journal of Psychology, 64(1), 152–167.Google Scholar
  33. Russell, J . A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102–141.CrossRefPubMedGoogle Scholar
  34. Russell, J. A. (1995). Facial expressions of emotion: What lies beyond minimal universality? Psychological Bulletin, 118(3), 379–391.Google Scholar
  35. Russell, J. A., & Fehr, B. (1987). Relativity in the perception of emotion in facial expressions. Journal of Experimental Psychology: General, 116(3), 223–237.CrossRefGoogle Scholar
  36. Sauter, D. A., Eisner, F., Ekman, P., & Scott, S. K. (2010). Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. PNAS, 107(6), 2408–2412.CrossRefPubMedGoogle Scholar
  37. Scott, D. W. (2015). Multivariate density estimation (2nd ed.). Hoboken, NJ: Wiley.Google Scholar
  38. Silverman, B. W. (1998). Density estimation for statistics and data analysis. Chapman & Hall/CRC Monographs on Statistics & Applied Probability. Chapman and Hall/CRC.Google Scholar
  39. Tian, Y., Kanade, T., & Cohn, J. F. (2011). Facial expression recognition. In S. Z. Li & A. K. Jain (Eds.), Handbook of face recognition (pp. 487–519). Springer: London.Google Scholar
  40. Watson, D. G., & Blagrove, E. (2012). Tagging multiple emotional stimuli: Negative valence has little benefit. Journal of Experimental Psychology: Human Perception and Performance, 38(3), 785–803.PubMedGoogle Scholar
  41. Wehrle, T., & Scherer, K. R. (2001). Towards computational modeling of appraisal theories. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotions: Theory, methods, research (pp. 350–365). New York: Oxford University Press.Google Scholar
  42. Whissell, C . M. (1989). The dictionary of affect in language. In R. Plutchik & H. Kellerman (Eds.), Emotion: Theory, research, and experience (pp. 112–131). New York: Academic Press.Google Scholar
  43. Widen, S. C., & Russell, J. A. (2010). Differentiation in preschooler’s categories of emotion. Emotion, 10(5), 651–661.CrossRefPubMedGoogle Scholar
  44. Woolfolk, A. (2006). Educational psychology (9th ed.). New York: Pearson Education.Google Scholar
  45. Yan, X., Andrews, T. J., & Young, A. W. (2016). Cultural similarities and differences in perceiving and recognizing facial expressions of basic emotions. Journal of Experimental Psychology: Human Perception and Performance, 42(3), 423–440.PubMedGoogle Scholar
  46. Zadra, J. R., & Clore, G. L. (2011). Emotion and perception: The role of affective information. WIREs Cognitive Science, 2(6), 676–685.CrossRefPubMedGoogle Scholar

Copyright information

© National Academy of Psychology (NAOP) India 2018

Authors and Affiliations

  1. 1.Department of Humanities and Social SciencesNITRourkelaIndia
  2. 2.Department of Electrical EngineeringNITRourkelaIndia

Personalised recommendations