Abstract
Previous studies have shown that the success of interpersonal interaction depends not only on the contents we communicate explicitly, but also on the social signals that are conveyed implicitly. In this paper, we present NovA (NOnVerbal behavior Analyzer), a system that analyzes and facilitates the interpretation of social signals conveyed by gestures, facial expressions and others automatically as a basis for computer-enhanced social coaching. NovA records data of human interactions, automatically detects relevant behavioral cues as a measurement for the quality of an interaction and creates descriptive statistics for the recorded data. This enables us to give a user online generated feedback on strengths and weaknesses concerning his social behavior, as well as elaborate tools for offline analysis and annotation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Mehrabian, A.: Silent messages: Implicit Communication of Emotions and Attitudes. Wadsworth Publishing Co Inc., Belmont (1981)
Eagle, N., Pentland, A.: Reality mining: Sensing complex social signales. J. of Personal and Ubiquitous Computing 10(4), 255–268 (2006)
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: Elan: A professional framework for multimodality research. In: Proc. of the Fifth International Conference on Language Resources and Evaluation (LREC), pp. 879–896 (2006)
Kipp, M.: Anvil: The video annotation research tool. In: Handbook of Corpus Phonology. Oxford University Press, Oxford (2013)
Schmidt, T.: Transcribing and annotating spoken language with exmaralda. In: Proc. of the LREC-Workshop on XML Based Richly Annotated Corpora, Lisbon 2004, pp. 879–896. ELRA, Paris (2004)
Curhan, J., Pentland, A.: Thin slices of negotiation: predicting outcomes from conversational dynamics withing the first 5 minutes. J. of Applied Psychology 92(3), 802–811 (2007)
Pentland, A.: Automatic mapping and modelling of human networks. Physica A(378), 59–67 (2007)
Schuller, B., MĂ¼ller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Höthker, A., Konosu, H.: Being bored? recognising natural interest by extensive audiovisual integration for real-life application. Image Vision Comput. 27(12), 1760–1774 (2009)
Rich, C., Ponsleur, B., Holroyd, A., Sidner, C.L.: Recognizing engagement in human-robot interaction. In: Proc. of the 5th ACM/IEEE Intl. Conf. on Human-Robot Interaction, HRI 2010, pp. 375–382. IEEE Press, Piscataway (2010)
Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009)
Pianesi, F., Mana, N., Cappelletti, A., Lepri, B., Zancanaro, M.: Multimodal recognition of personality traits in social interactions. In: Proc. of the 10th International Conference on Multimodal Interfaces, ICMI 2008, pp. 53–60. ACM, NY (2008)
Dong, W., Lepri, B., Cappelletti, A., Pentland, A.S., Pianesi, F., Zancanaro, M.: Using the influence model to recognize functional roles in meetings. In: Proc. of the 9th International Conference on Multimodal Interfaces, ICMI 2007, pp. 271–278. ACM, New York (2007)
Hung, H., Gatica-Perez, D.: Estimating cohesion in small groups using audio-visual nonverbal behavior. Trans. Multi. 12(6), 563–575 (2010)
Sandbach, G., Zafeiriou, S., Pantic, M., Yin, L.: Static and dynamic 3d facial expression recognition: A comprehensive survey. Image Vision Comput. 30(10), 683–697 (2012)
Caridakis, G., Raouzaiou, A., Karpouzis, K., Kollias, S.: Synthesizing gesture expressivity based on real sequences. In: Workshop on Multimodal Corpora: from Multimodal Behaviour Theories to Usable Models. LREC, Genoa (2006)
Vogt, T., André, E., Bee, N.: Emovoice - a framework for online recognition of emotions from voice. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds.) PIT 2008. LNCS (LNAI), vol. 5078, pp. 188–199. Springer, Heidelberg (2008)
Kleinsmith, A., Bianchi-Berthouze, N.: Form as a cue in the automatic recognition of non-acted affective body expressions. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.-C. (eds.) ACII 2011, Part I. LNCS, vol. 6974, pp. 155–164. Springer, Heidelberg (2011)
Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)
Camurri, A., Volpe, G., De Poli, G., Leman, M.: Communicating expressiveness and affect in multimodal interactive systems. IEEE MultiMedia 12(1) (2005)
Scherer, S., Marsella, S., Stratou, G., Xu, Y., Morbini, F., Egan, A., Rizzo, A(S.), Morency, L.-P.: Perception markup language: Towards a standardized representation of perceived nonverbal behaviors. In: Nakano, Y., Neff, M., Paiva, A., Walker, M. (eds.) IVA 2012. LNCS, vol. 7502, pp. 455–463. Springer, Heidelberg (2012)
Simpson, J.A., Harris, B.A.: Interpersonal attraction. In: Weber, A.L., Harvey, J.H. (eds.) Perspectives on Close Relationships, pp. 45–66. Prentice Hall (1994)
McGinley, H., LeFevre, R., McGinley, P.: The influence of a communicator’s body position on opinion. J. of Personality and Social Psychology 31(4), 686–690 (1975)
Schouwstra, S., Hoogstraten, J.: Head position and spinal position as determinants of perceived emotional state. Perceptual and Motor Skills 81, 673–674 (1995)
Sidner, C.L., Kidd, C.D., Lee, C., Lesh, N.: Where to look: a study of human-robot engagement. In: IUI 2004: Proc. of the 9th International Conference on Intelligent User Interfaces, pp. 78–84. ACM Press, New York (2004)
Pease, A.: Body Language. Sheldon Press, London (1988)
Bandura, A.: Self Efficacy: The Exercise of Control. Palgrave Macmillan, New York (1997)
Forgas, J.P., Cooper, J., Crano, W.D.: The Psychology of Attitudes and Attitude Change. Taylor & Francis Group, New York (2010)
Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (ssi) framework - multimodal signal processing and recognition in real-time. In: Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain (2013)
Kistler, F., Endrass, B., Damian, I., Dang, C., André, E.: Natural interaction with culturally adaptive virtual characters. Germany Journal on Multimodal User Interfaces Heidelberg/Berlin (2012)
Wallbott, H.: Bodily expression of emotion. European Jrl. of Social Psychology (28), 879–896 (1998)
Ruf, T., Ernst, A., KĂ¼blbeck, C.: Face detection with the sophisticated high-speed object recognition engine (shore). In: Microelectronic Systems, pp. 243–252. Springer (2011)
Hammer, T.: Mental health and social exclusion among unemployed youth in scandinavia. a comparative study. Intl. Jrnl. of Social Welfare 9(1), 53–63 (2000)
Pan, X., Gillies, M., Barker, C., Clark, D.M., Slater, M.: Socially anxious and confident men interact with a forward virtual woman: An experiment study. PLoS ONE 7(4) (2012) e32931
Damian, I., Baur, T., André, E.: Investigating social cue-based interaction in digital learning games. In: Proc. of the 8th International Conference on the Foundations of Digital Games, SASDG (2013)
Porayska-Pomsta, K., Anderson, K., Damian, I., Baur, T., André, E., Bernardini, S., Rizzo, P.: Modelling users’ affect in job interviews: Technological demo. In: Carberry, S., Weibelzahl, S., Micarelli, A., Semeraro, G. (eds.) UMAP 2013. LNCS, vol. 7899, pp. 353–355. Springer, Heidelberg (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Baur, T., Damian, I., Lingenfelser, F., Wagner, J., André, E. (2013). NovA: Automated Analysis of Nonverbal Signals in Social Interactions. In: Salah, A.A., Hung, H., Aran, O., Gunes, H. (eds) Human Behavior Understanding. HBU 2013. Lecture Notes in Computer Science, vol 8212. Springer, Cham. https://doi.org/10.1007/978-3-319-02714-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-02714-2_14
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02713-5
Online ISBN: 978-3-319-02714-2
eBook Packages: Computer ScienceComputer Science (R0)