More Than Words: Inference of Socially Relevant Information from Nonverbal Vocal Cues in Speech

  • Alessandro Vinciarelli
  • Hugues Salamin
  • Gelareh Mohammadi
  • Khiet Truong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6456)


This paper presents two examples of how nonverbal communication can be automatically detected and interpreted in terms of social phenomena. In particular, the presented approaches use simple prosodic features to distinguish between journalists and non-journalists in media, and extract social networks from turn-taking to recognize roles in different interaction settings (broadcast data and meetings). Furthermore, the article outlines some of the most interesting perspectives in this line of research.


Social Signal Processing Turn-Taking Prosody Nonverbal Behavior Roles Personality Speaking Style 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ajmera, J.: Robust audio segmentation. PhD thesis, Ecole Polytechnique Federale de Lausanne, EPFL (2004)Google Scholar
  2. 2.
    Bishop, C.M.: Pattern recognition and machine learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  4. 4.
    Blaauw, E.: The contribution of prosodic boundary markers to the perceptual difference between read and spontaneous speech. Speech Communication 14, 359–375 (1994)CrossRefGoogle Scholar
  5. 5.
    Boersma, P.: Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound. In: Proceedings of the Institute of Phonetic Sciences, Amsterdam, vol. 17, pp. 97–110 (1993)Google Scholar
  6. 6.
    Carletta, J.E., Ashby, S., Bourban, S., Flynn, M., Guillemot, M., Hain, T., Kadlec, J., Karaiskos, V., Kraaij, W., Kronenthal, M., Lathoud, G., Lincoln, M., Lisowska, A., McCowan, I., Post, W., Reidsma, D., Wellner, P.: The AMI meeting corpus: A pre-announcement. In: Renals, S., Bengio, S. (eds.) MLMI 2005. LNCS, vol. 3869, pp. 28–39. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Cios, K.J., Pedrycz, W., Swiniarski, R.W., Kurgan, L.A.: Data mining: a knowledge discovery approach. Springer, Heidelberg (2007)zbMATHGoogle Scholar
  8. 8.
    Jayagopi, D., Hung, H., Yeo, C., Gatica-Perez, D.: Modeling dominance in group conversations from non-verbal activity cues. IEEE Transactions on Audio, Speech and Language Processing 17(3), 501–513 (2009)CrossRefGoogle Scholar
  9. 9.
    Laan, G.P.M.: The contribution of intonation, segmental durations, and spectral features to perception of a spontaneous and a read speaking style. Speech Communication 22, 43–65 (1997)CrossRefGoogle Scholar
  10. 10.
    Llisterri, J.: Speaking style in speech research. In: ELSNET/ESCA/SALT Workshop on Integrating Speech and Natural Language (1992)Google Scholar
  11. 11.
    Massey Jr. F.J.: The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Association, 68–78 (1951)Google Scholar
  12. 12.
    Vinciarelli, A.: Capturing order in social interactions. IEEE Signal Processing Magazine 26(5), 133–137 (2009)CrossRefGoogle Scholar
  13. 13.
    Vinciarelli, A., Favre, S.: Broadcast news story segmentation using Social Network Analysis and Hidden Markov Models. In: Proceedings of ACM International Conference on Multimedia, pp. 261–264 (2007)Google Scholar
  14. 14.
    Vinciarelli, A., Pantic, M., Bourlard, H.: Social Signal Processing: Survey of an emerging domain. Image and Vision Computing Journal 27(12), 1743–1759 (2009)CrossRefGoogle Scholar
  15. 15.
    Weintraub, M., Taussing, K., Hunicke-Smith, K., Snodgrass, A.: Effect of speak- ing style on LVCSR performance. In: Proceedings of the International Conference on Spoken Language Processing (ICSLP), pp. 16–19 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Alessandro Vinciarelli
    • 1
    • 2
  • Hugues Salamin
    • 1
  • Gelareh Mohammadi
    • 2
    • 3
  • Khiet Truong
    • 4
  1. 1.University of GlasgowGlasgowUK
  2. 2.Idiap Research InstituteMartignySwitzerland
  3. 3.EPFLLausanneSwitzerland
  4. 4.University of TwenteEnschedeThe Netherlands

Personalised recommendations