A Generative Score Space for Statistical Dialog Characterization in Social Signalling

  • Anna Pesarin
  • Paolo Calanca
  • Vittorio Murino
  • Marco Cristani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6218)


The analysis of human conversations under a social signalling perspective recently raised the joint attention of pattern recognition and psychology researchers. In particular, the dialog classification represents an appealing recent application whose aim is to go beyond the meaning of the spoken words, focusing instead on the way the sentences are pronounced by capturing natural (or hidden) characteristics, such the mood of the conversation. An effective strategy to face this issue is to encode the turn-taking dynamics in a generative model, whose structure is composed by conditional dependencies among first-order Markov processes. In this paper, we follow this strategy, investigating how to boost the classification performances of this model and of the related higher-order Markov extensions, through the definition of a novel generative score space. Generative score spaces are employed to increase generative classification in a discriminative way, also allowing a deep understanding of the processed data through the use of standard pattern recognition strategies. Experiments on real data certify the goodness of our intuition.


social signalling dialogue analysis observed influence model 


  1. 1.
    Pantic, M., Pentland, A., Nijholt, A.: Special issue on human computing. IEEE Trans. on Systems, Man, and Cybernetics, Part B 39(1) (2009)Google Scholar
  2. 2.
    Vinciarelli, A., Pantic, M., Bourlard, H.: Social signal processing: Survey of an emerging domain. Image and Vision Computing 27(12), 1743–1759 (2009)CrossRefGoogle Scholar
  3. 3.
    Ambady, N., Rosenthal, R.: Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin 11(2), 256–274 (1992)CrossRefGoogle Scholar
  4. 4.
    Jayagopi, D., Hung, H., Yeo, C., Gatica-Perez, D.: Modeling dominance in group conversations using nonverbal activity cues. Trans. Audio, Speech and Lang. Proc. 17(3), 501–513 (2009)CrossRefGoogle Scholar
  5. 5.
    Choudhury, T., Basu, S.: Modeling conversational dynamics as a mixed-memory markov process. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems 17, pp. 281–288. MIT Press, Cambridge (2005)Google Scholar
  6. 6.
    Vinciarelli, A.: Speakers role recognition in multiparty audio recordings using social network analysis and duration distribution modeling. IEEE Transactions on Multimedia 9(6), 1215–1226 (2007)CrossRefGoogle Scholar
  7. 7.
    Basu, S., Choudhury, T., Clarkson, B., Pentland, A.: Learning human interaction with the influence model. MIT MediaLab, Tech. Rep. 539 (2001)Google Scholar
  8. 8.
    Pentland, A.: Social signal processing. IEEE Signal Processing Magazine 24(4), 108–111 (2007)CrossRefGoogle Scholar
  9. 9.
    Curhan, J., Pentland, A.: Thin slices of negotiation: Predicting outcomes from conversational dynamics within the first five minutes. Journal of Applied Psychology 92, 802–811 (2007)CrossRefGoogle Scholar
  10. 10.
    Cristani, M., Pesarin, A., Drioli, C., Perina, A., Tavano, A., Murino, V.: Auditory dialog analysis and understanding by generative modelling of interactional dynamics. In: Second IEEE Workshop on CVPR4HB, Miami, Florida (2009)Google Scholar
  11. 11.
    Zhang, D., Gatica-Perez, D., Bengio, S., McCowan, I., Lathoud, G.: Modeling individual and group actions in meetings with layered hmms. IEEE Transactions on Multimedia (May 2005)Google Scholar
  12. 12.
    Asavathiratham, C.: A tractable representation for the dynamics of networked markov chain. Ph.D. dissertation, Dept. of ECS, MIT (2000)Google Scholar
  13. 13.
    Saul, L., Jordan, M.: Mixed memory markov models: Decomposing complex stochastic processes as mixtures of simpler ones. Machine Learning 37(1), 75–87 (1999)zbMATHCrossRefGoogle Scholar
  14. 14.
    Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Proceedings of the 1998 conference on Advances in neural information processing systems II, pp. 487–493. MIT Press, Cambridge (1999)Google Scholar
  15. 15.
    Tsuda, K., Kawanabe, M., Rätsch, G., Sonnenburg, S., Müller, K.: A new discriminative kernel from probabilistic models. Neural Comput. 14(10), 2397–2414 (2002)zbMATHCrossRefGoogle Scholar
  16. 16.
    Bicego, M., Pekalska, E., Tax, D., Duin, R.: Component-based discriminative classification for hidden markov models. Pattern Recogn. 42(11), 2637–2648 (2009)zbMATHCrossRefGoogle Scholar
  17. 17.
    Perina, A., Cristani, M., Castellani, U., Murino, V., Jojic, N.: Free energy score space. In: Advances in Neural Information Processing Systems, vol. 22, pp. 1428–1436 (2009)Google Scholar
  18. 18.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. John Wiley and Sons, Chichester (2001)zbMATHGoogle Scholar
  19. 19.
    Smith, N., Gales, M.: Speech recognition using svms. In: NIPS, pp. 1197–1204 (2001)Google Scholar
  20. 20.
    Duin, R., Juszczak, P., Paclík, P., Pekalska, E., DeRidder, D., Tax, D.: Prtools version 4.1: A matlab toolbox for pattern recognition. Internet (2004),
  21. 21.
    Fukunaga, K.: Statistical Pattern Recognition, 2nd edn. Academic Press, London (1990)zbMATHGoogle Scholar
  22. 22.
    Lissack, T., Fu, K.: Error estimation in pattern recognition via l-distance between posterior density functions. IEEE Trans. Inform. Theory 22, 34–35 (1976)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Anna Pesarin
    • 1
  • Paolo Calanca
    • 1
    • 2
  • Vittorio Murino
    • 1
  • Marco Cristani
    • 1
    • 2
  1. 1.Dipartimento di InformaticaUniversity of VeronaItaly
  2. 2.Istituto Italiano di Tecnologia (IIT)GenovaItaly

Personalised recommendations