Abstract
Social dynamics, such as human interaction is important for understanding how a conclusion was reached in a meeting and determining whether the meeting was well organized. In this paper, a multimodal approach is proposed to infer human semantic interactions in meeting discussions. The human interaction, such as proposing an idea, giving comments, expressing a positive opinion, etc., implies user role, attitude, or intention toward a topic. Our approach infers human interactions based on a variety of audiovisual and high-level features, e.g., gestures, attention, speech tone, speaking time, interaction occasion, and information about the previous interaction. Four different inference models including Support Vector Machine (SVM), Bayesian Net, Naïve Bayes, and Decision Tree are selected and compared in human interaction recognition. Our experimental results show that SVM outperforms other inference models, we can successfully infer human interactions with a recognition rate around 80%, and our multimodal approach achieves robust and reliable results by leveraging on the characteristics of each single modality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Yu, Z., Nakamura, Y.: Smart Meeting Systems: A Survey of State-of-the-Art and Open Issues. ACM Computing Surveys 41(3) (2009)
Dey, A.K., Salber, D., Abowd, G.D., Futakawa, M.: The Conference Assistant: Combining Context-Awareness with Wearable Computing. In: Proc. ISWC 1999, pp. 21–28 (1999)
Chen, H., Finin, T., Joshi, A.: A Context Broker for Building Smart Meeting Rooms. In: Proc. of the Knowledge Representation and Ontology for autonomous systems symposium (AAAI spring symposium), pp. 53–60 (2004)
Kim, N., Han, S., Kim, J.W.: Design of Software Architecture for Smart Meeting Space. In: Proc. PerCom 2008, pp. 543–547 (2008)
Yu, Z., Ozeki, M., Fujii, Y., Nakamura, Y.: Towards Smart Meeting: Enabling Technologies and a Real-World Application. In: Proc. ICMI 2007, pp. 86–93 (2007)
Geyer, W., Richter, H., Abowd, G.D.: Towards a Smarter Meeting Record – Capture and Access of Meetings Revisited. Multimedia Tools and Applications 27(3), 393–410 (2005)
Chiu, P., Kapuskar, A., Reitmeier, S., Wilcox, L.: Room with a Rear View: Meeting Capture in a Multimedia Conference Room. IEEE Multimedia 7(4), 48–54 (2000)
Janin, A., et al.: The ICSI Meeting Project: Resources and Research. In: Proc. of NIST ICASSP Meeting Recognition Workshop (2004)
Junuzovic, S., Hegde, R., Zhang, Z., Chou, P.A., Liu, Z., Zhang, C.: Requirements and Recommendations for an Enhanced Meeting Viewing Experience. In: Proc. ACM Multimedia 2008, pp. 539–548 (2008)
DiMicco, J.M., et al.: The Impact of Increased Awareness While Face-to-Face. Human-Computer Interaction 22(1), 47–96 (2007)
Stiefelhagen, R., Chen, X., Yang, J.: Capturing Interactions in Meetings with Omnidirectional Cameras. International Journal of Distance Education Technologies 3(3), 34–47 (2005)
Yu, H., Finke, M., Waibel, A.: Progress in Automatic Meeting Transcription. In: Proc. of 6th European Conference on Speech Communication and Technology (Eurospeech-1999), vol. 2, pp. 695–698 (1999)
Otsuka, K., Sawada, H., Yamato, J.: Automatic Inference of Cross-modal Nonverbal Interactions in Multiparty Conversations. In: Proc. ICMI 2007, pp. 255–262 (2007)
Nijholt, A., Rienks, R.J., Zwiers, J., Reidsma, D.: Online and Off-line Visualization of Meeting Information and Meeting Support. The Visual Computer 22(12), 965–976 (2006)
Nijholt, A., Zwiers, J., Peciva, J.: Mixed reality participants in smart meeting rooms and smart home environments. Personal and Ubiquitous Computing 13(1), 85–94 (2009)
Sumi, Y., et al.: Collaborative capturing, interpreting, and sharing of experiences. Personal and Ubiquitous Computing 11(4), 265–271 (2007)
Hillard, D., Ostendorf, M., Shriberg, E.: Detection of Agreement vs. Disagreement in Meetings: Training with Unlabeled Data. In: Proc. HLT-NAACL 2003, pp. 34–36 (2003)
Tomobe, H., Nagao, K.: Discussion Ontology: Knowledge Discovery from Human Activities in Meetings. In: Washio, T., Satoh, K., Takeda, H., Inokuchi, A. (eds.) JSAI 2006. LNCS, vol. 4384, pp. 33–41. Springer, Heidelberg (2007)
Kim, T., Chang, A., Holland, L., Pentland, A.: Meeting Mediator: Enhancing Group Collaboration using Sociometric Feedback. In: Proc. CSCW 2008, pp. 457–466 (2008)
Garg, N.P., Favre, S., Salamin, H., Tur, D.H., Vinciarelli, A.: Role Recognition for Meeting Participants: an Approach Based on Lexical Information and Social Network Analysis. In: Proc. ACM Multimedia 2008, pp. 693–696 (2008)
Yu, Z., Aoyama, H., Ozeki, M., Nakamura, Y.: Collaborative Capturing and Detection of Human Interactions in Meetings. In: Adjunct Proc. of PERVASIVE 2008, pp. 65–69 (2008)
Julius speech recognition engine (2008), http://julius.sourceforge.jp/en/
Rabiner, L.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc. IEEE 77(2), 257–286 (1989)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
Weka (2008), http://www.cs.waikato.ac.nz/ml/weka/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yu, Z., Yu, Z., Ko, Y., Zhou, X., Nakamura, Y. (2009). Inferring Human Interactions in Meetings: A Multimodal Approach. In: Zhang, D., Portmann, M., Tan, AH., Indulska, J. (eds) Ubiquitous Intelligence and Computing. UIC 2009. Lecture Notes in Computer Science, vol 5585. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02830-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-02830-4_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02829-8
Online ISBN: 978-3-642-02830-4
eBook Packages: Computer ScienceComputer Science (R0)