Abstract
In this paper, we propose a computational model for social interaction between three people in a conversation, and demonstrate results using human video motion synthesis. We utilised semi-supervised computer vision techniques to label social signals between the people, like laughing, head nod and gaze direction. Data mining is used to deduce frequently occurring patterns of social signals between a speaker and a listener in both interested and not interested social scenarios, and the mined confidence values are used as conditional probabilities to animate social responses. The human video motion synthesis is done using an appearance model to learn a multivariate probability distribution, combined with a transition matrix to derive the likelihood of motion given a pose configuration. Our system uses social labels to more accurately define motion transitions and build a texture motion graph. Traditional motion synthesis algorithms are best suited to large human movements like walking and running, where motion variations are large and prominent. Our method focuses on generating more subtle human movement like head nods. The user can then control who speaks and the interest level of the individual listeners resulting in social interactive conversational agents.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. In: Proc. of ACM SIGGRAPH, July 2002, vol. 21(3), pp. 473–482 (2002)
Agrawal, A., Imielinski, T., Swami, A.: Mining association rules between sets of items in large databases. In: Proc. of the 1993 ACM SIGMOD Int. Conf. on Management of Data SIGMOD 1993 (1993)
Szummer, M., Picard, R.: Temporal texture modeling. In: Proc. of IEEE Int. Conf. on Image Processing, pp. 823–826 (1996)
Efros, A., Leung, T.: Texture synthesis by non-paramteric sampling. In: Int. Conf. on Computer Vision, pp. 1033–1038 (1999)
Kwatra, V., Schodl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures. In: ACM Trans. on Graphics, SIGGRAPH 2003, vol. 22(3), pp. 277–286 (2003)
Bhat, K., Seitz, S., Hodgins, J., Khosla, P.: Flow-based video synthesis and editing. In: ACM Trans. on Graphics, SIGGRAPH 2004 (2004)
Troje, N.F.: Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2, 371–387 (2002)
Pullen, K., Bregler, C.: Synthesis of cyclic motions with texture (2002)
Okwechime, D., Bowden, R.: A generative model for motion synthesis and blending using probability density estimation. In: Fifth Conference on Articulated Motion and Deformable Objects, Mallorca, Spain, (July 9-11, 2008)
Tanco, L.M., Hilton, A.: Realistic synthesis of novel human movements from a database of motion captured examples. In: Proc. of the IEE Workshop on Human Motion (HUMO 2000) (2000)
Arikan, O., Forsyth, D., O’Brien, J.: Motion synthesis from annotation. In: ACM Transaction on Graphics, SIGGRAPH 2003, July 2003, vol. 22(3), pp. 402–408 (2003)
Okwechime, D., Ong, E.J., Bowden, R.: Real-time motion control using pose space probability density estimation. In: IEE Int. Workshop on Human-Computer Interaction (2009)
Treuille, A., Lee, Y., Popovic, Z.: Near-optimal character animation with continuous control. In: Proceedings of SIGGRAPH 2007, vol. 26(3) (2007)
Rachel, H., Gleicher, M.: Parametric motion graph. In: 24th Int. Symposium on Interactive 3D Graphics and Games, pp. 129–136 (2007)
Shin, H., Oh, H.: Fat graphs: Constructing an interactive character with continuous controls. In: Proc. of the 2006 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, vol. 298 (2006)
Balci, K., Akarun, L.: Generating motion graphs from clusters of individual poses. In: 24th Int. Symposium on Computer and Information Sciences, pp. 436–441 (2009)
Lee, J., Chai, J., Reitsma, P., Hodgins, J., Pollard, N.: Interactive control of avatars animated with human motion data. ACM Trans. on Graphics 21, 491–500 (2002)
Schödl, A., Szeliski, R., Salesin, D., Essa, I.: Video textures. In: Proc. of the 27th Annual Conf. on Computer Graphics and Interactive Techniques, SIGGRAPH 2000, pp. 489–498. ACM Press/Addison-Wesley Publishing Co., New York (2000)
Flagg, M., Nakazawa, A., Zhang, Q., Kang, S., Ryu, Y., Essa, I., Rehg, J.: Human video textures. In: Proc. of the 2009 Symposium on Interactive 3D Graphics and Games, pp. 199–206. ACM, New York (2009)
Ekman, P., Friesen, W.: Facial action coding system. Consulting Psychologists Press, Palo Alto (1977)
Argyle, M.: Bodily communication. Methuen (1987)
Beaudoin, P., Coros, S., van de Panne, M., Poulin, P.: Motion-motif graphs. In: Proc. of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 117–126 (2008)
Pentland, A.: A computational model of social signaling. In: 18th Int. Conf. on Pattern Recognition, ICPR (2006)
Mertins, A., Rademacher, J.: Frequency-warping invariant features for automatic speech recognition. In: Proceedings of 2006 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006, vol. 5 (2006)
Viola, P., Jones, M.: Rapid Object Detection using a Boosted Cascade of Simple Features. In: Proc. IEEE CVPR 2001 (2002)
Ong, E.J., Lan, Y., Theobald, B.J., Harvey, R., Bowden, R.: Robust facial feature tracking using selected multi-resolution linear predictors. In: Int. Conf. Computer Vision. ICCV 2009 (2009)
Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Proc. of 20th Int. Conf. on Very Large Data Bases, VLDB 1994, pp. 487–499 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Okwechime, D., Ong, EJ., Gilbert, A., Bowden, R. (2011). Social Interactive Human Video Synthesis. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision – ACCV 2010. ACCV 2010. Lecture Notes in Computer Science, vol 6492. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19315-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-19315-6_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-19314-9
Online ISBN: 978-3-642-19315-6
eBook Packages: Computer ScienceComputer Science (R0)