Abstract
Generation of animated human figures especially in crowd scenes has many applications in such domains as the special effects industry, computer games or for the simulation of the evacuation from crowded areas. Current systems allow for partially automatic generation of scenes involving a few interacting characters but expensive manual labour is still necessary in order to enrich the characters’ behaviour repertoire. In this chapter we explore the possibility of applying reinforcement learning to acquire new high-level actions for animated characters. The chosen algorithm is the deterministic version of Q-learning. This allows for easy definition of the task, since only the ultimate goal of the learning agent must be defined. Generated actions can then be used to enrich the animation produced by an animation system. Results achieved when training agents with forward and inverse kinematics control are also demonstrated and compared.
The work was partially supported by the British Council/KBN (Polish State Committee for Scientific Research) grant, project number 239/2002.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson F. C. and Pandy M. G., Three-Dimensional Computer Simulation Of Gait, Bioengineering Conference Big Sky, Montana, June 16–20, 1999
Arvo J. and Kirk D., A survey of ray tracing acceleration techniques. In Introduction to Ray Tracing Course Notes, Proceedings of ACM SIGGRAPH’88, 1989.
Bertsekas D.P. and Tsitsiklis J.N., Neuro-Dynamic Programming, Athena Scientific, 1996
Blumberg B., Downie M. Ivanov Y. Berlin M. Johnson M. P. Tomlinson B., Integrated learning for interactive synthetic characters, ACM Transactions on Graphics, Vol. 21, Iss. 3 July 2002, pp. 417–426
Brogan, D.C. and Hodgins, J. K. Group Behaviors for Systems with Significant Dynamics, Autonomous Robots 4 (1), pp. 137–153, 1997
Burke R., Isla D., Downie M., Ivanov Y., Blumberg, B., Creature smarts: The art and architecture of a virtual brain. In Proceedings of the Computer Game Developers Conference, 2001
Cassell, J., Vilhjâlmsson, H., Bickmore, T., BEAT: the Behavior Expression Animation Toolkit, Proceedings of SIGGRAPH ’01, pp.477–486, August 12–17, Los Angeles, CA, 2001
Davison D. E. and Bortoff S. A., Acrobot software and hardware guide, Technical Report Number 9406, Systems Control Group, University of Toronto, Toronto, Ontario M5S 1A4, Canada, June 1994
Dorigo M., Maniezzo V., Colorni A., Positive Feedback as a Search Strategy, Tech. Rep. no 91–016, Politecnico di Milano, Italy, 1991
Downie M., Behavior, Animation and Music: The Music and Movement of Synthetic Characters, M.Sc. Thesis, The Media Lab. MIT, 2000
Downie M., Tomlinson B., Blumberg B., Developing an aesthetic: character-based interactive installations, Computer Graphics Vol. 36, Issue 2, May 2002
Faloutsos P., Composable Controllers for Physics-Based Character Animation, Ph.D. Thesis, Department of Computer Science, University of Toronto,2002.
Faloutsos P., van de Panne M., Terzopoulos D., The virtual stuntman: dynamic characters with a repertoire of autonomous motor skills, Computers and Graphics, Volume 25, Issue 6, pp. 933953, December, 2001
Faloutsos P., van de Panne M., Terzopoulos D., Composable Controllers for Physics-Based Character Animation, ACM SIG-GRAPH 2001, Los Angeles, California, 12–17 August 2001
Flake S., Geiger C., Küster J. M., Towards UML-based Analysis and Design of Multi-Agent Systems, in Proceedings of International NAISO Symposium on Information Science Innovations in Engineering of Natural and Artificial Intelligent Systems (ENAIS’2001), Dubai, March 2001
Forte P., Szarowicz A., The Application of AI Techniques for Automatic Generation of Crowd Scenes, The 11`h International Symposium on Intelligent Information Systems, Advances in Soft Computing, Physica-Verlag: pp209–216, Sopot, Poland, 2002
Francik J., A Framework for Program Control of Animation of Human Avatars. Studia Informatica, Vol. 24, No. 4 (56), 2003, pp. 55–65.
Funge J. D., Making Them Behave: Cognitive Models for Computer Animation, PhD thesis, Department of Computer Science, University of Toronto, 1998
Funge J. D., AI for Games and Animation. A Cognitive Modeling Approach, A K Peters Natick, Massachusetts, 1999
Funge J. D., Tu X., Terzopoulos D., Cognitive Modeling: Knowledge, reasoning and planning for intelligent characters, Computer Graphics Proceedings: SIGGRAPH 99, Aug 1999
Gottschalk S., Separating axis theorem. Technical report TR96024, Dept. of Computer Science, UNC, Chapel Hill, 1996.
Gottschalk S., Lin M. C. and Manocha D., OBBTree: A Hierarchical Structure for Rapid Interference Detection. Proceedings of ACM SIGGRAPH, New Orleans, Lo, 1996, pp. 171— 180.
Grzeszczuk R. and Terzopoulos D., Automated Learning of MuscleActuated Locomotion Through Control Abstraction, Proceedings of SIGGRAPH 95 ACM SIGGRAPH, pp. 63–70, 1995
Grzeszczuk R., PhD Thesis, NeuroAnimator: Fast Neural Network Emulation and Control of Physics-Based Models, Dept. of Computer Science, University of Toronto, May 1998
Grzeszczuk R., Terzopoulos D., Hinton G., NeuroAnimator: Fast Neural Network Emulation and Control of Physics-Based Models, proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, pp. 9–20, Orlando, Florida, 1998
Haykin S., Neural Networks a Comprehensive Foundation, Prentice Hall, 1999
Hodgins, J. K., Wooten, W. L., Brogan, D. C., O’Brien, J. F., Animating Human Athletics, Proceedings of Siggraph ‘85, In Computer Graphics, pp 71–78, 1995
Hodgins, J. K. and Pollard, N. S., Adapting Simulated Behaviors For New Characters, SIGGRAPH 97, Los Angeles, CA, 1997
Isla D., Burke R., Downie M., Blumberg B., A Layered Brain Architecture for Synthetic Creatures, pp. 1051–1058, in Proceedings of Seventeenth Joint Conference on Artificial Conference IJCAI-01, 4–10 August, Seattle, USA, 2001
Koza, J.R., On the Programming of Computers by Means of Natural Selection Artificial System, MIT Press, 1992
Lach E., Genetic Programming in the Animation of Human Avatars. 3rd International PhD Students’ Workshop on Control and Information Technology 1WCIT’03, Gliwice, Poland, pp. 43–48.
Laszlo J., van de Panne M., Fiume E., Limit Cycle Control and its Application to the Animation of Balancing and Walking, Proceedings of SIGGRAPH 1996, (New Orleans, LA, August 4–9, 1996 ), in Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, pp. 155–162, 1996
Metoyer, R. A., Hodgins, J. K., Animating Athletic Motion Planning By Example. Proceedings of Graphics Interface 2000, pp. 61–68, Montreal, Quebec, Canada, May 15–17, 2000
Mitchell T.M., Machine Learning, McGraw Hill, 1997
Musse S. R, Thalmann D., Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE Trans. on Visualization and Computer Graphics, 2001, Vol.7, No2, pp. 152–164.
Mylopoulos J., Kolp M., Castro J., UML for Agent-Oriented Software Development: The Tropos Proposal, in Proceedings of the Fourth International Conference on the Unified Modeling Language, Toronto, Canada, October 2001
O’Sullivan C., Cassell J., Vilhjâlmsson H., Dingliana J., Dobbyn S., McNamee B., Peters C., Giang T., Levels of Detail for Crowds and Groups, Computer Graphics Forum, Vol. 21 (4) pp 733–742, November 2002
Pandy M. G. and Anderson F. C., Three-Dimensional Computer Simulation Of Jumping and Walking Using the Same Model, in Proceedings of the VIIth International Symposium on Computer Simulation in Biomechanics, August 1999
Pollard, N. S. and Hodgins, J. K.. Adapting Behaviors to New Environments, Characters, and Tasks. Yale Workshop on Adaptive and Learning Systems, 1998
Rao A. S., Georgeff M. O., BDI Agents: From Theory to Practice, in Proceedings of the First International Conference on Multiagent Systems ICMAS95, June 12–14, 1995
Raupp Musse, S. and Thalmann, D., A Hierarchical Model for Real Time Simulation of Virtual Human Crowds, IEEE Transactions on Visualization and Computer Graphics, V. 7, N.2, pp. 152164, April-June, 2001
Reynolds, C. W., Flocks, herds, and schools: A distributed behavioral model, Computer Graphics, SIGGRAPH ’87 Conference Proceedings, vol. 21 (4) pp 25–34, ACM SIGGRAPH 1987
Schaal S. and Atkeson C., Robot juggling: An implementation of memory-based learning. Control Systems Magazine, 14, 1994
Schraudolph N. N., Dayan P., Sejnowski T. J, Temporal difference learning of position evaluation in the game of Go. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances in Neural Information Processing Systems 6, pp. 817–824, Morgan Kaufmann, San Mateo, CA, 1994
Sutton R.S.and Barto A.G., Reinforcement Learning: an introduction, MIT Press, 1998
A. Szarowicz, P. Forte, “Combining Intelligent Agents and Animation”, AIxIA 2003 - Eighth National Congress on AI, September 22–26, Pisa, Italy, 2003
Szarowicz, A., Amiguet-Vercher, J., Forte, P., Briggs, J., Gelepithis, P.A.M., Remagnino, P., The Application of AI to Automatically Generated Animation, in AI2001: Advances in Artificial Intelligence, 14th Australian Joint Conference on Artificial
Intelligence, AI’01, Springer LNAI 2256, pp 487–494, Adelaide, Australia, 2001
Terzopoulos D., Tu X., Grzeszczuk R., Artificial Fishes: Autonomous Locomotion, Perception, Behavior, and Learning in a Simulated Physical World, Artificial Life, 1 (4) pp. 327–351, 1994
Terzopoulos D., Rabie T., Grzeszczuk R., Perception and Learning in Artificial Animals, Artificial Life V: Proc. 5th Inter. Conf. on the Synthesis and Simulation of Living Systems, Nara, Japan, 1996
Tesauro G., TD-Gammon, a self-teaching backgammon program achieves master-level play. Neural Computation, 6(2) pp.215219, 1994
Thrun S., Learning to play the game of chess. In G. Tesauro, D. S. Touretzky, and T. K. Leen, editors, Advances in Neural Information Processing Systems 7, MIT Press Cambridge, MA, 1995
Tomlinson B., Blumberg B., Nain D., Expressive autonomous cinematography for interactive virtual environments, Proceedings of the Fourth International Conference on Autonomous Agents, B.rcelona, Spain, pp. 317–324, 2000
Tomlinson B., Downie M., Berlin M., Gray J., Lyons D., Cochran J., Blumberg B., Leashing the AlphaWolves: mixing user direction with autonomous emotion in a pack of semi-autonomous virtual characters, Proceedings of the ACM SIGGRAPH symposium on Computer Animation, San Antonio, Texas, pp. 7–14, 2002
Tu X. and Terzopoulos D., Artificial Fishes: Physics, Locomotion, Perception, Behavior, Proc. of ACM SIGGRAPH’94, Orlando, FL, in ACM Computer Graphics Proceedings, p. 43–50, 1994
van de Panne M., Laszlo J., Huang P., Faloutsos P., Dynamic Human Simulation: Towards Agile Animated Characters, Proceedings of the IEEE International Conference on Robotics and Automation 2000, pp. 682–687, San Francisco, CA, 2000
Yoon S.Y., Blumberg B. M., Schneider G. E., Motivation driven learning for interactive synthetic characters. In Proceedings of Autonomous Agents 2000
Wan T. R. and Tang W., Simulating Virtual Character’s Learning Behaviour as An Evolutionary Process Using Genetic Algorithms, Journal of WSCG, Volume 10, Number 3, 2002
Watkins, C.J.C.H., Learning from delayed rewards, PhD thesis, University of Cambridge, Psychology Department, 1989
Winikoff M., Padgham L., and Harland J., Simplifying the Development of Intelligent Agents, in AI2001: Advances in Artificial Intelligence, 14th Australian Joint Conference on Artificial Intelligence, LNAI 2256, pp. 557–568, Adelaide, December 2001
Wood M. and DeLoach S.A., An Overview of the Multiagent Systems Engineering Methodology, in Agent-Oriented Software Engineering. P. Ciancarini, M. Wooldridge, (Eds.) LNAI Vol. 1957, Springer Verlag, Berlin, January 2001
Wooldridge M., Jennings N.R., Kinny, D., The Gaia Methodology for Agent-Oriented Analysis and Design, Autonomous Agents and Multi-Agent Systems, Vol. 3, No. 3, pp. 285–312, 2000
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Szarowicz, A., Mittmann, M., Francik, J. (2005). Intelligent Action Acquisition for Animated Learning Agents. In: Design of Intelligent Multi-Agent Systems. Studies in Fuzziness and Soft Computing. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-44516-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-44516-6_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-06177-6
Online ISBN: 978-3-540-44516-6
eBook Packages: EngineeringEngineering (R0)