Abstract
It/I is a two-character theater play where the human character I is taunted and played by a vision-based, autonomous computerized character — called It — which controls computer-graphics, sound, and stage lights. Unlike previous immersive interactive systems, the computer vision system recognizes the human character’s actions by considering not only tracking and gestural information and the character’s internal variables but also the context provided by the current situation in the story. This paper focuses on a methodology to represent and recognize the human and computer characters’ actions that is based on interval scripts, a paradigm that uses Allen’s temporal primitives to describe the relationships among the different actions and reactions. The system was tested in six public performances held at the MIT Media Laboratory in 1997, when the computer graphics character ran automatically during the 30-minute duration of the play.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
James F. Allen. Towards a general theory of action and time. Artificial Intelligence, 23:123–154, 1984.
Joseph Bates, A. Bryan Loyall, and W. Scott Reilly. An architecture for action, emotion, and social behavior. In Proceedings of the Fourth European Workshop on Modeling Autonomous Agents in a Multi-Agent World, S Martino al Cimino, Italy, July 1992.
Bruce Blumberg. Old Tricks, New Dogs: Ethology and Interactive Creatures. PhD thesis, M.I.T. Media Arts and Sciences Program, 1996.
Aaron F. Bobick. Movement, activity, and action: The role of knowledge in the perception of motion. Phil. Trans. Royal Society London B, 352:1257–1265, 1997.
Aaron F. Bobick, Stephen Intille, Jim Davis, Freedom Baird, Claudio Pinhanez, Lee Campbell, Yuri Ivanov, Arjan Schutte, and Andy Wilson. The KidsRoom: A perceptually-based interactive and immersive story environment. Technical Report 398, M.I.T. Media Laboratory Perceptual Computing Section, November 1996.
M. Cecelia Buchanan and Polle T. Zellweger. Automatic temporal layout mechanisms. In Proc. of ACM Multimedia’93, pages 341–350, Ahaheim, California, August 1993.
James W. Davis and A. Bobick. The representation and recognition of human movement using temporal templates. In Proc. of CVPR’97, pages 928–934, June 1997.
James W. Davis and Aaron F. Bobick. Virtual PAT: a virtual personal aerobics trainer. Technical Report 436, M.I.T. Media Laboratory Perceptual Computing Section, January 1998.
Rei Hamakawa and Jun Rekimoto. Object composition and playback models for handling multimedia data. In Proc. of ACM Multimedia’ 93, pages 273–281, Ahaheim, California, August 1993.
Yuri Ivanov, Aaron Bobick, and John Liu. Fast lighting independent background subtraction. In Proc. of the IEEE Workshop on Visual Surveillance-VS’98, pages 49–55, Bombay, India, January 1998.
Myron W. Krueger. Artificial Reality II. Addison-Wesley, 1990.
A. K. Mackworth. Consistency in networks of relations. Artificial Intelligence, 8(1):99–118, 1977.
MacroMind Inc. Director’s User Manual. 1990.
Pattie Maes, Trevor Darrell, Bruce Blumberg, and Alex Pentland. The ALIVE system: Full-body interaction with autonomous agents. In Proc. of the Computer Animation’ 95 Conference, Geneva, Switzerland, April 1995.
R. Mann, A. Jepson, and Jeffrey Siskind. The computational perception of scene dynamics. In Proc. of Fourth European Conference in Computer Vision, April 1996.
Itay Meiri. Combining qualitative and quantitative constraints in temporal reasoning. Artificial Intelligence, 87(1-2):343–385, November 1996.
Hans-Hellmut Nagel. A vision of ‘vision and language’ comprises action: An example from road traffic. Artificial Intelligence Review, 8:189–214, 1995.
Ken Perlin and Athomas Goldberg. Improv: A system for scripting interactive actors in virtual worlds. In Proc. of SIGGRAPH’96, August 1996.
Claudio S. Pinhanez. Computer theater. In Proc. of the Eighth International Symposium on Electronic Arts (ISEA’97), Chicago, Illinois, September 1997.
Claudio S. Pinhanez and Aaron F. Bobick. Fast constraint propagation on specialized Allen networks and its application to action recognition and control. Technical report # 456, M.I.T. Media Laboratory Perceptual Computing Section, January 1998.
Claudio S. Pinhanez and Aaron F. Bobick. Human action detection using PNF propagation of temporal constraints. In Proc. of CVPR’ 98, pages 898–904, Santa Barbara, California, June 1998.
Claudio S. Pinhanez and Aaron F. Bobick. “It/I”: A theater play featuring an autonomous computer graphics character. Technical report # 455, M.I.T. Media Laboratory Perceptual Computing Section, January 1998.
Claudio S. Pinhanez, Kenji Mase, and Aaron F. Bobick. Interval scripts: A design paradigm for story-based interactive systems. In CHI’97, pages 287–294, Atlanta, Georgia, March 1997.
Christa Sommerer and Laurent Mignonneau. Art as a living system. Leonardo, 30(5), October 1997.
Naoko Tosa, Hideki Hashimoto, Kaoru Sezaki, Yasuharu Kunii, Toyotoshi Yamada, Kotaro Sabe, Ryosuke Nishino, Hiroshi Harashima, and Fumio Harashima. Network-based neuro-baby with robotic hand. In Proc. of IJCAI’95 Workshop on Entertainment and AI/Alife, Montreal, Canada, August 1995.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pinhanez, C., Bobick, A. (1999). Using Computer Vision to Control a Reactive Computer Graphics Character in a Theater Play. In: Computer Vision Systems. ICVS 1999. Lecture Notes in Computer Science, vol 1542. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49256-9_5
Download citation
DOI: https://doi.org/10.1007/3-540-49256-9_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65459-9
Online ISBN: 978-3-540-49256-6
eBook Packages: Springer Book Archive