Cell Phone Puppets: Turning Mobile Phones into Performing Objects

  • Michael Nitsche
  • Sanjeev Nayak
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7522)


We introduce Puppettime, a digital puppetry project that uses mobile phones as interfaces to control virtual puppetry via motion gestures. The goal of the project is to explore cell phones as performative objects in novel interaction designs. Combining the evolution of mobile devices into tangible interfaces with traditional puppetry, the paper discusses the connections between these domains. It describes the design rationale behind the Puppettime project as well as its implementation and first feedback, focusing on the core thesis that puppetry provides a valuable and underused metaphor for interface design that supports digital entertainment between co-present players.


Puppet tangible user interface virtual character 


  1. 1.
    Wexelblat, A.: Research Challenges in Gesture: Open Issues and Unsolved Problems. In: Wachsmuth, I., Fröhlich, M. (eds.) GW 1997. LNCS (LNAI), vol. 1371, pp. 1–11. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  2. 2.
    Ullmer, B., Ishii, H.: Emerging frameworks for tangible user interfaces. IBM Syst. J. 39(3-4), 915–931 (2000), doi:10.1147/sj.393.0915CrossRefGoogle Scholar
  3. 3.
    Dourish, P.: What we talk about when we talk about context. Personal Ubiquitous Computing 8(1), 19–30 (2004), doi:10.1007/s00779-003-0253-8CrossRefGoogle Scholar
  4. 4.
    Mazalek, A., Chandrasekharan, S., Nitsche, M., Welsh, T., Clifton, P., Quitmeyer, A., Peer, F., Kirschner, F., Athreya, D.: I’m in the Game: Embodied Puppet Interface Improves Avatar Control. In: Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 129–136. ACM, New York (2011)CrossRefGoogle Scholar
  5. 5.
    Vajk, T., Coulton, P., Bamford, W., Edwards, R.: Using a Mobile Phone as a “Wii-like” Controller for Playing Games on a Large Public Display. International Journal of Computer Games Technology (2), 1–7 (2008)Google Scholar
  6. 6.
    Chehimi, F., Coulton, P.: Motion controlled mobile 3D multiplayer gaming. In: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pp. 267–270. ACM, New York (2008)Google Scholar
  7. 7.
    Fan, M., Li, X., Zhong, Y., Tian, L., Shi, Y., Wang, H.: Surprise Grabber: a co-located tangible social game using phone hand gesture. In: Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work, pp. 625–628. ACM, New York (2011)Google Scholar
  8. 8.
    Hinckley, K., Song, H.: Sensor synaesthesia: touch in motion, and motion in touch. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, pp. 801–810. ACM, New York (2011)CrossRefGoogle Scholar
  9. 9.
    Edge, D., Blackwell, A.F.: Bimanual tangible interaction with mobile phones. In: Proceedings of the 3rd International Conference on Tangible and Embedded Interaction, pp. 131–136. ACM, New York (2009)CrossRefGoogle Scholar
  10. 10.
    Jones, E., Alexander, J., Andreou, A., Irani, P., Subramanian, S.: GesText: Accelerometer-Based Gestural Text-Entry Systems. In: Proceedings of CHI 2010, pp. 2173–2182. ACM, New York (2010)Google Scholar
  11. 11.
    Ruiz, J., Li, Y.: DoubleFlip: A Motion Gesture Delimiter for Mobile Interaction. In: CHI 2011: ACM Conference on Human Factors in Computing Systems, pp. 2717–2720. ACM, New York (2011)Google Scholar
  12. 12.
    Rico, J., Brewster, S.: Usable Gestures for Mobile Interfaces: Evaluating Social Acceptability. In: Proceedings of CHI 2010, pp. 887–896. ACM, New York (2010)Google Scholar
  13. 13.
    Sall, A., Grinter, B.: Let’s Get Physical! In, Out and Around the Gaming Circle of Physical Gaming at Home. Computer Supported Cooperative Work (CSCW) 16(1-2), 199–229 (2007)CrossRefGoogle Scholar
  14. 14.
    Blumenthal, E.: Puppetry: A World History. Harry N. Abrams, New York (2005)Google Scholar
  15. 15.
    Bottoni, P., Faralli, S., Labella, A., Malizia, A., Pierro, M., Ryu, S.: CoPuppet: Collaborative Interaction in Virtual Puppetry. In: Adams, R., Gibson, S., Arisona, S.M. (eds.) Transdisciplinary Digital Art. Sound, Vision and the New Screen. CCIS, vol. 7, pp. 326–341. Springer, Heidelberg (2008), doi:10.1007/978-3-540-79486-8_27CrossRefGoogle Scholar
  16. 16.
    Walser, R.: Elements of a Cyberspace Playhouse. Paper Presented at the Proceedings of the National Computer Graphics Association, Anaheim, CA, March 19-22 (1990)Google Scholar
  17. 17.
    Marshall, P., Rogers, Y., Scaife, M.: PUPPET: Playing and Learning in a Virtual World. International Journal of Continuing Engineering Education and Life-long Learning 14(6), 519–531 (2004)CrossRefGoogle Scholar
  18. 18.
    Mayora, O., Costa, C., Papliatseyeu, A.: iTheater Puppets Tangible Interactions for Storytelling. Paper Presented at the Intelligent Technologies for Interactive Entertainment, pp. 110–118. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  19. 19.
    Tillis, S.: The Art of Puppetry in the Age of Media Production. TDR 43(3), 182–195 (1999)CrossRefGoogle Scholar
  20. 20.
    Eide, P.: Digital Puppetry. The Puppetry Journal 60(1), 12–13 (2008)Google Scholar
  21. 21.
    Lu, F., Tian, F., Jiang, Y., Cao, X., Luo, W., Li, G., Zhang, X., Dai, G., Wang, H.: ShadowStory: Creative and Collaborative Digital Storytelling Inspired by Cultural Heritage. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, pp. 1919–1928. ACM, New York (2011)CrossRefGoogle Scholar
  22. 22.
    Kim, S., Zhang, X., Kim, Y.J.: Haptic Puppetry for Interactive Games. In: Pan, Z., Aylett, R.S., Diener, H., Jin, X., Göbel, S., Li, L. (eds.) Edutainment 2006. LNCS, vol. 3942, pp. 1292–1302. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Guiard, Y.: Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. Journal of Motor Behavior 19, 486–517 (1987)CrossRefGoogle Scholar
  24. 24.
    Kipp, M., Nguyen, Q.: Multitouch puppetry: creating coordinated 3D motion for an articulated arm. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, pp. 147–156. ACM, New York (2010)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2012

Authors and Affiliations

  • Michael Nitsche
    • 1
  • Sanjeev Nayak
    • 1
  1. 1.Georgia Institute of Technology/ Digital MediaAtlantaUSA

Personalised recommendations