Tangible Images: Bridging the Real and Virtual Worlds

  • James A. Ferwerda
  • Benjamin A. Darling
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7786)


In this paper we describe our efforts to create tangible imaging systems that provide rich virtual representations of real-world surfaces. Tangible imaging systems have three main properties: 1) the images produced must be visually realistic; 2) the images must be responsive to user interaction; and 3) the images must be situated, appearing to be integrated with their environments. Our current system, based on a computer, LCD display, light and position sensors, and graphics rendering tools meets all these requirements; supporting the accurate simulation of the appearances of surfaces with complex textures and material properties, and allowing users to interact with and experience these virtual surfaces as if they were real ones. We first describe the components of our current system and its implementation. We then illustrate the system’s capabilities for simulating the appearances and behaviors of real-world surfaces. Finally we describe some potential applications of tangible imaging systems and discuss limitations and future work.


tangible interfaces realistic image synthesis surface appearance 


  1. 1.
    Darling, B.A., Ferwerda, J.A.: The tangiBook: a tangible display system for direct interaction with virtual surfaces. In: Proc. IS&T 17th Color Imaging Conference, pp. 260–266 (2009)Google Scholar
  2. 2.
    Darling, B.A., Ferwerda, J.A.: Tangible display systems: direct interfaces for computer-based studies of surface appearance. In: Proc. SPIE Human Vision and Electronic Imaging, vol. 7257, pp. 1–12 (2010)Google Scholar
  3. 3.
    Blatner, A.M., Ferwerda, J.A., Darling, B.A., Bailey, R.J.: TangiPaint: a tangible digital painting system. In: Proc. IS&T 19th Color Imaging Conference, pp. 120–107 (2011)Google Scholar
  4. 4.
  5. 5.
    Sutherland, I.E.: A head-mounted three-dimensional display. Proc. AIFPS 33, 757–764 (1968)Google Scholar
  6. 6.
    Vickers, D.L.: Sorcerer’s apprentice: head-mounted display and wand. Ph.D. dissertation, Univ. of Utah (1974)Google Scholar
  7. 7.
    Callahan, M.A.: A 3-D display head-set for personalized computing. M.S. thesis, MIT (1983)Google Scholar
  8. 8.
    Fisher, S.S., McGreevy, M., Humphries, J., Robinett, W.: Virtual environment display system. In: Proc. Workshop on Interactive 3D Graphics, pp. 71–87 (1986)Google Scholar
  9. 9.
    State, A., Keller, K., Fuchs, H.: Simulation-based design and rapid prototyping of a parallax-free, orthoscopic video see-through head-mounted display. In: Proc. IEEE Computer Society of Washington, DC, pp. 28–31 (2005)Google Scholar
  10. 10.
    VPL Research, See also,
  11. 11.
    SensAble Technologies,
  12. 12.
    Ascension Technology Corp,
  13. 13.
    Cruz-Neira, C., Sandin, D.J., DeFanti, T.A.: Surround-screen projection-based virtual reality: the design and implementation of the CAVE. In: Proc. ACM SIGGRAPH 1993, pp. 135–142 (1993)Google Scholar
  14. 14.
    Raskar, R., Welch, G., Low, K., Bandyopadhyay, D.: Shader lamps: animating real objects with image-based illumination. In: Proc. 12th Eurographics Workshop on Rendering Techniques, pp. 89–102 (2001)Google Scholar
  15. 15.
    Raskar, R., van Baar, J., Beardsley, P., Willwacher, T., Rao, S., Forlines, C.: iLamps: geometrically aware and self-configuring projectors. In: Proc. ACM SIGGRAPH 2003, pp. 809–818 (2003)Google Scholar
  16. 16.
    Bandyopadhyay, D., Raskar, R., Fuchs, H.: Dynamic shader lamps: painting on real objects. In: Proc. International Symposium on Augmented Reality (ISAR 2001), pp. 207–216 (2001)Google Scholar
  17. 17.
    Bimber, O., Raskar, R.: Spatial augmented reality. A K Peters Ltd., Wellesley (2005)Google Scholar
  18. 18.
    Fitzmaurice, G.W.: Situated information spaces and spatially aware palmtop computers. Communications of the ACM 36(7), 39–49 (1993)CrossRefGoogle Scholar
  19. 19.
    Tsang, M., Fitzmaurice, G.W., Kurtenbach, G., Khan, A., Buxton, W.A.S.: Boom chameleon: simultaneous capture of 3D viewpoint, voice and gesture annotations on a spatially-aware display. Proc. ACM UIST in CHI Letters 4(2), 111–120 (2002)Google Scholar
  20. 20.
    Francois, A.R.J., Kang, E.Y.: A handheld mirror simulation. In: Proc. International Conference on Multimedia and Expo., pp. 6–9 (2003)Google Scholar
  21. 21.
    Lazzari, M., Mclaughlin, M.L., Jaskowiak, J., Wong, W., Akbarian, M.: A haptic exhibition of daguerreotype cases for USC’s Fisher Gallery. In: McLaughlin, M.L., Hespanha, J., Sukhatme, G. (eds.) Touch in Virtual Environments: Haptics and the Design of Interactive System, pp. 260–269. Prentice-Hall, Upper Saddle River (2002)Google Scholar
  22. 22.
    Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proc. ACM Conference on Human Factors in Computing Systems (CHI 1997), pp. 234–241 (1997)Google Scholar
  23. 23.
    Buxton, W.: Surface and tangible computing, and the small matter of people and design. In: IEEE International Solid-State Circuits Conference Digest of Technical Papers, vol. 51, pp. 24–29 (2008)Google Scholar
  24. 24.
    Darling, B.A., Ferwerda, J.A.: Real-time multispectral rendering with complex illumination. In: Proc. IS&T 19th Color Imaging Conference, pp. 345–351 (2011)Google Scholar
  25. 25.
    Ward, G.J.: Measuring and modeling anistropic reflection. Proc. ACM SIGGRAPH 1992, Computer Graphics 26(2), 265–272 (1992)CrossRefGoogle Scholar
  26. 26.
    Gardner, A., Tchou, C., Hawkins, T., Debevec, P.: Linear light source reflectometry. Proc. SIGGRAPH 2003, ACM Transactions on Graphics 22(3), 749–758 (2003)CrossRefGoogle Scholar
  27. 27.
    Darling, B.A., Ferwerda, J.: Seeing virtual objects: simulating reflective surfaces on emissive displays. In: Proc. IS&T 20th Color Imaging Conference, pp. 1–7 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • James A. Ferwerda
    • 1
  • Benjamin A. Darling
    • 1
  1. 1.Munsell Color Science Laboratory, Chester F. Carlson Center for Imaging ScienceRochester Institute of TechnologyUSA

Personalised recommendations