Multimedia Systems

, Volume 17, Issue 5, pp 435–447 | Cite as

An interactive handheld spherical 3D object display system

  • Zhaorong Li
  • Kin-Hong WongEmail author
  • Man-Chuen Leung
  • Hoi-Fung Ko
  • Kai-Ki Lee
  • Michael Ming-Yuen Chang
Regular Paper


Traditional display systems usually display 3D objects on static screens (monitor, wall, etc.) and the manipulation of virtual objects by the viewer is usually achieved via indirect tools such as keyboard or mouse. It would be more natural and direct if we display the object onto a handheld surface and manipulate it with our hands as if we were holding the real 3D object. In this paper, we propose a prototype system by projecting the object onto a handheld foam sphere. The aim is to develop an interactive 3D object manipulation and exhibition tool without the viewer having to wear spectacles. In our system, the viewer holds the sphere with his hands and moves it freely. Meanwhile we project well-tailored images onto the sphere to follow its motion, giving the viewer a virtual perception as if the object were sitting inside the sphere and being moved by the viewer. The design goal is to develop a low-cost, real-time, and interactive 3D display tool. An off-the-shelf projector-camera pair is first calibrated via a simple but efficient algorithm. Vision-based methods are proposed to detect the sphere and track its subsequent motion. The projection image is generated based on the projective geometry among the projector, sphere, camera and the viewer. We describe how to allocate the view spot and warp the projection image. We also present the result and the performance evaluation of the system.


3D interactive display Projector-camera system Object tracking Virtual and augmented reality 

Supplementary material

Supplementary material 1 (MPG 14830 kb)


  1. 1.
    Cruz-Neira, C., Sandin, D.J., Defanti, T.A.: Surround-screen projection-based virtual reality: the design and implementation of the cave. In: Proceedings of ACM SIGGRAPH, pp. 135–142 (1993)Google Scholar
  2. 2.
    Gibbs, S.J., Arapis, C., Breiteneder, C.J.: Teleport–towards immersive copresence. Multimed. Syst. 7(3), 214–221 (1999)CrossRefGoogle Scholar
  3. 3.
    Raskar, R., Low, K., Welch, G.: Shader Lamps: Animating Real Objects with Image Based Illumination. Technical report, Chapel Hill (2000)Google Scholar
  4. 4.
    Low, K.L., Welch, G., Lastra, A., Fuchs, H.: Life-sized projector-based dioramas. In: ACM Symposium on Virtual Reality Software and Technology, pp. 91–101 (2001)Google Scholar
  5. 5.
    Grossberg, M., Peri, H., Nayar, S., Belhumeur, P.: Making one object look like another: controlling appearance using a projector-camera system. In: Proceedings of Computer Vision and Pattern Recognition (2004)Google Scholar
  6. 6.
    Sukthankar, R., Stockton, R.G., Mullin, M.D.: Smarter presentations: exploiting homography in camera-projector systems. In: Proceedings of Intl. conf. on Computer Vision (2001)Google Scholar
  7. 7.
    Okatani, T., Deguchi, K.: Autocalibration of a projector-camera system. IEEE Trans. Pattern Anal. Mach. Intell. 27(15), 1845–1855 (2005)CrossRefGoogle Scholar
  8. 8.
    Raskar, R., Beardsley, P.: A self-correcting projector. In: Proceedings of Computer Vision and Pattern Recognition, pp. 504–508 (2001)Google Scholar
  9. 9.
    Raij, A., Pollefeys, M.: Auto-calibration of multi-projector display walls. In: Proceedings of intl. conf. on Pattern Recognition, pp.14-17 (2004)Google Scholar
  10. 10.
    Brown, M., Majurnder, A., Yang, R.: Camera-based calibration techniques for seamless multiprojector displays. IEEE Trans. Vis. Comput. Graphics 11(2), 193–206 (2005)CrossRefGoogle Scholar
  11. 11.
    Bimber, O., Iwai, D., Wetzstein, G., Grundhofer, A.: The visual computing of projector-camera systems. In: ACM SIGGRAPH, pp. 1–25 (2008)Google Scholar
  12. 12.
    Kondoh, D., Kijima, R.: Proposal of a free form projection display using the principle of duality rendering. In: Proceedings of VSMM, pp. 346–352 (2002)Google Scholar
  13. 13.
    Raskar, R., Brown, M.S., Yang, R.G., Chen, W.C., Welch, G., Towles, H., Seales, W.B., Fuchs, H.: Multi-projection displays using camera-based registration. In: IEEE Visualization, pp. 161–168 (1999)Google Scholar
  14. 14.
    Raskar, R., Jeroen, van B.: Low-cost multi-projector curved screen displays. In: International Symposium Society for Information Display (2005)Google Scholar
  15. 15.
    Bandyopadhyay, D., Raskar, R., Fuchs, H.: Dynamic shader lamps: painting on movable objects. In: Proceedings of International Symposium on Augmented Reality, pp. 207–216 (2001)Google Scholar
  16. 16.
    Borkowski, S., Riff, O., Crowley, J.L.: Projecting rectified images in an augmented environment. In: International Workshop on Projector Camera System (2003)Google Scholar
  17. 17.
    Gupta, S., Jaynes, C.: Active pursuit tracking in a projector-camera system with application to augmented reality. In Computer Vision and Pattern Recognition Workshop (2005)Google Scholar
  18. 18.
    Leung, M.C., Lee, K.K., Wong, K.H., Chang, MM.Y.: A projector based movable hand-held display system. In: Proceedings of Computer Vision and Pattern Recognition (2009)Google Scholar
  19. 19.
    Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge. ISBN 0521540518 (2004)Google Scholar
  20. 20.
    Wong, K.Y., Schnieders, D., Li, S.: Recovering light directions and camera poses from a single sphere. In: Proceedings of European Conference on Computer Vision, pp. 631–642 (2008)Google Scholar
  21. 21.
  22. 22.
    Zhang, Z.Y.: A flexible new technique for camera calibration. IEEE Trans Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  23. 23.
    Arulamplam, M., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle fitters for online nonlinear/non-gaussian bayesian tracking. IEEE Trans. Signal Process. 50(2), 174–188 (2002)CrossRefGoogle Scholar
  24. 24.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3d-faces. In: ACM SIGGRAPH, pp. 187–194 (1999)Google Scholar
  25. 25.
    Levmar: Levmar: levenberg-marquardt nonlinear least squares algorithm in c/c++.

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Zhaorong Li
    • 1
  • Kin-Hong Wong
    • 1
    Email author
  • Man-Chuen Leung
    • 1
  • Hoi-Fung Ko
    • 1
  • Kai-Ki Lee
    • 2
  • Michael Ming-Yuen Chang
    • 2
  1. 1.Department of Computer Science and EngineeringThe Chinese University of Hong KongShatinHong Kong
  2. 2.Department of Information EngineeringThe Chinese University of Hong KongShatinHong Kong

Personalised recommendations