Vision-Based Markerless Gaming Interface

  • Pietro Azzari
  • Luigi Di Stefano
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5716)


The paper proposes a novel human machine interface for gaming applications based on computer vision. The key idea is to allow the user to interact with the game by simply moving a hand-held consumer grade camera. Detection of natural features in the incoming video stream avoids instrumenting the scene with optical markers while preserving real-time computation and accuracy. The paper presents also a prototype videogame developed as proof-of-concept of our camera-based gaming interface. Thanks to recent advances in real-time extraction and matching of natural features from images on mobile platforms, our proposal holds the potential to enable a new generation of camera-controlled videogames for hand-held mobile devices.


human-machine interfaces camera pose estimation videogames keypoint matching 


  1. 1.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. Computer Vision and Image Understanding 110(3), 346–359 (2008)CrossRefGoogle Scholar
  2. 2.
    Cho, K., Kang, W., Soh, J., Lee, J., Yang, H.S.: Ghost hunter: a handheld augmented reality game system with dynamic environment. In: Proc. of Intl. Conf. on Entertainment Computing, pp. 10–15 (2007)Google Scholar
  3. 3.
    Close, B., Donoghue, J., Squires, J., De Bondi, P., Morris, M., Piekarski, W., Thomas, B.: Arquake: an outdoor/indoor augmented reality first person application. In: Proc. of IEEE Intl. Symp. on Wearable Computers, pp. 139–146 (2000)Google Scholar
  4. 4.
    Intel Corp. Opencv 1.1, open computer vision library (2000-2008),
  5. 5.
    Fiala, M.: Artag, a fiducial marker system using digital techniques. In: Proc. of IEEE Intl. Conf. on Computer Vision, pp. 590–596 (2005)Google Scholar
  6. 6.
    Freeman, W.T., Tanaka, K., Ohta, J., Kyuma, K.: Computer vision for computer games. In: Proc. of Intl. Conf. on Automatic Face and Gesture Recognition, pp. 100–105 (1996)Google Scholar
  7. 7.
    Govil, A., You, S., Neumann, U.: A video-based augmented reality golf simulator. In: Proc. of ACM Multimedia, pp. 489–490 (2000)Google Scholar
  8. 8.
    Khronos Group. Opengl 2.1, open computer graphics library (1992-2008),
  9. 9.
    Hartley, R., Zisserman, A.: Multiple view Geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  10. 10.
    Harville, M., Li, D.: Fast, integrated person tracking and activity recognition with plan-view templates from a single stereo camera. In: Proc. of Intl. Conf. on Computer Vision and Pattern Recognition, pp. 398–405 (2004)Google Scholar
  11. 11.
    Isard, M., Blake, A.: Condensation - conditional density propagation for visual tracking. Intl. Journal of Computer Vision 29(1), 5–28 (1998)CrossRefGoogle Scholar
  12. 12.
    Lam, D.: Tokamak, open physics engine library,
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Intl. Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  14. 14.
    Lu, P., Chen, Y., Zeng, X., Wang, Y.: A vision-based game control method. In: Proc. of Intl. Conf. on Computer Vision, Workshop on Human Machine Interaction, pp. 70–78 (2005)Google Scholar
  15. 15.
    Lu, P., Zeng, X.Y., Huang, X., Wang, Y.: Navigation in 3d game by markov model based head pose estimating. In: Proc. of Intl. Conf. on Image and Graphics, pp. 493–496 (2004)Google Scholar
  16. 16.
    Nintendo©. Wii,
  17. 17.
    Oda, O., Lister, L.J., White, S., Feiner, S.: Developing an augmented reality racing game. In: Proc. of Intl. Conf. on Intelligent Technologies for Interactive Environment (2008)Google Scholar
  18. 18.
    Salti, S., Di Stefano, L.: Svr-based jitter reduction for markerless augmented reality. In: Proc. of Intl. Conf. on Image Analysis and Processing (2008) (submitted paper)Google Scholar
  19. 19.
    Simon, G., Fitzgibbon, A.W., Zisserman, A.: Markerless tracking using planar structures in the scene. In: Proc. of Intl. Symposium on Augmented Reality, May-June 2000, pp. 120–128 (2000)Google Scholar
  20. 20.
    Viola, P., Jones, M.J.: Robust real-time face detection. Intl. Journal of Computer Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  21. 21.
    Wagner, D., Pintaric, T., Ledermann, F., Schmalstieg, D.: Towards massively multi-user augmented reality on handheld devices. In: Proc. of Intl. Conf. on Pervasive Computing, pp. 208–219 (2005)Google Scholar
  22. 22.
    Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T., Schmalstieg, D.: Pose tracking from natural features on mobile phones. In: Proc. of Intl. Symp. on Mixed and Augmented Reality, pp. 125–134 (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Pietro Azzari
    • 1
  • Luigi Di Stefano
    • 1
  1. 1.ARCES - DEISUniversity of BolognaBolognaItaly

Personalised recommendations