Skip to main content
SpringerLink
Log in

Enhancing Interactivity in Handheld AR Environments

  • Chapter
  • First Online:
Handbook of Augmented Reality

  • 11k Accesses

Abstract

In this paper we present a trial of development of handheld AR supporting two interaction techniques based on the understanding that both real and virtual objects should be compatible in manipulation. One is the ability to contact virtual objects by user’s finger, as well as to physical objects in the real world. The virtual objects change their position, shape, and/or other graphical features according to his/her manipulation. The other technique we discuss in the paper concerns pseudo manipulation of virtual objects. The motion such as shake and tilt to a handheld device causes a certain effect to the virtual objects in the AR content.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. F. Zhou, H. B. L. Duh, and M. Billinghurst, “Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR,” Proceedings of the IEEE International Symposium on Mixed Reality and Augmented Reality (ISMAR2008), 2008.

    Google Scholar 

  2. K. L. Kroeker, “Mainstreaming Augmented Reality,” Communications of the ACM, Vol. 53, No. 7, 2010, pp.19–21.

    Article  Google Scholar 

  3. M. Billinghurst, H. Kato, and I. Poupyrev, “Tangible Augmented Reality,” Proceeding of SIGGRAPH Asia ’08, 2008.

    Google Scholar 

  4. http://www.hitl.washington.edu/artoolkit/

  5. J. Neubert, J. Pretlove and T. Drummond, “Semi-Autonomous Generation of Appearance-based Edge Models from Image Sequences,” Proceedings of the IEEE International Symposium on Mixed Reality and Augmented Reality (ISMAR2007), 2007.

    Google Scholar 

  6. D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg, “Real-Time Detection and Tracking for Augmented Reality on Mobile Phones,” IEEE Transactions on Visualization and Computer Graphics, Vol. 16, No. 3, May/June 2010, 355–368.

    Google Scholar 

  7. G. Klein and D. Murray, “Parallel Tracking and Mapping on a Camera Phone,” Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR2009), 2009.

    Google Scholar 

  8. J. Herling and W. Broll, “An Adaptive Training-free Feature Tracker for Mobile Phones,” Proceedings of ACM Symposium on Virtual Reality Software and Technology (VRST2010), 2010.

    Google Scholar 

  9. J. Caarls, P. Jonker, Y. Kolstee, J. Rotteveel, and W. van Eck, “Augmented Reality for Art, Design and Cultural Heritage - System Design and Evaluation,” EURASIP Journal on Image and Video Processing, Vol. 2009, Article ID 716160, 2009.

    Google Scholar 

  10. S. J. Vaughan-Nichols, “Augmeted Reality: No Longer a Novelty,” IEEE Computer, Vol. 42, No. 12, 2009, pp.19–22.

    Article  Google Scholar 

  11. S. Asai and M. Hirakawa, “Dual Face Interaction in Handheld Augmented Reality Environments,” Proceedings of IEEE International Symposium on Multimedia, 2009.

    Google Scholar 

  12. G. A. Lee, M. Billinghurst, and G. J. Kim, “Occlusion based Interaction Methods for Tangible Augmented Reality Environments,” Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI ’04), 2004.

    Google Scholar 

  13. G. Heidemann, I. Bax, and H. Bekel, “Multimodal Interaction in an Augmented Reality Scenario,” Proceedings of Sixth International Conference on Multimodal Interfaces (ICMI 04), 2004.

    Google Scholar 

  14. Y. Kojima, Y. Yasumuro, H. Sasaki, I. Kanaya, O. Oshiro, T. Kuroda, Y. Manabe, and K. Chihara, “Hand Manipulation of Virtual Objects in Wearable Augmented Reality,” Proceedings of the Seventh International Conference on Virtual Systems and Multimedia (VSMM’01), 2001.

    Google Scholar 

  15. A. D. Cheok, N. W. C. Edmund, and A. W. Eng, “Inexpensive Non-Sensor Based Augmented Reality Modeling of Curves and Surfaces in Physical Space,” Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR’02), 2002.

    Google Scholar 

  16. V. Buchmann, S. Violich, M. Billinghurst, and A. Cockburn, “FingARtips – Gesture Based Direct Manipulation in Augmented Reality,” Proceedings of the 2nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE ’04), 2004.

    Google Scholar 

  17. J. Park, “Augmented Reality based Re-formable Mock-up for Design Evaluation,” Proceedings of International Symposium on Ubiquitous Virtual Reality (ISUVR2008), 2008.

    Google Scholar 

  18. G. A. Lee, C. Nelles, M. Billinghurst, and G. J. Kim, “Immersive Authoring of Tangible Augmented Reality Applications,” Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2004), 2004.

    Google Scholar 

  19. P. Song, H. Yu, and S. Winkler, “Vision-based 3D Finger Interactions for Mixed Reality Games with Physics Simulation,” Proceedings of The 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI2008), 2008.

    Google Scholar 

  20. A. D. Cheok, S. W. Fong, K. H. Goh, X. Yang, W. Liu, and F. Farzbiz, “Human Packman: A Sensing-based Mobile Entertainment System with Ubiquitous Computing and Tangible Interaction,” Proceedings of the 2nd Workshop on Network and System Support for Games (NetGame’03), 2003.

    Google Scholar 

  21. K. Dorfmüller-Ulhaas and D. Schmalstieg, “Finger Tracking for Interaction in Augmented Environments,” Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR’01), 2001.

    Google Scholar 

  22. D. Molyneaux and H. Gellersen, “Projected Interfaces: Enabling Serendipitous Interaction with Smart Tangible Objects,” Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI’09), 2009.

    Google Scholar 

  23. S. Henderson and S. Feiner, “Opportunistic Tangible User Interfaces for Augmented Reality,” IEEE Transactions on Visualization and Computer Graphics, Vol. 16, No. 1, January/February 2010, pp. 4–16.

    Google Scholar 

  24. D. Zhang, Y. Shen, S. K. Ong, and A. Y. C. Nee, “An Affordable Augmented Reality based Rehabilitation System for Hand Motions,” Proceedings of 2010 International Conference on Cyberworlds, 2010.

    Google Scholar 

  25. A. Anagnostopoulos and A. Pnevmatikakis, “A Realtime Mixed Reality System for Seamless Interaction between Real and Virtual Objects,” Proceedings of Third International Conference on Digital Interactive Media in Entertainment and Arts (DIMEA’08), 2008.

    Google Scholar 

  26. T. N. Hoang, S. R. Porter, and B. H. Thomas, “Augmenting Image Plane AR 3D Interactions for Wearable Computers,” Proceedings of Tenth Australasian User Interface Conference (AUIC2009), 2009.

    Google Scholar 

  27. M. Koelsch, R. Bane, T. Höllerer, and M. Turk, “Multimodal Interaction with a Wearable Augmented Reality System,” IEEE Computer Graphics and Applications, Volume 26, Issue 3, May/June 2006.

    Google Scholar 

  28. H. Seichter, R. Grasset, J. Looser, and M. Billinghurst, “Multitouch Interaction for Tangible User Interfaces,” Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR2009), 2009.

    Google Scholar 

  29. J. Looser, R. Grasset, and M. Billinghurst, “A 3D Flexible and Tangible Magic Lens in Augmented Reality,” Proceedings of 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2007), 2007.

    Google Scholar 

  30. S. White, D. Feng, and S. Feiner, “Interaction and Presentation Techniques for Shake Menus in Tangible Augmented Reality,” Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR2009), 2009.

    Google Scholar 

  31. G. Bianchi, C. Jung, B. Knoerlein, G. Székely, and M. Harders, “High-Fidelity Visuo-Haptic Interaction with Virtual Objects in Multi-Modal AR Systems,” Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2006), 2006.

    Google Scholar 

  32. D. Calife, J. L. Bernardes, Jr., and R. Tori, “Robot Arena: An Augmented Reality Platform for Game Development,” ACM Computers in Entertainment, Vo. 7, Article 11, 2009.

    Google Scholar 

  33. H. Kobayashi, T. Osaki, T. Okuyama, J. Gramm, A. Ishino, and A. Shinohara, “Development of an Interactive Augmented Environment and Its Application to Autonomous Learning for Quadruped Robots,” IEICE Transactions on Information and Systems, Vol. E92, No. 9, 2009, pp.1752–1761.

    Article  Google Scholar 

  34. T. Harviainen, O. Korkalo, and C. Woodward, “Camera-based Interactions for Augmented Reality,” Proceedings of International Conference on Advances in Computer Entertainment Technology (ACE2009), 2009.

    Google Scholar 

  35. O. Oda, L. J. Lister, S. White, and S. Feiner. “Developing an Augmented Reality Racing Game,” Proceedings of the Second International Conference on Intelligent Technologies for Interactive Entertainment (ICST INTETAIN2008), 2008.

    Google Scholar 

  36. C. Bichlmeier, F. Wimmer, S. M. Heining, and N. Navab, “Contextual Anatomic Mimesis: Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality,” Proceedings of 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR2007), 2007.

    Google Scholar 

  37. A. Henrysson, J. Marshall, and M. Billinghurst, “Experiments in 3D Interaction for Mobile Phone AR,” Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE 2007), 2007.

    Google Scholar 

  38. B. K. Seo, J. Choi, J. H. Han, H. Park, and J. I. Park, “One-Handed Interaction with Augmented Virtual Objects on Mobile Devices,” Proceedings of ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCIA 08), 2008.

    Google Scholar 

  39. C. Ludwig and C. Reimann, “Augmented Reality: Information at Focus,” C-LAB Report, Vol. 4, No. 1, 2005.

    Google Scholar 

  40. A. Henrysson and M. Billinghurst, “Using a Mobile Phone for 6 DOF Mesh Editing,” Proceedings of the 7th ACM SIGCHI New Zealand Chapter’s International Conference on Computer-Human Interaction (CHINZ2007), 2007.

    Google Scholar 

  41. A. Henrysson, M. Billinghurst, and M. Ollila, “Face to Face Collaborative AR on Mobile Phones,” Proceedings of IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2005), 2005.

    Google Scholar 

  42. S. Bucolo, M. Billinghurst, and D. Sickinger, “User Experiences with Mobile Phone Camera Game Interfaces,” Proceedings of the 4th International Conference on Mobile and Ubiquitous Multimedia (MUM2005), 2005.

    Google Scholar 

  43. A. Joshi and P. Rheingans, “Illustration-Inspired Techniques for Visualizing Time-Varying Data,” Proceedings of IEEE Visualization, 2005.

    Google Scholar 

  44. A. Joshi, J. Caban, P. Rheingans, and L. Sparling, “Case Study on Visualizing Hurricanes Using Illustration-Inspired Techniques,” IEEE Transactions on Visualization and Computer Graphics, Vol. 15, No. 5, September/October, 2009, pp. 709–718.

    Google Scholar 

  45. M. Hirakawa, “Going Beyond Completeness in Information Retrieval,” International Journal of Computational Science and Engineering, Vol. 5, No.2, 2010, pp. 110–117.

    Article  Google Scholar 

Download references

Acknowledgments

This research was funded in part by Shimane Industrial Promotion Foundation.

Author information

Authors and Affiliations

  1. Shimane University, Shimane, Japan

    Masahito Hirakawa, Shu’nsuke Asai & Kengo Sakata

  2. Roots Co. Ltd, Shimane, Japan

    Shuhei Kanagu, Yasuhiro Sota & Kazuhiro Koyama

Authors
  1. Masahito Hirakawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shu’nsuke Asai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kengo Sakata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuhei Kanagu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yasuhiro Sota
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kazuhiro Koyama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masahito Hirakawa .

Editor information

Editors and Affiliations

  1. Dept. of Computer Science & Engineering, Florida Atlantic University, Boca Raton, 33431, Florida, USA

    Borko Furht

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Hirakawa, M., Asai, S., Sakata, K., Kanagu, S., Sota, Y., Koyama, K. (2011). Enhancing Interactivity in Handheld AR Environments. In: Furht, B. (eds) Handbook of Augmented Reality. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-0064-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-0064-6_12

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-0063-9

  • Online ISBN: 978-1-4614-0064-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation