Skip to main content
SpringerLink
Log in

Mobile Reality: A PDA-Based Multimodal Framework Synchronizing a Hybrid Tracking Solution with 3D Graphics and Location-Sensitive Speech Interaction

  • Conference paper
  • First Online:
UbiComp 2002: Ubiquitous Computing (UbiComp 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2498))

Included in the following conference series:

Abstract

A maintenance engineer who talks to pumps and pipes may not seem like the ideal person to entrust with keeping a factory running smoothly, but we hope that our Mobile Reality framework will enable such behavior in the future to be anything but suspicious! Described in this paper is how the Mobile Reality framework, running entirely on a Pocket PC, synchronizes a hybrid tracking solution to offer the user a seamless, location-dependent, mobile multimodal interface. The user interface juxtaposes a three-dimensional graphical view with a context-sensitive speech dialog centered upon objects located in the immediate vicinity of the mobile user. In addition, support for collaboration enables shared VRML browsing with annotation and a full-duplex voice channel.

This research was conducted while working at Siemens Corporate Research, Inc.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aalburg, S., Beaugeant, C., Stan, S., Fingscheidt, T., Balan, R. and Rosca, J., Single and Two-Channel Noise Reduction for Robust Speech Recognition, to appear in ISCA Workshop Multi-Modal Dialogue in Mobile Environments, June 2002.

    Google Scholar 

  2. Abowd, G., Atkeson, C., Dey, A., Hong, J., Long, S., Kooper R. and Pinkerton, M., Cyberguide: A Mobile Context-Aware Tourguide, ACM Wireless Networks Volume 3, pages 421–433, November 1997.

    Article  Google Scholar 

  3. Beigl, M., Memoclip: A Location-based Rembrance Applicance, Journal of Personal Technologies, 4(4):230–234, Springer Press, 2000.

    Article  Google Scholar 

  4. Dai, F., Virtual Reality for Industrial Applications, Springer-Verlag, 1998.

    Google Scholar 

  5. Eyeled GmbH, Saarbrücken, Germany http://www.eyeled.de/

  6. Fitzmaurice, G., Zhai, Z. and Chignell, M. Virtual Reality for Palmtop Computers, ACM Transactions on Office Information Systems, 11(3):197–218, July, 1993.

    Google Scholar 

  7. Goose, S., Gruber, I., Sudarsky, S., Hampel, K., Baxter, B. and Navab, N., 3D Interaction and Visualization in the Industrial Environment, Proceedings of the 9th International Conference on Human Computer Interaction, New Orleans, USA, Volume 1, pages 31–35, August, 2001.

    Google Scholar 

  8. Grasso, M., Ebert, D. and Finin, T., The Integrality of Speech in Multimodal Interfaces, ACM Transactions on Computer Human Interaction, 5(4):303–325, December 1998.

    Article  Google Scholar 

  9. Hinkley, K, Pierce, J., Sinclair, M. and Horvitz, E., Sensing Techniques for Mobile Interaction, ACM UIST, San Diego, USA, November 2000.

    Book  Google Scholar 

  10. Hull, R., Neaves, P. and Bedford-Roberts, J. Towards Situated Computing, Proceeding of IEEE First International Symposium on Wearable Computing, Cambridge, USA, pages 146–153, October 1997.

    Google Scholar 

  11. Infrared Data Association: http://www.irda.org

  12. Klinker, G., Reicher, T. and Bruegge, B., Distributed User Tracking Concepts for Augmented Reality Applications, Proceedings of ISAR 2000, Munich, Germany, pages 37–44, October, 2000.

    Google Scholar 

  13. Kortuem, G., Segall, Z., Thompson, T., Close Encounters: Supporting Mobile Collaboration through Interchange of User Profiles, Proceedings of the First International Symposium on Handheld and Ubiquitous Computing, Karlsruhe, Germany, pages 171–185, September 1999.

    Google Scholar 

  14. Lewis, R. and Séquin, C. H., Generation of Three-Dimensional Building Models from Two-Dimensional Architectural Plans, Computer-Aided Design, 30:10, pages 765–779, 1998.

    Article  Google Scholar 

  15. Luff, P. and Heath, C., Mobility in Collaboration, Proceedings of CSCW’ 98, Seattle, USA, November 1998.

    Google Scholar 

  16. Marmasse, N. and Schmandt, C., Location-aware Information Delivery with ComMotion, Proceedings of the Second International Symposium on Handheld and Ubiquitous Computing, Bristol, U.K., pages 157–171, September 2000.

    Google Scholar 

  17. Microsoft Portrait, http://research.microsoft.com/~jiangli/portrait/

  18. Mynatt, E., Back, M., Want, R., Baer, M. and Ellis, J., Designing Audio Aura, ACM International Conference on Computer Human Interaction, Los Angeles, USA, pages 566–573, 1998.

    Google Scholar 

  19. Newman, J., Ingram, D. and Hopper, A., Augmented Reality in a Wide Area Sentient Environment, Proceedings of ISAR 2000, New York, USA, pages 77–86, October, 2001.

    Google Scholar 

  20. Nilsson, J., Sokoler, T., Binder, T. and Wetcke, N., Beyond the Control Room: Mobile Devices for Spatially Distributed Interaction on Industrial Process Plants, Proceedings of the Second International Symposium on Handheld and Ubiquitous Computing, Bristol, U.K., pages 30–45, September 2000.

    Google Scholar 

  21. Parallel Graphics, http://www.parallelgraphics.com/products/cortonace

  22. Rekimoto, J. and Ayatsuka, Y., Cybercode: Designing Augmented Reality Environments With Visual Tags, Designing Augmented Reality Environments, 2000.

    Google Scholar 

  23. Ressler, S. and Wang, Q., Making VRML Accessible for People with Disabilities, ASSETS 98, Marina del Rey, USA, pages 50–55, April 1998.

    Google Scholar 

  24. Saffo, P., Sensors: The Next Wave of Infortech Innovation, Institute for the Future: 1997 Ten-Year Forecast, pages 115–122.

    Google Scholar 

  25. SALT Forum, http://www.saltforum.org/

  26. Smailagic, A. and Bennington, B., Wireless and Mobile Computing in Training Maintenance and Diagnosis, IEEE Vehicular Technology Conference, Phoenix, AZ, May 1997.

    Google Scholar 

  27. VoiceXML: http://www.voicexml.org

  28. Wan, D., Magic Medicine Cabinet: A Situated Portal for Consumer Healthcare, Proceedings of the First International Symposium on Handheld and Ubiquitous Computing, Karlsruhe, Germany, pages 352–355, September 1999.

    Google Scholar 

  29. Want, R., Hopper, A., Falcao, V. and Gibbons, J., The Active Badge Location System, ACM Transactions on Information Systems, 10(1):91–102, 1992.

    Article  Google Scholar 

  30. Zhai, S., Milgram, P. and Drasic, D., An Evaluation of four 6 Degree-of-Freedom Input Techniques, ACM Conference on Human Factors in Computer Systems, Asmterdam, Netherlands, 1993.

    Google Scholar 

  31. Zhang, X. and Navab, N., Tracking and Pose Estimation for Computer Assisted Localization in Industrial Environments, IEEE Workshop on Applications of Computer Vision, pages 241–221, 2000.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Multimedia Technology Department, Siemens Corporate Research, Inc., 755 College Road East, Princeton, NJ, 08540, USA

    Stuart Goose

  2. Computer Science Department, University of Saarlandes, Postfach 15 11 50, 66041, Saarbrücken, Germany

    Heiko Wanning

  3. Computer Science Department, Fachhochschule Trier, Standort Schneidershof, D-54293, Trier, Germany

    Georg Schneider

Authors
  1. Stuart Goose
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heiko Wanning
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Georg Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of Washington, 98195, Seattle, Washington, USA

    Gaetano Borriello

  2. Intel Research Seattle, 98105, Seattle, Washington, USA

    Gaetano Borriello

  3. Future Applications Lab, Viktoria Institute, Box 620, 40530, Göteborg, Sweden

    Lars Erik Holmquist

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Goose, S., Wanning, H., Schneider, G. (2002). Mobile Reality: A PDA-Based Multimodal Framework Synchronizing a Hybrid Tracking Solution with 3D Graphics and Location-Sensitive Speech Interaction. In: Borriello, G., Holmquist, L.E. (eds) UbiComp 2002: Ubiquitous Computing. UbiComp 2002. Lecture Notes in Computer Science, vol 2498. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45809-3_3

Download citation

  • DOI: https://doi.org/10.1007/3-540-45809-3_3

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-44267-7

  • Online ISBN: 978-3-540-45809-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation