Skip to main content
SpringerLink
Log in

Portable Device for Touch, Taste and Smell Sensations in Augmented Reality Experiences

  • Conference paper
  • First Online:
INCREaSE (INCREaSE 2017)

Abstract

The purpose of this work is to present a portable hardware device that can provide touch, taste and smell sensations to an augmented reality experience. The proposed hardware is part of a mobile five senses augmented reality system for Museums, to improve and augment, as much as possible, the visiting of a museum, i.e., see, ear, touch, feel and experience all its interesting objects. The existing solutions related to the augment of sensing experiences consist of big hardware systems and they are far from being portable. In this work, a new small and portable device is presented, to integrate and connect with the user’s smartphone to provide the complete five-sense experience. The implemented device adds to the complete augmented system, the touch, smell and taste experiences. Moreover, the device is flexible enough to adapt itself to different sizes of the user’s smartphone or tablet. It is powered by a rechargeable battery, which gives the module the ability to keep the system running during the visit of the museum. The core unit is a microcontroller, it receives instructions from the mobile application in the user’s smartphone and acts accordingly, controlling the remaining hardware of the portable device to deliver the five-sense experience to the user. The communication between the device and the mobile application is possible through wireless communication, using a Bluetooth interface. The communication with the remaining module, denoted here as physical interfaces, will be wired. Preliminary results of device’s first prototype are also presented in this paper.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Jung, T., Chung, N., Leue, M.C.: The determinants of recommendations to use augmented reality technologies: the case of a Korean theme park. Tourism Manag. 49, 75–86 (2015)

    Article  Google Scholar 

  2. Yuan, Z., Bi, T., Muntean, G.M., Ghinea, G.: Perceived synchronization of multisemedia services. IEEE Trans. Multimedia 17(7), 957–966 (2015)

    Article  Google Scholar 

  3. Kovács, P.T., Rozinaj, G., Murray, N., Sulema, Y., Rybárová, R.: Application of immersive technologies for education: state of the art. Paper presented at the 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), Thessaloniki, Greece (2015)

    Google Scholar 

  4. Wolf, M.J.: The Video Game Explosion: A History From PONG to Playstation and Beyond. ABC-CLIO (2008)

    Google Scholar 

  5. Shirali-Shahreza, M., Shirali-Shahreza, S.: Examining the usage of feedback vibration in Nintendo DS handheld game console. In: 11th International Conference on Advanced Communication Technology, ICACT 2009, vol. 3, pp. 1997–2000. IEEE (2009)

    Google Scholar 

  6. Brewster, S., Chohan, F., Brown, L.: Tactile feedback for mobile interactions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 159–162. ACM, San Jose, CA (2007)

    Google Scholar 

  7. Lin, J., Nishino, H., Kagawa, T., Utsumiya, K.: A handy image explorer with tactile feedback for diagnostic imaging. In: 2012 Seventh International Conference on Broadband, Wireless Computing, Communication and Applications (BWCCA), pp. 316–321. IEEE (2012)

    Google Scholar 

  8. Bau, O., Poupyrev, I., Israr, A., Harrison, C.: TeslaTouch: electrovibration for touch surfaces. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 283–292. ACM (2010)

    Google Scholar 

  9. Israr, A., Bau, O., Kim, S.C., Poupyrev, I.: Tactile feedback on flat surfaces for the visually impaired. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, pp. 1571–1576. ACM (2012)

    Google Scholar 

  10. Kim, S.C., Israr, A., Poupyrev, I.: Tactile rendering of 3D features on touch surfaces. In: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, pp. 531–538. ACM (2013)

    Google Scholar 

  11. Alexander, J., Marshall, M.T., Subramanian, S.: Adding haptic feedback to mobile tv. In: CHI’11 Extended Abstracts on Human Factors in Computing Systems, pp. 1975–1980. ACM (2011)

    Google Scholar 

  12. Long, B., Seah, S.A., Carter, T., Subramanian, S.: Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Trans. Graphics (TOG) 33(6), 181 (2014)

    Article  Google Scholar 

  13. Subramanian, S., Seah, S.A., Shinoda, H., Hoggan, E., Corenthy, L.: Mid-air haptics and displays: systems for un-instrumented mid-air interactions. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3446–3452. ACM (2016)

    Google Scholar 

  14. Moon, T., Kim, G.J.: Design and evaluation of a wind display for virtual reality. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 122–128. ACM (2004)

    Google Scholar 

  15. Deligiannidis, L., Jacob, R.J.: The vr scooter: wind and tactile feedback improve user performance. In: 3D User Interfaces (3DUI’06), pp. 143–150. IEEE (2006)

    Google Scholar 

  16. Suzuki, Y., Kobayashi, M.: Air jet driven force feedback in virtual reality. IEEE Comput. Graphics Appl. 25(1), 44–47 (2005)

    Article  Google Scholar 

  17. Sodhi, R., Poupyrev, I., Glisson, M., Israr, A.: AIREAL: interactive tactile experiences in free air. ACM Trans. Graphics (TOG) 32(4), 134 (2013)

    Article  Google Scholar 

  18. Gupta, S., Morris, D., Patel, S.N., Tan, D.: AirWave: non-contact haptic feedback using air vortex rings. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 419–428. ACM (2013)

    Google Scholar 

  19. CJ 4Dplex: (2016). Retrieved: 2 Dec 2016 from http://www.cj4dx.com

  20. Kitada, T., Kunii, Y., Hashimoto, H.: 20 dof five fingered glove type haptic interface-sensor glove ii. J. Robot. Mechatron. 9, 171–176 (1997)

    Article  Google Scholar 

  21. Koyama, T., Yamano, I., Takemura, K., Maeno, T.: Multi-fingered exoskeleton haptic device using passive force feedback for dexterous teleoperation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2002, vol. 3, pp. 2905–2910. IEEE (2002)

    Google Scholar 

  22. Sarakoglou, I., Brygo, A., Mazzanti, D., Hernandez, N.G., Caldwell, D.G., Tsagarakis, N.G.: HEXOTRAC: a highly under-actuated hand exoskeleton for finger tracking and force feedback. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1033–1040. IEEE (2016)

    Google Scholar 

  23. AxonVR: (2016). Retrieved: 6 Dec 2016 from http://axonvr.com

  24. Harrison, C., Hudson, S.E.: Providing dynamically changeable physical buttons on a visual display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 299–308. ACM (2009)

    Google Scholar 

  25. Tactus Technology: (2016). Retrieved: 2 Dec 2016 from http://tactustechnology.com/

  26. Stanley, A.A., Hata, K., Okamura, A.M.: Closed-loop shape control of a haptic jamming deformable surface. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2718–2724. IEEE (2016)

    Google Scholar 

  27. Sinclair, M., Pahud, M., Benko, H.: TouchMover 2.0-3D touchscreen with force feedback and haptic texture. In: 2014 IEEE Haptics Symposium (HAPTICS), pp. 1–6. IEEE (2014)

    Google Scholar 

  28. Geomagic: (2016). Retrieved: 6 Dec 2016 from http://www.geomagic.com

  29. Wang, H., Kaleas, D., Ruuspakka, R., Tartz, R.: Haptics using a smart material for eyes free interaction in mobile devices. In: SPIE Smart Structures and Materials+ Nondestructive Evaluation and Health Monitoring, pp. 83410I–83410I. International Society for Optics and Photonics (2012)

    Google Scholar 

  30. Benali-Khoudja, M., Hafez, M., Alexandre, J.M., Benachour, J., Kheddar, A.: Thermal feedback model for virtual reality. In: International Symposium on Micromechatronics and Human Science, vol. 1, pp. 153–158 (2003)

    Google Scholar 

  31. Kurogi, T., Nakayama, M., Sato, K., Kamuro, S., Fernando, C.L., Furukawa, M., Tachi, S.: Haptic transmission system to recognize differences in surface textures of objects for telexistence. In: 2013 IEEE Virtual Reality (VR), pp. 137–138. IEEE (2013)

    Google Scholar 

  32. Yanagida, Y.: A survey of olfactory displays: making and delivering scents. In: Sensors, 2012 IEEE, pp. 1–4. IEEE (2012)

    Google Scholar 

  33. Kaye, J.N.: Symbolic olfactory display (Doctoral dissertation, Massachusetts Institute of Technology) (2001)

    Google Scholar 

  34. PC World: PC World from IDG (2006). Retrieved: 6 Dec 2016 from http://www.pcworld.com/article/125772/worst_products_ever.html

  35. Yamada, T., Yokoyama, S., Tanikawa, T., Hirota, K., Hirose, M.: Wearable olfactory display: using odor in outdoor environment. In: IEEE Virtual Reality Conference (VR 2006), pp. 199–206. IEEE (2006)

    Google Scholar 

  36. Matsukura, H., Yoneda, T., Ishida, H.: Smelling screen: development and evaluation of an olfactory display system for presenting a virtual odor source. IEEE Trans. Vis. Comput. Graphics 19(4), 606–615 (2013)

    Article  Google Scholar 

  37. Engadget: (2014). Retrieved: 6 Dec 2016 from https://www.engadget.com/2014/06/18/ophone-duo/

  38. Onotes: (2016). Retrieved: 6 Dec 2016 from http://www.onotes.com/

  39. Hashimoto, K., Nakamoto, T.: Tiny olfactory display using surface acoustic wave device and micropumps for wearable applications. IEEE Sens. J. 16(12), 4974–4980 (2016)

    Article  Google Scholar 

  40. FeelReal: (2016). Retrieved: 6 Dec 2016 from http://feelreal.com/

  41. Tanikawa, T., Hirose, M.: A study of multi-modal display system with visual feedback. In: ISUC’08. Second International Symposium on Universal Communication, pp. 285–292. IEEE (2008)

    Google Scholar 

  42. Narumi, T., Kajinami, T., Nishizaka, S., Tanikawa, T., Hirose, M.: Pseudo-gustatory display system based on cross-modal integration of vision, olfaction and gustation. In: Virtual Reality Conference (VR), 2011 IEEE, pp. 127–130. IEEE (2011)

    Google Scholar 

  43. Ranasinghe, N., Cheok, A.D., Fernando, O.N.N., Nii, H., Ponnampalam, G.: Electronic taste stimulation. In: Proceedings of the 13th International Conference on Ubiquitous Computing, pp. 561–562. ACM (2011)

    Google Scholar 

  44. Ranasinghe, N., Nakatsu, R., Nii, H., Gopalakrishnakone, P.: Tongue mounted interface for digitally actuating the sense of taste. In: 2012 16th International Symposium on Wearable Computers (ISWC), pp. 80–87. IEEE (2012)

    Google Scholar 

  45. Ranasinghe, N., Do, E.Y.L.: Digital lollipop: studying electrical stimulation on the human tongue to simulate taste sensations. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 13(1), 5 (2016)

    Google Scholar 

  46. Sloman, A.W., Buggs, P., Molloy, J., Stewart, D.: A microcontroller-based driver to stabilize the temperature of an optical stage to within 1 mK in the range, using a Peltier heat pump and a thermistor sensor. Meas. Sci. Technol. 7(11), 1653 (1996)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Portuguese Foundation for Science and Technology (FCT), project LARSyS (UID/EEA/50009/2013), CIAC, and project M5SAR I&DT nr.3322 financed by CRESC ALGARVE2020, PORTUGAL2020 and FEDER. We also thank Faro Municipal Museum and our M5SAR project leader, SPIC—Creative Solutions [www.spic.pt].

Author information

Authors and Affiliations

  1. ISE—University of the Algarve, Faro, Portugal

    J. D. P. Sardo, J. Semião, J. M. Monteiro, João A. R. Pereira, Marco A. G. de Freitas, E. Esteves & João M. F. Rodrigues

  2. INESC-ID, Lisbon, Portugal

    J. Semião & J. M. Monteiro

  3. LARSyS, ISR, Lisbon, Portugal

    J. D. P. Sardo, João A. R. Pereira & João M. F. Rodrigues

  4. CCMAR, Faro, Portugal

    E. Esteves

Authors
  1. J. D. P. Sardo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Semião
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. João A. R. Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco A. G. de Freitas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. E. Esteves
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. João M. F. Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. Semião .

Editor information

Editors and Affiliations

  1. Institute of Engineering, University of Algarve, Faro, Portugal

    António Mortal

  2. Institute of Engineering, University of Algarve, Faro, Portugal

    Jaime Aníbal

  3. Institute of Engineering, University of Algarve, Faro, Portugal

    Jânio Monteiro

  4. Institute of Engineering, University of Algarve, Faro, Portugal

    Cláudia Sequeira

  5. Institute of Engineering, University of Algarve, Faro, Portugal

    Jorge Semião

  6. Institute of Engineering, University of Algarve, Faro, Portugal

    Manuela Moreira da Silva

  7. Institute of Engineering, University of Algarve, Faro, Portugal

    Miguel Oliveira

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sardo, J.D.P. et al. (2018). Portable Device for Touch, Taste and Smell Sensations in Augmented Reality Experiences. In: Mortal, A., et al. INCREaSE . INCREaSE 2017. Springer, Cham. https://doi.org/10.1007/978-3-319-70272-8_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70272-8_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70271-1

  • Online ISBN: 978-3-319-70272-8

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation