Skip to main content
SpringerLink
Log in

A Systematic Review of the Convergence of Augmented Reality, Intelligent Virtual Agents, and the Internet of Things

  • Chapter
  • First Online:
Book cover Artificial Intelligence in IoT

Abstract

In recent years we are beginning to see the convergence of three distinct research fields: augmented reality (AR), intelligent virtual agents (IVAs), and the Internet of things (IoT). Each of these has been classified as a disruptive technology for our society. Since their emergence, the advancement of knowledge and development of technologies and systems in these fields were traditionally performed with limited input from each other. However, over recent years, we have seen research prototypes and commercial products being developed that cross the boundaries between these distinct fields to leverage their collective strengths. In this paper, we review the body of literature published at the intersections between each two of these fields, and we discuss a vision for the nexus of all three technologies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., & Ayyash, M. (2015). Internet of things: A survey on enabling technologies, protocols, and applications. IEEE Communications Surveys & Tutorials, 17(4), 2347–2376.

    Article  Google Scholar 

  2. Al-Turjman, F. (2018). Information-centric framework for the Internet of Things (IoT): Traffic modeling & optimization. Future Generation Computer Systems Journal, 80, 63–75.

    Article  Google Scholar 

  3. Al-Turjman, F., & Alturjman, S. (2018). Context-sensitive access in Industrial Internet of Things (IIoT) healthcare applications. IEEE Transactions on Industrial Informatics, 14, 2736–2744.

    Article  Google Scholar 

  4. Anabuki, M., Kakuta, H., Yamamoto, H., & Tamura, H. (2000). Welbo: An embodied conversational agent living in mixed reality space. CHI ‘00 Extended Abstracts on Human Factors in Computing Systems. Proceedings of the ACM SIGCHI Conference on Human Factors 659 in Computing Systems (pp. 10–11).

    Google Scholar 

  5. Ashton, K. (2009). That ‘internet of things’ thing. RFID journal, 22(7), 97–114.

    Google Scholar 

  6. Austerjost, J., Porr, M., Riedel, N., Geier, D., Becker, T., Scheper, T., et al. (2018). Introducing a virtual assistant to the lab: A voice user Interface for the intuitive control of laboratory instruments. SLAS TECHNOLOGY: Translating Life Sciences Innovation, 23(5), 476–482.

    Google Scholar 

  7. Azuma, R. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6(4), 355–385.

    Article  Google Scholar 

  8. Barakonyi, I., & Schmalstieg, D. (2005). Augmented reality agents in the development pipeline of computer entertainment. International Conference on Entertainment Computing (pp. 345–356).

    Google Scholar 

  9. Barakonyi, I., & Schmalstieg, D. (2006). Ubiquitous animated agents for augmented reality. 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 145–154).

    Google Scholar 

  10. Barakonyi, I., Psik, T., & Schmalstieg, D. (2004). Agents that talk and hit back: Animated agents in augmented reality. IEEE and ACM International Symposium on Mixed and Augmented Reality (pp. 141–150).

    Google Scholar 

  11. Barakonyi, I., Weilguny, M., Psik, T., & Schmalstieg, D. (2005). Monkey Bridge: Autonomous agents in augmented reality games. Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (pp. 172–175).

    Google Scholar 

  12. Bimber, O., & Raskar, R. (2005). Spatial augmented reality: Merging real and virtual worlds. Wellesley, MA: A.K. Peters.

    Book  Google Scholar 

  13. Blascovich, J. (2002). Social influence within immersive virtual environments. In The social life of avatars (pp. 127–145). London: Springer.

    Chapter  Google Scholar 

  14. Blum, L., Wetzel, R., McCall, R., Oppermann, L., & Broll, W. (2012). The final TimeWarp: Using form and content to support player experience and presence when designing location-aware mobile augmented reality games. Proceedings of the Designing Interactive Systems Conference on (pp. 711–720).

    Google Scholar 

  15. Bölöni, L., & Turgut, D. (2017). Value of information based scheduling of cloud computing resources. Future Generation Computer Systems Journal, 71, 212–220.

    Article  Google Scholar 

  16. Campagna, G., Ramesh, R., Xu, S., Fischer, M., & Lam, M. (2017). Almond: The architecture of an open, crowdsourced, privacy-preserving, programmable virtual assistant. WWW ‘17 Proceedings of the 26th International Conference on World Wide Web (pp. 341–350).

    Google Scholar 

  17. Charles, F., Cavazza, M., Mead, S., Martin, O., Nandi, A., & Marichal, X. (2004). Compelling experiences in mixed reality interactive storytelling. Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (pp. 32–40).

    Google Scholar 

  18. Chekhlov, D., Gee, A., Calway, A., & Mayol-Cuevas, W. (2007). Ninja on a plane: Automatic discovery of physical planes for augmented reality using visual SLAM. 6th IEEE and ACM International Symposium on Mixed and Augmented reality (pp. 153–156).

    Google Scholar 

  19. Chung, H., Iorga, M., Voas, J., & Lee, S. (2017). Alexa, Can I Trust You? IEEE Computer, 50(9), 100–104.

    Article  Google Scholar 

  20. Chung, H., Park, J., & Lee, S. (2017). Digital forensic approaches for Amazon Alexa ecosystem. Digital Investigation, 22, S15–S25.

    Article  Google Scholar 

  21. Daher, S., Kim, K., Lee, M., Bruder, G., Schubert, R., Bailenson, J., et al. (2017). Can social presence be contagious? Effects of social presence priming on interaction with virtual humans. 2017 IEEE Symposium on 3D User Interfaces (3DUI) (pp. 201–202).

    Google Scholar 

  22. Dey, A., Billinghurst, M., Lindeman, R., & Swan, J. (2018). A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI, 5, 37.

    Article  Google Scholar 

  23. Dow, S., Mehta, M., Harmon, E., MacIntyre, B., & Mateas, M. (2007). Presence and engagement in an interactive drama. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1475–1484).

    Google Scholar 

  24. Druga, S., Williams, R., Breazeal, C., & Resnick, M. (2017). "Hey Google is it OK if I eat you?": Initial explorations in child-agent interaction. Proceedings of the 2017 Conference on Interaction Design and Children (pp. 595–600).

    Google Scholar 

  25. Evans, D. (2011). The internet of things: How the next evolution of the internet is changing everything. Cisco Blog, 1, 1–1.

    Google Scholar 

  26. García-Macías, J. A., Alvarez-Lozano, J., Estrada-Martinez, P., & Avilés-López, E. (2011). Browsing the internet of things with sentient visors. Computer, 44(5), 46–52.

    Article  Google Scholar 

  27. Gatebox, Inc. (2018). Retrieved from https://gatebox.ai.

  28. Gibson, J. (1979). The ecological approach to visual perception. Dallas: Houghtom Mifflin.

    Google Scholar 

  29. Gimenez, R., & Pous, M. (2010). Augmented reality as an enabling factor for the internet of things. Proceedings of the W3C Workshop: Augmented Reality on the Web.

    Google Scholar 

  30. Growing Convergence Research. (2018). Retrieved from National Science Foundation: https://www.nsf.gov/news/special_reports/big_ideas/convergent.jsp.

  31. Hantono, B., Nugroho, L., & Santosa, P. (2016). Review of augmented reality agent in education. 2016 6th International Annual Engineering Seminar (InAES) (pp. 150–153).

    Google Scholar 

  32. Hao, Y., & Helo, P. (2017). The role of wearable devices in meeting the needs of cloud manufacturing: A case study. Robotics and Computer-Integrated Manufacturing, 45, 168–179.

    Article  Google Scholar 

  33. Helal, A., Cho, K., Lee, W., Sung, Y., Lee, J., & Kim, E. (2012). 3D modeling and simulation of human activities in smart spaces. 2012 9th International Conference on Ubiquitous Intelligence and Computing and 9th International Conference on Autonomic and Trusted Computing (pp. 112–119).

    Google Scholar 

  34. Heun, V., Hobin, J., & Maes, P. (2013). Reality editor: Programming smarter objects. Proceedings of the 2013 ACM conference on Pervasive and Ubiquitous Computing Adjunct Publication (pp. 307–310).

    Google Scholar 

  35. Hoffman, D., & Novak, T. (2018). Consumer and object experience in the internet of things: An assemblage theory approach. Journal of Consumer Research, 44(6), 1178–1204.

    Article  Google Scholar 

  36. Holz, T., Campbell, A., O’Hare, G., Stafford, J., Martin, A., & Dragone, M. (2011). MiRA-Mixed Reality Agents. International Journal of Human-Computer Studies/International Journal of Man-Machine Studies, 69(4), 251–268.

    Google Scholar 

  37. Jo, D., & Kim, G. (2016). ARIoT: Scalable augmented reality framework for interacting with internet of things appliances everywhere. IEEE Transactions on Consumer Electronics, 62(3), 334–340.

    Article  Google Scholar 

  38. Kasahara, S., Niiyama, R., Heun, V., & Ishii, H. (2013). exTouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality. Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (pp. 223–228).

    Google Scholar 

  39. Kim, K., Bruder, G., Maloney, D., & Welch, G. (2016). The influence of real human personality on social presence with a virtual human in augmented reality. ICAT-EGVE ‘16 proceedings of the 26th International Conference on Artificial Reality and Telexistence and the 21st Eurographics Symposium on Virtual Environments (pp. 115–122).

    Google Scholar 

  40. Kim, K., Bruder, G., & Welch, G. (2017). Exploring the effects of observed physicality conflicts on real-virtual human interaction in augmented reality. Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology (p. 31).

    Google Scholar 

  41. Kim, K., Billinghurst, M., Bruder, G., Duh, H. B.-L., & Welch, G. (2018). Revisiting trends in augmented reality research: A review of the 2nd decade of ISMAR (2008–2017). IEEE Transactions on Visualization and Computer Graphics (TVCG) Special Issue on the International Symposium on Mixed and Augmented Reality (ISMAR).

    Google Scholar 

  42. Knote, R., Janson, A., Eigenbrod, L., & Söllner, M. (2018). The what and how of smart personal assistants: Principles and application domains for IS research. In: Multikonferenz Wirtschaftsinformatik (MKWI). Lüneburg: Germany.

    Google Scholar 

  43. Kollee, B., Kratz, S., & Dunnigan, A. (2014). Exploring gestural interaction in smart spaces using head mounted devices with ego-centric sensing. Proceedings of the 2nd ACM symposium on spatial user interaction (pp. 40–49).

    Google Scholar 

  44. Kotranza, A., & Lok, B. (2008). Virtual human + tangible interface = mixed reality human an initial exploration with a virtual breast exam patient. 2008 IEEE Virtual Reality Conference (pp. 99–106).

    Google Scholar 

  45. Kotranza, A., Lok, B., Deladisma, A., Pugh, C., & Lind, D. (2009). Mixed reality humans: Evaluating behavior, usability, and acceptability. IEEE Transactions on Visualization and Computer Graphics, 15(3), 369–382.

    Article  Google Scholar 

  46. Krum, D., Suma, E., & Bolas, M. (2012). Augmented reality using personal projection and retroreflection. Ubiquitous Computing, 16(1), 17–26.

    Article  Google Scholar 

  47. Lee, L.H. & Hui, P. (2018). Interaction Methods for Smart Glasses: A survey. IEEE Access, (pp. 28712–28732).

    Article  Google Scholar 

  48. Lee, M., Kim, K., Daher, S., Raij, A., Schubert, R., Bailenson, J., et al. (2016a). The wobbly table: Increased social presence via subtle incidental movement of a real-virtual table. 2016 IEEE Virtual Reality (VR) (pp. 11–17).

    Google Scholar 

  49. Lee, W., Cho, S., Chu, P., Vu, H., Helal, S., Song, W., et al. (2016b). Automatic agent generation for IoT-based smart house simulator. Neurocomputing, 209, 14–24.

    Article  Google Scholar 

  50. Lee, M., Bruder, G., & Welch, G. (2017). Exploring the effect of vibrotactile feedback through the floor on social presence in an immersive virtual environment. 2017 IEEE Virtual Reality (VR) (pp. 105–111).

    Google Scholar 

  51. Lee, M., Bruder, G., Hollerer, T., & Welch, G. (2018). Effects of unaugmented periphery and vibrotactile feedback on proxemics with virtual humans in AR. IEEE Transactions on Visualization and Computer Graphics, 24(4), 1525–1534.

    Article  Google Scholar 

  52. Lok, B., Chuah, J., Robb, A., Cordar, A., Lampotang, S., Wendling, A., et al. (2014). Mixed-reality humans for team training. IEEE Computer Graphics and Applications, 34(3), 72–75.

    Article  Google Scholar 

  53. López, G., Quesada, L., & Guerrero, L. A. (2017). Alexa vs. Siri vs. Cortana vs. Google assistant: A comparison of speech-based natural user interfaces. Proceedings of the International Conference on Applied Human Factors and Ergonomics (pp. 241–250).

    Google Scholar 

  54. Magic Leap, Inc. (2018). Retrieved from https://www.magicleap.com.

  55. Martin, K., & Laviola, J. (2016). The transreality interaction platform: Enabling interaction across physical and virtual reality. 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) (pp. 177–186).

    Google Scholar 

  56. Mayle, A., Bidoki, N. H., Masnadi, S., Bölöni, L., & Turgut, D. (2017). Investigating the value of privacy within the internet of things. Proceedings of IEEE GLOBECOM (pp. 1–6).

    Google Scholar 

  57. Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995). Augmented reality: A class of displays on the reality-virtuality continuum. Telemanipulator and Telepresence Technologies, 2351, 282–292.

    Article  Google Scholar 

  58. Newsroom Gartner. (2016). Retrieved from Gartner says worldwide spending on VPA-enabled wireless speakers will top $2 billion by 2020: https://www.gartner.com/newsroom/id/3464317.

  59. Norouzi, N., Kim, K., Hochreiter, J., Lee, M., Daher, S., Bruder, G., et al. (2018). A systematic survey of 15 years of user studies Published in the Intelligent virtual agents conference. International Conference on Intelligent Virtual Agents (IVA).

    Google Scholar 

  60. Papagiannis, H. (2017). Augmented human: How technology is shaping the new reality. Bejing: O’Reilly Media.

    Google Scholar 

  61. Paul, Z., Margarita, P., Vasilis, M., & George, P. (2016). Life-sized group and crowd simulation in Mobile AR. Proceedings of the 29th International Conference on Computer Animation and Social Agents (pp. 79–82).

    Google Scholar 

  62. Raskar, R. (2001). Projector-based three dimensional graphics. Chapel Hill: University of North Carolina.

    Google Scholar 

  63. Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., & Fuchs, H. (1998). The office of the future: A unified approach to image-based modeling and spatially immersive displays. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (pp. 179–188).

    Google Scholar 

  64. Raskar, R., Welch, G., & Chen, W.-C. (1999). Table-top spatially-augmented realty: Bringing physical models to life with projected imagery. Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR’99) (pp. 64–71).

    Google Scholar 

  65. Raskar, R., Welch, G., Low, K.-L., & Bandyopadhyay, D. (2001). Shader lamps: Animating real objects with image-based illumination. Proceedings of the 12th Eurographics Workshop on Rendering Techniques (pp. 89–102).

    Google Scholar 

  66. Reis, A., Paulino, D., Paredes, H., & Barroso, J. (2017). Using intelligent personal assistants to strengthen the Elderlies’ social bonds. International Conference on Universal Access in Human-Computer Interaction (pp. 593–602).

    Chapter  Google Scholar 

  67. Robb, A., Cordar, A., Lampotang, S., White, C., Wendling, A., & Lok, B. (2015). Teaming up with virtual humans: How other people change our perceptions of and behavior with virtual teammates. IEEE Transactions on Visualization and Computer Graphics, 21(4), 511–519.

    Article  Google Scholar 

  68. Salman, T., & Jain, R. (2017). A survey of protocols and standards for internet of things. Advanced Computing and Communications, 1(1), 1–20.

    Google Scholar 

  69. Seo, D., Kim, H., Kim, J., & Lee, J. (2016). Hybrid reality-based user experience and evaluation of a context-aware smart home. Computers in Industry, 76, 11–23.

    Article  Google Scholar 

  70. Skarbez, R., Welch, G., Brooks, F., & Whitton, M. (2017). Coherence changes gaze behavior in virtual human interactions. 2017 IEEE Virtual Reality (VR) (pp. 287–288).

    Google Scholar 

  71. Soda, S., Nakamura, M., Matsumoto, S., Izumi, S., Kawaguchi, H., & Yoshimoto, M. (2012). Implementing virtual agent as an interface for smart home voice control. 2012 19th Asia-Pacific Software Engineering Conference, 1, pp. 342–345.

    Google Scholar 

  72. Sutherland, I. (1968). A head-mounted three dimensional display. Proceedings of the December 9–11, 1968, Fall Joint Computer Conference, Part I on (pp. 757–764).

    Google Scholar 

  73. Turgut, D., & Bölöni, L. (2017, September). Value of information and cost of privacy in the internet of things. IEEE Communications Magazine, 55(9), 62–66.

    Article  Google Scholar 

  74. Vugt, H., Bailenson, J., Hoorn, J., & Konijn, E. (2010). Effects of facial similarity on user responses to embodied agents. ACM Transactions on Computer-Human Interaction, 17(2), 7.

    Article  Google Scholar 

  75. Wagner, D., Billinghurst, M., & Schmalstieg, D. (2006). How real should virtual characters be. Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (p. 57).

    Google Scholar 

  76. Wirtz, H., Rüth, J., Serror, M., Link, J., & Wehrle, K. (2014). Opportunistic interaction in the challenged internet of things. Proceedings of the 9th ACM MobiCom Workshop on Challenged Networks (pp. 7–12).

    Google Scholar 

  77. Zehtabian, S., Khodadadeh, S., Pearlman, R., Willenberg, B., Kim, B., Turgut, D., et al. (2018). Supporting rehabilitation prescription compliance with an IoT-augmented four-legged walker. Workshop on AI for Aging, Rehabilitation and Independent Assisted Living (ARIAL’18) in Conjunction with International Joint Conference on Artificial Intelligence (IJCA’18).

    Google Scholar 

  78. Zhang, B., Chen, Y.-H., Tuna, C., Dave, A., Li, Y., Lee, E., et al. (2014). HOBS: Head orientation-based selection in physical spaces. Proceedings of the 2nd ACM Symposium on Spatial User Interaction (pp. 17–25).

    Google Scholar 

  79. Cisco. (2018). Retrieved from Internet of things at a glance: https://www.cisco.com/c/dam/en/us/products/collateral/se/internet-of-things/at-a-glance-c45-731471.pdf.

Download references

Author information

Authors and Affiliations

  1. University of Central Florida, Orlando, FL, USA

    Nahal Norouzi, Gerd Bruder, Brandon Belna, Damla Turgut & Greg Welch

  2. Muhlenberg College, Allentown, PA, USA

    Stefanie Mutter

Authors
  1. Nahal Norouzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerd Bruder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brandon Belna
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefanie Mutter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Damla Turgut
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Greg Welch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nahal Norouzi .

Editor information

Editors and Affiliations

  1. Computer Engineering Department, Antalya Bilim University, Antalya, Turkey

    Fadi Al-Turjman

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Norouzi, N., Bruder, G., Belna, B., Mutter, S., Turgut, D., Welch, G. (2019). A Systematic Review of the Convergence of Augmented Reality, Intelligent Virtual Agents, and the Internet of Things. In: Al-Turjman, F. (eds) Artificial Intelligence in IoT. Transactions on Computational Science and Computational Intelligence. Springer, Cham. https://doi.org/10.1007/978-3-030-04110-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04110-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04109-0

  • Online ISBN: 978-3-030-04110-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation