Skip to main content
SpringerLink
Log in

Using Artificial Intelligence to Bring Accurate Real-Time Simulation to Virtual Reality

  • Chapter
  • First Online:
Book cover Advanced Computational Intelligence Techniques for Virtual Reality in Healthcare

Part of the book series: Studies in Computational Intelligence ((SCI,volume 875))

  • 782 Accesses

Abstract

There always has been an excruciating gap between theoretical possibilities, clinical trial and real world applications in the Medical Industry. Any new research, experimentation or training in this sector has always been subject to extreme scrutiny and legal intricacies, due to the complexity of the human body and any resulting complications that might arise from the application of prematurely tested techniques or tools. The introduction of Virtual Reality in the Medical Industry is bringing all these troubles to their heel. Simulations generated by virtual reality are currently being explored to impart education and practical medical experience to students and doctors alike, generate engaging environments for patients and thus assisting in various aspects ranging from treatment of medical conditions to rehabilitation. This book chapter aims to develop an understanding on how virtual reality is being applied in the healthcare industry. A formal study of various solutions for reducing the latency is presented along with research being done in the area for improving the performance and making the experience more immersive. It is evident that motion to photons latency plays a crucial role in determining a genuine virtual reality experience. Among many, foveated rendering and gaze tracking systems seem to be the most promising in creating exciting opportunities for virtual reality systems in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sutherland, I. E. (1965). The ultimate display. Proceedings of IFIP 65, 2, 506–508.

    Google Scholar 

  2. Fuchs, H., Bishop, G., et al. (1992). Research directions in virtual environments. NFS Invitational Workshop, University North Carolina.

    Google Scholar 

  3. Gigante, M. (1993). Virtual reality: Definitions, history and applications. Virtual Reality Systems, 3–14. ISBN 0-12-22-77-48-1.

    Google Scholar 

  4. Steur, J. (1995). Defining virtual reality: Dimensions determining telepresence. In F. L. Biocca (Ed.), Communication in the age of virtual reality. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  5. Briggs, J. C. (1996). The promise of virtual reality. The Futurist, 30.

    Google Scholar 

  6. Alqahtani, A. S., Daghestani, L., & Ibrahim, L. F. (2017). Environments and system types of virtual reality technology in STEM: A survey.

    Google Scholar 

  7. Alqahtani, A., Daghestani, L., & Ibrahim, L. F. (2017). Environments and system types of virtual reality technology in STEM: A survey.

    Google Scholar 

  8. Shakibamanesh, A. (2014). Improving results of urban design research by enhancing advanced semi-experiments in virtual environments. IJAUP, 24(2), 131–141.

    Google Scholar 

  9. Fairén González, M., Farrés, M., Moyes Ardiaca, J., & Insa, E. (2017). Virtual reality to teach anatomy. In Eurographics 2017: education papers (pp. 51–58). European Association for Computer Graphics (Eurographics).

    Google Scholar 

  10. Codd, A. M., & Choudhury, B. (2011). Virtual reality anatomy: Is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anatomical Sciences Education, 4(3), 119–125.

    Article  Google Scholar 

  11. Basdogan, C., & Srinivasan, M. A. (2002). Haptic rendering in virtual environments. In Handbook of virtual environments (pp. 157–174). CRC Press.

    Google Scholar 

  12. Górski, F., Buń, P., Wichniarek, R., Zawadzki, P., & Hamrol, A. (2015). Immersive city bus configuration system for marketing and sales education. Procedia Computer Science, 75, 137–146.

    Article  Google Scholar 

  13. Second Life [Online Virtual World]. (2019). Accessible from http://secondlife.com.

  14. Life, S. (2017). An overview of the potential of 3-D virtual worlds in medical and health education. Health Information and Libraries Journal, 24(4), 233–245.

    Google Scholar 

  15. Wii [Video Game Console]. (2019). Retrieved from URL: http://wii.com.

  16. The Future of Healthcare Communication. (2007). Second health. [Online Virtual World]. Retrieved from URL: http://secondhealth.wordpress.com.

  17. Hansen, M. (2008). Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. Journal of medical Internet research, 10(3), e26.

    Article  Google Scholar 

  18. Aïm, F., Lonjon, G., Hannouche, D., & Nizard, R. (2016). Effectiveness of virtual reality training in orthopaedic surgery. Arthroscopy: the journal of arthroscopic & related surgery, 32(1), 224–232.

    Google Scholar 

  19. Mohan, P. V. R., & Chaudhry, R. (2009). Laparoscopic simulators: Are they useful! Medical Journal Armed Forces India, 65(2), 113–117.

    Article  Google Scholar 

  20. Lap Mentor [Simulator]. (2017). Accessible from https://simbionix.com/simulators/lap-mentor/.

  21. LapSim [Simulator]. (2018). Accessible from https://surgicalscience.com/systems/lapsim/.

  22. Simendo [Simulator]. (2018). Accessible from https://www.simendo.eu.

  23. Wilson, M. S., Middlebrook, A., Sutton, C., Stone, R., & McCloy, R. F. (1997). MIST VR: A virtual reality trainer for laparoscopic surgery assesses performance. Annals of the Royal College of Surgeons of England, 79(6), 403.

    Google Scholar 

  24. Vaughan, N., Dubey, V. N., Wainwright, T. W., & Middleton, R. G. (2016). A review of virtual reality based training simulators for orthopaedic surgery. Medical Engineering & Physics, 38(2), 59–71.

    Article  Google Scholar 

  25. SimOrtho [Simulator]. (2019). Retrieved from https://ossimtech.com/en-us/Simulators.

  26. Salb, T., Weyrich, T., & Dillmann, R. (1999, April). Preoperative planning and training simulation for risk reducing surgery. In International Training and Education Conference (ITEC).

    Google Scholar 

  27. Cara, M. (2015, October 23). VR tests could diagnose very early onset Alzeimers. Retrieved from https://www.wired.co.uk/article/alzheimers-virtual-reality.

  28. Universal Health Network. (2015, September 23). Retrieved from https://www.uhn.ca/corporate/News/Pages/more_than_a_videogame_virtual_reality_helps_eye_research.aspx.

  29. Xu, X., Chen, K. B., Lin, J. H., & Radwin, R. G. (2015). The accuracy of the Oculus Rift virtual reality head-mounted display during cervical spine mobility measurement. Journal of Biomechanics, 48(4), 721–724.

    Article  Google Scholar 

  30. Strickland, D. (1997). Virtual reality for the treatment of autism. Studies in Health Technology and Informatics, 81–86.

    Google Scholar 

  31. Mirelman, A., Maidan, I., Herman, T., Deutsch, J. E., Giladi, N., & Hausdorff, J. M. (2011). Virtual reality for gait training: can it induce motor learning to enhance complex walking and reduce fall risk in patients with Parkinson’s disease? The Journals of Gerontology: Series A, 66(2), 234–240.

    Article  Google Scholar 

  32. White, P. J., & Moussavi, Z. (2016). Neurocognitive treatment for a patient with Alzheimer’s disease using a virtual reality navigational environment. Journal of experimental neuroscience, 10, JEN-S40827.

    Google Scholar 

  33. Gega, L. (2017). The virtues of virtual reality in exposure therapy. The British Journal of Psychiatry, 210(4), 245–246.

    Article  Google Scholar 

  34. Visual Search 2. (1995). Proceedings of the 2nd International Conference on Visual Search, p. 270, Optican.

    Google Scholar 

  35. AltDev Blog, John Carmack. (2013, February 22). Latency mitigation strategies. [Blog Post]. Retrieved from https://web.archive.org/web/20140719053303/; http://www.altdev.co/2013/02/22/latency-mitigation-strategies/.

  36. Rambling in Valve Time, Abrash, M. (2012, December 29). Latency—The sine qua non of AR and VR. [Blog post]. Retrieved from http://blogs.valvesoftware.com/abrash/latency-the-sine-qua-non-of-ar-and-vr/.

  37. Kanter, D. (2015). Graphics processing requirements for enabling immersive Vr.

    Google Scholar 

  38. LaValle, S. M. (2016). Visual rendering, virtual reality (Chap. 7). Retrieved from http://msl.cs.uiuc.edu/vr/vrch3.pdf.

  39. Duchowski, A. T. (2018). Gaze-based interaction: A 30 year retrospective. Computers & Graphics. https://doi.org/10.1016/j.cag.2018.04.002.

    Article  Google Scholar 

  40. Xu, Y., Dong, Y., Wu, J., Sun, Z., Shi, Z., Yu, J., & Gao, S. (2018). Gaze prediction in dynamic 360° immersive videos. CVPR.T.

    Google Scholar 

  41. Land, M. F. (2004). The coordination of rotations of the eyes head and trunk in saccadic turns produced in natural situations. Experimental Brain Research, 159(2), 151–160.

    Article  Google Scholar 

  42. Zhang, M., et al. (2017). Deep future gaze: gaze anticipation on egocentric videos using adversarial networks. CVPR.

    Google Scholar 

  43. Guenter, B., Finch, M., Drucker, S., Tan, D., and Snyder, J. 2012. Foveated 3D graphics. ACM Transactions on Graphics 31, 6, 164:1–164:10.

    Article  Google Scholar 

  44. Levoy, M., & Whitaker, R. (1989). Gaze-directed volume rendering. Technical report, University of North Carolina.

    Google Scholar 

  45. Ohshima, T., Yamamoto, H., & Tamura, H. (1996). Gaze directed adaptive rendering for interacting with virtual space. In Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), IEEE Computer Society, Washington, DC, USA, VRAIS ’96 (pp. 103–110).

    Google Scholar 

  46. Patney, A., Kim, J., Salvi, M., Anton Kaplanyan, M., Wyman, C., Benty, N., Lefohn, A., Luebke, D. (2016). Perceptually-based foveated virtual reality. In ACM SIGGRAPH 2016 Emerging Technologies (SIGGRAPH ‘16). ACM, New York, NY, USA, Article 17, 2 pp. https://doi.org/10.1145/2929464.2929472.

  47. Oculus Go. Fixed foveated rendering. Documentation. Retrieved from https://developer.oculus.com/documentation/unreal/latest/concepts/unreal-ffr/.

  48. Wang, Y.-Z., Bradley, A., & Thibos, L. N. (1997). Aliased frequencies enable the discrimination of compound gratings in peripheral vision. Vision Research, 37(3), 283–290.

    Article  Google Scholar 

  49. Albert, R., Patney, A., Luebke, D., & Kim, J. (2017). Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception, 14(4), 1–13. https://doi.org/10.1145/3127589.

    Article  Google Scholar 

  50. Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35, 6, Article 179 (November 2016), 12 pp. DOI:https://doi.org/10.1145/2980179.2980246.

    Article  Google Scholar 

  51. Williams, L. (1983). Pyramidal parametrics. SIGGRAPH. Computational Graphics, 17(3), 1–11.

    Article  Google Scholar 

  52. Olano, M., & Baker, D. (2010). Lean mapping. In Symposium on Interactive 3D Graphics and Games (pp. 181–188).

    Google Scholar 

  53. Lauritzne, A., Salvi, M., & Lefohn, A. (2011). Sample distribution shadow maps. In Symposium on Interactive 3D Graphics and Games (pp. 97–102).

    Google Scholar 

  54. Karis, B. (2014). High-quality temporal supersampling. In Advances in Real-Time Rendering in Games, SIGGRAPH Courses.

    Google Scholar 

  55. Fridman, L., Jenik, B., Keshvari, S., Reimer, B., Zetzsche, C., Rosenholtz, R. (2017). SideEye: A generative neural network based simulator of human peripheral vision. arXiv:1706.04568v2 [cs.NE].

  56. Albert, R., Patney, A., Luebke, D., Kim, J. (2017). Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception, 14(4), Article 25 (September 2017), 13 pp.

    Article  Google Scholar 

  57. Chaitanya, C. R. A., Kaplanyan, A. S., Schied, C., Salvi, M., Lefohn, A., Nowrouzezahrai, D., & Aila, T. (2017). Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics, 36(4), Article 98 (July 2017), 12 pp. https://doi.org/10.1145/3072959.3073601.

    Article  Google Scholar 

  58. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. ArXiv e-prints.

    Google Scholar 

  59. LaValle, S. M. (2016). Bird’s eye view, virtual reality (Chap. 2). Retrieved from http://msl.cs.uiuc.edu/vr/vrch2.pdf.

  60. Road to VR. Dr Morgan McGuire. (2017, 29 November). How NVIDIA Research is reinventing the display pipeline for the future of VR. [Blog Post]. Retreived from https://www.roadtovr.com/exclusive-how-nvidia-research-is-reinventing-the-display-pipeline-for-the-future-of-vr-part-1

  61. Peripheral Vision. Retrieved from https://commons.wikimedia.org/wiki/File:Peripheral_vision.svg#filelinks

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Netaji Subhas University of Technology (Formerly Netaji Subhas Institute of Technology), New Delhi, India

    Deepak Kumar Sharma, Arjun Khera & Dharmesh Singh

Authors
  1. Deepak Kumar Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arjun Khera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dharmesh Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deepak Kumar Sharma .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Maharaja Agrasen Institute of Technology, Guru Gobind Singh Indraprastha University, New Delhi, India

    Deepak Gupta

  2. Faculty of Computers and Information, Cairo University, Cairo, Egypt

    Aboul Ella Hassanien

  3. Department of Computer Science and Engineering, Maharaja Agrasen Institute of Technology, Guru Gobind Singh Indraprastha University, New Delhi, India

    Ashish Khanna

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sharma, D.K., Khera, A., Singh, D. (2020). Using Artificial Intelligence to Bring Accurate Real-Time Simulation to Virtual Reality. In: Gupta, D., Hassanien, A., Khanna, A. (eds) Advanced Computational Intelligence Techniques for Virtual Reality in Healthcare. Studies in Computational Intelligence, vol 875. Springer, Cham. https://doi.org/10.1007/978-3-030-35252-3_8

Download citation

Publish with us

Policies and ethics

Search

Navigation