Advertisement

Using Artificial Intelligence to Bring Accurate Real-Time Simulation to Virtual Reality

  • Deepak Kumar SharmaEmail author
  • Arjun Khera
  • Dharmesh Singh
Chapter
First Online:
  • 300 Downloads
Part of the Studies in Computational Intelligence book series (SCI, volume 875)

Abstract

There always has been an excruciating gap between theoretical possibilities, clinical trial and real world applications in the Medical Industry. Any new research, experimentation or training in this sector has always been subject to extreme scrutiny and legal intricacies, due to the complexity of the human body and any resulting complications that might arise from the application of prematurely tested techniques or tools. The introduction of Virtual Reality in the Medical Industry is bringing all these troubles to their heel. Simulations generated by virtual reality are currently being explored to impart education and practical medical experience to students and doctors alike, generate engaging environments for patients and thus assisting in various aspects ranging from treatment of medical conditions to rehabilitation. This book chapter aims to develop an understanding on how virtual reality is being applied in the healthcare industry. A formal study of various solutions for reducing the latency is presented along with research being done in the area for improving the performance and making the experience more immersive. It is evident that motion to photons latency plays a crucial role in determining a genuine virtual reality experience. Among many, foveated rendering and gaze tracking systems seem to be the most promising in creating exciting opportunities for virtual reality systems in the future.

This is a preview of subscription content, log in to check access.

References

  1. 1.
    Sutherland, I. E. (1965). The ultimate display. Proceedings of IFIP 65, 2, 506–508.Google Scholar
  2. 2.
    Fuchs, H., Bishop, G., et al. (1992). Research directions in virtual environments. NFS Invitational Workshop, University North Carolina.Google Scholar
  3. 3.
    Gigante, M. (1993). Virtual reality: Definitions, history and applications. Virtual Reality Systems, 3–14. ISBN 0-12-22-77-48-1.Google Scholar
  4. 4.
    Steur, J. (1995). Defining virtual reality: Dimensions determining telepresence. In F. L. Biocca (Ed.), Communication in the age of virtual reality. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  5. 5.
    Briggs, J. C. (1996). The promise of virtual reality. The Futurist, 30.Google Scholar
  6. 6.
    Alqahtani, A. S., Daghestani, L., & Ibrahim, L. F. (2017). Environments and system types of virtual reality technology in STEM: A survey.Google Scholar
  7. 7.
    Alqahtani, A., Daghestani, L., & Ibrahim, L. F. (2017). Environments and system types of virtual reality technology in STEM: A survey.Google Scholar
  8. 8.
    Shakibamanesh, A. (2014). Improving results of urban design research by enhancing advanced semi-experiments in virtual environments. IJAUP, 24(2), 131–141.Google Scholar
  9. 9.
    Fairén González, M., Farrés, M., Moyes Ardiaca, J., & Insa, E. (2017). Virtual reality to teach anatomy. In Eurographics 2017: education papers (pp. 51–58). European Association for Computer Graphics (Eurographics).Google Scholar
  10. 10.
    Codd, A. M., & Choudhury, B. (2011). Virtual reality anatomy: Is it comparable with traditional methods in the teaching of human forearm musculoskeletal anatomy? Anatomical Sciences Education, 4(3), 119–125.CrossRefGoogle Scholar
  11. 11.
    Basdogan, C., & Srinivasan, M. A. (2002). Haptic rendering in virtual environments. In Handbook of virtual environments (pp. 157–174). CRC Press.Google Scholar
  12. 12.
    Górski, F., Buń, P., Wichniarek, R., Zawadzki, P., & Hamrol, A. (2015). Immersive city bus configuration system for marketing and sales education. Procedia Computer Science, 75, 137–146.CrossRefGoogle Scholar
  13. 13.
    Second Life [Online Virtual World]. (2019). Accessible from http://secondlife.com.
  14. 14.
    Life, S. (2017). An overview of the potential of 3-D virtual worlds in medical and health education. Health Information and Libraries Journal, 24(4), 233–245.Google Scholar
  15. 15.
    Wii [Video Game Console]. (2019). Retrieved from URL: http://wii.com.
  16. 16.
    The Future of Healthcare Communication. (2007). Second health. [Online Virtual World]. Retrieved from URL: http://secondhealth.wordpress.com.
  17. 17.
    Hansen, M. (2008). Versatile, immersive, creative and dynamic virtual 3-D healthcare learning environments: a review of the literature. Journal of medical Internet research, 10(3), e26.CrossRefGoogle Scholar
  18. 18.
    Aïm, F., Lonjon, G., Hannouche, D., & Nizard, R. (2016). Effectiveness of virtual reality training in orthopaedic surgery. Arthroscopy: the journal of arthroscopic & related surgery, 32(1), 224–232.Google Scholar
  19. 19.
    Mohan, P. V. R., & Chaudhry, R. (2009). Laparoscopic simulators: Are they useful! Medical Journal Armed Forces India, 65(2), 113–117.CrossRefGoogle Scholar
  20. 20.
    Lap Mentor [Simulator]. (2017). Accessible from https://simbionix.com/simulators/lap-mentor/.
  21. 21.
    LapSim [Simulator]. (2018). Accessible from https://surgicalscience.com/systems/lapsim/.
  22. 22.
    Simendo [Simulator]. (2018). Accessible from https://www.simendo.eu.
  23. 23.
    Wilson, M. S., Middlebrook, A., Sutton, C., Stone, R., & McCloy, R. F. (1997). MIST VR: A virtual reality trainer for laparoscopic surgery assesses performance. Annals of the Royal College of Surgeons of England, 79(6), 403.Google Scholar
  24. 24.
    Vaughan, N., Dubey, V. N., Wainwright, T. W., & Middleton, R. G. (2016). A review of virtual reality based training simulators for orthopaedic surgery. Medical Engineering & Physics, 38(2), 59–71.CrossRefGoogle Scholar
  25. 25.
    SimOrtho [Simulator]. (2019). Retrieved from https://ossimtech.com/en-us/Simulators.
  26. 26.
    Salb, T., Weyrich, T., & Dillmann, R. (1999, April). Preoperative planning and training simulation for risk reducing surgery. In International Training and Education Conference (ITEC).Google Scholar
  27. 27.
    Cara, M. (2015, October 23). VR tests could diagnose very early onset Alzeimers. Retrieved from https://www.wired.co.uk/article/alzheimers-virtual-reality.
  28. 28.
  29. 29.
    Xu, X., Chen, K. B., Lin, J. H., & Radwin, R. G. (2015). The accuracy of the Oculus Rift virtual reality head-mounted display during cervical spine mobility measurement. Journal of Biomechanics, 48(4), 721–724.CrossRefGoogle Scholar
  30. 30.
    Strickland, D. (1997). Virtual reality for the treatment of autism. Studies in Health Technology and Informatics, 81–86.Google Scholar
  31. 31.
    Mirelman, A., Maidan, I., Herman, T., Deutsch, J. E., Giladi, N., & Hausdorff, J. M. (2011). Virtual reality for gait training: can it induce motor learning to enhance complex walking and reduce fall risk in patients with Parkinson’s disease? The Journals of Gerontology: Series A, 66(2), 234–240.CrossRefGoogle Scholar
  32. 32.
    White, P. J., & Moussavi, Z. (2016). Neurocognitive treatment for a patient with Alzheimer’s disease using a virtual reality navigational environment. Journal of experimental neuroscience, 10, JEN-S40827.Google Scholar
  33. 33.
    Gega, L. (2017). The virtues of virtual reality in exposure therapy. The British Journal of Psychiatry, 210(4), 245–246.CrossRefGoogle Scholar
  34. 34.
    Visual Search 2. (1995). Proceedings of the 2nd International Conference on Visual Search, p. 270, Optican.Google Scholar
  35. 35.
    AltDev Blog, John Carmack. (2013, February 22). Latency mitigation strategies. [Blog Post]. Retrieved from https://web.archive.org/web/20140719053303/; http://www.altdev.co/2013/02/22/latency-mitigation-strategies/.
  36. 36.
    Rambling in Valve Time, Abrash, M. (2012, December 29). Latency—The sine qua non of AR and VR. [Blog post]. Retrieved from http://blogs.valvesoftware.com/abrash/latency-the-sine-qua-non-of-ar-and-vr/.
  37. 37.
    Kanter, D. (2015). Graphics processing requirements for enabling immersive Vr.Google Scholar
  38. 38.
    LaValle, S. M. (2016). Visual rendering, virtual reality (Chap. 7). Retrieved from http://msl.cs.uiuc.edu/vr/vrch3.pdf.
  39. 39.
    Duchowski, A. T. (2018). Gaze-based interaction: A 30 year retrospective. Computers & Graphics.  https://doi.org/10.1016/j.cag.2018.04.002.CrossRefGoogle Scholar
  40. 40.
    Xu, Y., Dong, Y., Wu, J., Sun, Z., Shi, Z., Yu, J., & Gao, S. (2018). Gaze prediction in dynamic 360° immersive videos. CVPR.T.Google Scholar
  41. 41.
    Land, M. F. (2004). The coordination of rotations of the eyes head and trunk in saccadic turns produced in natural situations. Experimental Brain Research, 159(2), 151–160.CrossRefGoogle Scholar
  42. 42.
    Zhang, M., et al. (2017). Deep future gaze: gaze anticipation on egocentric videos using adversarial networks. CVPR.Google Scholar
  43. 43.
    Guenter, B., Finch, M., Drucker, S., Tan, D., and Snyder, J. 2012. Foveated 3D graphics. ACM Transactions on Graphics 31, 6, 164:1–164:10.CrossRefGoogle Scholar
  44. 44.
    Levoy, M., & Whitaker, R. (1989). Gaze-directed volume rendering. Technical report, University of North Carolina.Google Scholar
  45. 45.
    Ohshima, T., Yamamoto, H., & Tamura, H. (1996). Gaze directed adaptive rendering for interacting with virtual space. In Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), IEEE Computer Society, Washington, DC, USA, VRAIS ’96 (pp. 103–110).Google Scholar
  46. 46.
    Patney, A., Kim, J., Salvi, M., Anton Kaplanyan, M., Wyman, C., Benty, N., Lefohn, A., Luebke, D. (2016). Perceptually-based foveated virtual reality. In ACM SIGGRAPH 2016 Emerging Technologies (SIGGRAPH ‘16). ACM, New York, NY, USA, Article 17, 2 pp.  https://doi.org/10.1145/2929464.2929472.
  47. 47.
    Oculus Go. Fixed foveated rendering. Documentation. Retrieved from https://developer.oculus.com/documentation/unreal/latest/concepts/unreal-ffr/.
  48. 48.
    Wang, Y.-Z., Bradley, A., & Thibos, L. N. (1997). Aliased frequencies enable the discrimination of compound gratings in peripheral vision. Vision Research, 37(3), 283–290.CrossRefGoogle Scholar
  49. 49.
    Albert, R., Patney, A., Luebke, D., & Kim, J. (2017). Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception, 14(4), 1–13.  https://doi.org/10.1145/3127589.CrossRefGoogle Scholar
  50. 50.
    Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016). Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics, 35, 6, Article 179 (November 2016), 12 pp. DOI: https://doi.org/10.1145/2980179.2980246.CrossRefGoogle Scholar
  51. 51.
    Williams, L. (1983). Pyramidal parametrics. SIGGRAPH. Computational Graphics, 17(3), 1–11.CrossRefGoogle Scholar
  52. 52.
    Olano, M., & Baker, D. (2010). Lean mapping. In Symposium on Interactive 3D Graphics and Games (pp. 181–188).Google Scholar
  53. 53.
    Lauritzne, A., Salvi, M., & Lefohn, A. (2011). Sample distribution shadow maps. In Symposium on Interactive 3D Graphics and Games (pp. 97–102).Google Scholar
  54. 54.
    Karis, B. (2014). High-quality temporal supersampling. In Advances in Real-Time Rendering in Games, SIGGRAPH Courses.Google Scholar
  55. 55.
    Fridman, L., Jenik, B., Keshvari, S., Reimer, B., Zetzsche, C., Rosenholtz, R. (2017). SideEye: A generative neural network based simulator of human peripheral vision. arXiv:1706.04568v2 [cs.NE].
  56. 56.
    Albert, R., Patney, A., Luebke, D., Kim, J. (2017). Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception, 14(4), Article 25 (September 2017), 13 pp.CrossRefGoogle Scholar
  57. 57.
    Chaitanya, C. R. A., Kaplanyan, A. S., Schied, C., Salvi, M., Lefohn, A., Nowrouzezahrai, D., & Aila, T. (2017). Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics, 36(4), Article 98 (July 2017), 12 pp.  https://doi.org/10.1145/3072959.3073601.CrossRefGoogle Scholar
  58. 58.
    He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. ArXiv e-prints.Google Scholar
  59. 59.
    LaValle, S. M. (2016). Bird’s eye view, virtual reality (Chap. 2). Retrieved from http://msl.cs.uiuc.edu/vr/vrch2.pdf.
  60. 60.
    Road to VR. Dr Morgan McGuire. (2017, 29 November). How NVIDIA Research is reinventing the display pipeline for the future of VR. [Blog Post]. Retreived from https://www.roadtovr.com/exclusive-how-nvidia-research-is-reinventing-the-display-pipeline-for-the-future-of-vr-part-1
  61. 61.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Deepak Kumar Sharma
    • 1
    Email author
  • Arjun Khera
    • 1
  • Dharmesh Singh
    • 1
  1. 1.Department of Information TechnologyNetaji Subhas University of Technology (Formerly Netaji Subhas Institute of Technology)New DelhiIndia

Personalised recommendations

Cite chapter

Buy options