Framework for developing alternative reality environments to engineer large, complex systems

Abstract

We present a framework for alternative reality (XR) technologies to enable an understanding of what constitutes an XR environment when used in the context of design and engineering of large, complex systems. The framework provides guidelines for implementing the corresponding desired human sensory experience. This work is founded on the existing literature which defines theoretical Spectra, such as Fidelity, to systematically characterize an XR environment taxonomy. We identify landmarks for four XR categories within these Spectra and provide definitions that can used to establish a common vernacular. We further map these to specific human sensing modalities that are influenced by XR, such as tactility and vision, and define the technical requirements needed to augment the human experience in the desired XR environment. Finally, we connect the theoretical elements to the technical requirements to create an integrated XR framework. The utility of this framework is demonstrated in a case study addressing the use of XR technologies for five stakeholder groups involved in the evaluation of spacecraft habitat design and operations. This demonstrates the utility of the proposed XR taxonomy in a spacecraft habitat design process, which could be extended to other similar applications.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

adapted from Milgram and Kishino. Increasing virtual abstraction is achieved from left to right

Fig. 5

References

  1. Acemyan CZ, Kortum P (2018) Does the type of presentation medium impact assessments of the built environment? An examination of environmental usability ratings across three modes of presentation. J Environ Psychol 56(April):30–35. https://doi.org/10.1016/j.jenvp.2018.02.006

    Article  Google Scholar 

  2. Adams E (2017) Ford designers use microsoft hololens augmented reality to make better cars|WIRED. September 2017. https://www.wired.com/story/ford-design-microsoft-hololens/.

  3. Arayici Y, Coates P, Koskela L, Kagioglou M, Usher C, K. O’reilly. (2011) BIM adoption and implementation for architectural practices. Struct Surv 29(1):7–25

    Article  Google Scholar 

  4. Azuma RT (1997) A survey of augmented reality. Pres Teleoper Virt Environ 6(4):355–385

    Article  Google Scholar 

  5. Begault, D. R. T. (2000). 3-D Sound for virtual reality and multimedia. https://ntrs.nasa.gov/search.jsp?R=20010044352.

  6. Bowman DA, Ryan PM (2007) Virtual reality: How much immersion is enough? Computer 40(7):36–43. https://doi.org/10.1109/MC.2007.257

    Article  Google Scholar 

  7. Castronovo F, Dragana N, Liu Y, Messner J (2013) An evaluation of immersive virtual reality systems for design reviews. Comput Aided Design Appl 9:892

    Google Scholar 

  8. Cohen, M. (2012a). Mockups 101: Code and standard research for space habitat analogues. In: American Institute of Aeronautics and Astronautics. https://doi.org/10.2514/6.2012-5153.

  9. Cohen, M. (2012b). Mockups 101: Code and standard research for space habitat analogues. In: American Institute of Aeronautics and Astronautics. https://doi.org/10.2514/6.2012-5153.

  10. Dubois E, Philip G, Laurence N (eds) (2010) The engineering of mixed reality systems. Human–computer interaction series. Springer, London

  11. Eastman CM (ed) (2008) BIM handbook: a guide to building information modeling for owners, managers, designers, engineers, and contractors. Wiley, Hoboken

    Google Scholar 

  12. Engelberg D, Seffah A (2002) A Framework for rapid mid-fidelity prototyping of web sites. In: Hammond J, Gross T, Wesson J (eds) Usabilit, vol 99. Springer, Boston, pp 203–215. https://doi.org/10.1007/978-0-387-87035610-5_14

    Google Scholar 

  13. Fishkin KP (2004) A taxonomy for and analysis of tangible interfaces. Pers Ubiquit Comput 8(5):347. https://doi.org/10.1007/s00779-004-0297-4

    Article  Google Scholar 

  14. Goulding J, Rahimian F, Wang X (2014) Virtual reality-based cloud BIM platform for integrated AEC projects. J Inf Technol Constr 19:308–325

    Google Scholar 

  15. Gugenheimer J, Stemasov E, Frommel J, Rukzio E (2017). ShareVR: enabling co-located experiences for virtual reality between HMD and non-HMD users. In: ACM Press, pp 4021–433. https://doi.org/10.1145/3025453.3025683.

  16. Hale KS, Stanney KM, Malone L (2009) Enhancing virtual environment spatial awareness training and transfer through tactile and vestibular cues. Ergonomics 52(2):187–203. https://doi.org/10.1080/00140130802376000

    Article  Google Scholar 

  17. Hammond W (1999) Space transportation: A systems approach to analysis and design AIAA educational series. American Institute of Aeronautics and Astronautics, Reston

    Google Scholar 

  18. Hays T (1980) Simulation systems technical area. Technical Report 490. U.S. Army Research Institute of Behavioral and Social Sciences.

  19. Hettinger L, Riccio G (1992) Visually induced motion sickness in virtual environments. Pres Teleoper Virt Environ 1(3):306

    Article  Google Scholar 

  20. Heydarian A, Carneiro JP, Gerber D, Becerik-Gerber B, Hayes T, Wood W (2015) Immersive virtual environments versus physical built environments: a benchmarking study for building design and user-built environment explorations. Autom Constr 54(June):116–126. https://doi.org/10.1016/j.autcon.2015.03.020

    Article  Google Scholar 

  21. Higdon KP, Klaus DM (2008) Effective integration of rapid prototyping into multidisciplinary design optimization for the development of human spacecraft. In: American Society of Civil Engineers, pp 1–10. https://doi.org/10.1061/40988(323)96.

  22. Ishii H (2008) Tangible bits: beyond pixels. In: xv. ACM Press. https://doi.org/10.1145/1347390.1347392.

  23. Kanas N, Manzey D (2008) Space psychology and psychiatry. Space technology library, 2nd edn. Springer, El Segundo

    Google Scholar 

  24. Kelly DH (1979) Motion and vision. II. Stabilized spatio-temporal threshold surface. J Opt Soc Am 69(10):10

    Article  Google Scholar 

  25. Kim M, Jeon C, Kim J (2017) A study on immersion and presence of a portable hand haptic system for immersive virtual reality. Sensors; Basel 17(5):1141

    Article  Google Scholar 

  26. Klaus DM, Higdon KP (2009) Academic principles of human space habitat design. In: International conference on environmental systems, vol 8

  27. Larsson P, Väljamäe A, Västfjäll D, Tajadura-Jiménez A, Kleiner M (2010) Auditory-induced presence in mixed reality environments and related technology. In: Dubois E, Gray P, Nigay L (eds) The engineering of mixed reality systems. Springer, London, pp 143–163. https://doi.org/10.1007/978-1-84882-733-2_8

    Google Scholar 

  28. Leithinger D, Follmer S, Olwal A, Ishii H (2015) Shape displays: spatial interaction with dynamic physical form. IEEE Comput Graph Appl 35(5):5–11. https://doi.org/10.1109/MCG.2015.111

    Article  Google Scholar 

  29. Lentz T, Schröder D, Vorländer M, Assenmacher I (2007) Virtual reality system with integrated sound field simulation and reproduction. EURASIP J Adv Signal Process 2007:1. https://doi.org/10.1155/2007/70540

    Article  Google Scholar 

  30. Liu D, Macchiarella ND, Vincenzi DA (2009) Simulation fidelity. Hum Factors Simul Training 13:561

    Google Scholar 

  31. Mackay, W. (1998). Augmented reality: Linking real and virtual worlds a new paradigm for interacting with computers. In: ACM conference on advanced visual interfaces Proceedings of AVI’9. ACM Press, pp 13–21.

  32. Maldovan KD, Messner JI, Faddoul M (2006) Framework for reviewing mockups in an immersive environment, vol 7. https://www.researchgate.net/profile/John_Messner2/publication/228399774_Framework_for_Reviewing_Mockups_in_an_Immersive_Environment/links/004635231e625e739f000000.pdf

  33. Milgram P, Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst 77(12):1321–1329

    Google Scholar 

  34. Mugge W, David AA, Alfred CS, van der Frans CT, Helm JHA, Carel GMM (2013) Force control in the absence of visual and tactile feedback. Exp Brain Res 224(4):635–645. https://doi.org/10.1007/s00221-012-3341-z

    Article  Google Scholar 

  35. Mulenburg GM, Gundo DP (2002) Design by Prototype: examples from the National Aeronautics and Space Administration. Miami, FL.

  36. NASA (1995) NASA systems engineering handbook. NASA.

  37. NASA (2011) NASA space flight human system standard volume 2: Human factors, habitability, and environmental health. NASA-STD-3001 VOL II.

  38. Nolle S, Gudrun K (2006) Augmented reality as a comparison tool in automotive industry. In: IEEE, pp 249–250. https://doi.org/10.1109/ISMAR.2006.297829.

  39. Paige JB, Morin KH (2013) Simulation fidelity and cueing: a systematic review of the literature. Clin Simul Nurs 9(11):e481–e489. https://doi.org/10.1016/j.ecns.2013.01.001

    Article  Google Scholar 

  40. Pamungkas DS, Ward K (2016) Electro-tactile feedback system to enhance virtual reality experience. Int J Comput Theory Eng 8(6):465–470. https://doi.org/10.7763/IJCTE.2016.V8.1090

    Article  Google Scholar 

  41. Picinali L, Amandine A, Michel D, Brian FGK (2014) Exploration of architectural spaces by blind people using auditory virtual reality for the construction of spatial knowledge. Int J Hum Comput Stud 72(4):393–407. https://doi.org/10.1016/j.ijhcs.2013.12.008

    Article  Google Scholar 

  42. Pontonnier C, Dumont G, Samani A, Madeleine P, Badawi M (2014) Designing and evaluating a workstation in real and virtual environment: toward virtual reality based ergonomic design sessions. J Multimodal User Interfaces 8(2):199–208. https://doi.org/10.1007/s12193-013-0138-8

    Article  Google Scholar 

  43. Riecke BE, Jörg S-P, Franck C, Heinrich H B (n.d.) Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality, vol 9. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.490.7866&rep=rep1&type=pdf

  44. Robinett W (1992) Synthetic experience: a proposed taxonomy. Pres Teleoper Virt Environ 1(2):229–247

    Article  Google Scholar 

  45. Rudd J, Stern K, Isensee S (1996) Low vs. high-fidelity prototyping debate. Interactions 3:76–85

    Article  Google Scholar 

  46. Ryu J, Naoki H, Makoto S, Masashi S, Ryuzo O (2007) Application of human-scale immersive VR system for environmental design assessment: a proposal for an architectural design evaluation tool. J Asian Archit Build Eng 6(1):57–64. https://doi.org/10.3130/jaabe.6.57

    Article  Google Scholar 

  47. Seth A, Vance JM, Oliver JH (2011) Virtual reality for assembly methods prototyping: a review. Virt Real 15(1):5–20. https://doi.org/10.1007/s10055-009-0153-y

    Article  Google Scholar 

  48. Sheridan TB (1992) Musings on telepresence and virtual presence. Pres Teleoper Virt Environ 1(1):120–126

    Article  Google Scholar 

  49. Västfjäll D (2003) The subjective sense of presence, emotion recognition, and experienced emotions in auditory virtual environments. Cyber Psychol Behav 6(2):181–188. https://doi.org/10.1089/109493103321640374

    Article  Google Scholar 

  50. Whitmore M, McGuire K, Margerum S, Thompson S, Allen C, Bowen C, Adelstein B, Schuh S, Byrne V, Wong D (2013) Evidence report: risk of incompatible vehicle/habitat design. NASA Human Research Program, Johnson Space Center, Houston

    Google Scholar 

  51. Zeltzer D (1992) Autonomy, interaction, and presence. Pres Teleoper Virt Environ 1(1):127–132

    Article  Google Scholar 

  52. Zimmermann P (2008) Virtual reality aided design: A survey of the use of VR in automotive industry. In: Talaba D, Amditis A (eds) Product engineering. Springer, Dordrecht, pp 277–296

    Google Scholar 

Download references

Acknowledgements

This work was funded by the NASA Human Research Program, Grant 80NSSC18K0198.

Author information

Affiliations

Authors

Corresponding authors

Correspondence to Allison Anderson or Abhishektha Boppana.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: interview protocol

Appendix 1: interview protocol

figurea
figureb

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Anderson, A., Boppana, A., Wall, R. et al. Framework for developing alternative reality environments to engineer large, complex systems. Virtual Reality 25, 147–163 (2021). https://doi.org/10.1007/s10055-020-00448-4

Download citation

Keywords

  • Virtual reality
  • Augmented reality
  • Hybrid reality
  • Spacecraft design