Skip to main content
SpringerLink
Log in
Book cover

The Engineering of Mixed Reality Systems pp 33–55Cite as

A Holistic Approach to Design and Evaluation of Mixed Reality Systems

  • Chapter
  • First Online:

Part of the book series: Human-Computer Interaction Series ((HCIS))

Abstract

This chapter addresses issues related to usability and user experience of mixed reality (MR) systems based on a naturalistic iterative design approach to the development of MR applications. Design and evaluation of MR applications are still mostly based on methods used for development of more traditional desktop graphical user interfaces. MR systems are in many aspects very different from desktop computer applications, so these traditional methods are not sufficient for MR applications. There is a need for new approaches to user-centred design and development of MR systems. One such approach is based on the concepts of cognitive systems engineering (CSE). In this chapter we show how this approach can be applied to the development of MR systems. Two case studies are described, where a holistic CSE approach to design, implementation and evaluation has been used. The results show that allowing real end users (field/domain experts) to interact in a close to naturalistic setting provides insights on how to design MR applications that are difficult to attain otherwise. We also show the importance of iterative design, again involving real end users.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    All participant excerpts are translated from Swedish to English.

  2. 2.

    http://www.3dvisor.com/

References

  1. Azuma, R.: A survey of augmented reality. Presence 6(4), 355–385 (1997)

    Google Scholar 

  2. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Computer Graphics and Applications 21(6), 34–47 (2001). http://computer.org/cga/cg2001/g6034abs.htm

    Google Scholar 

  3. Billinghurst, M., Kato, H.: Collaborative augmented reality. Communications of the ACM 45(7), 64–70 (2002)

    Article  Google Scholar 

  4. Clark, H.H.: Using Language. Cambridge University Press, Cambridge (1996)

    Book  Google Scholar 

  5. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3) (1989)

    Google Scholar 

  6. Dekker, S., Hollnagel, E.: Human factors and folk models. Cognition Technology Work 6(2), 79–86 (2004). DOIhttp://dx.doi.org/10.1007/s10111-003-0136-9

  7. Dünser, A., Grasset, R., Billinghurst, M.: A survey of evaluation techniques used in augmented reality studies. Tech. Rep. Technical Report TR2008-02, Human Interface Technology Laboratory New Zealand (2008)

    Google Scholar 

  8. Engeström, Y., Miettinen, R., Punamaki, R. (eds.): Perspectives on activity theory (Learning in doing social, cognitive and computational perspectives). Cambridge University Press, Cambridge (1999)

    Google Scholar 

  9. Fiala, M.: Artag rev2 fiducial marker system: Vision based tracking for AR. In: Workshop of Industrial Augmented Reality, Wienna Austria (2005)

    Google Scholar 

  10. Fuhrmann, A., Löffelmann, H., Schmalstieg, D.: Collaborative augmented reality: Exploring dynamical systems. In: R. Yagel, H. Hagen (eds.) IEEE Visualization 97, pp. 459–462. IEEE (1997)

    Google Scholar 

  11. Gabbard, J.L., Swan II, J.E.: Usability engineering for augmented reality: Employing user-based studies to inform design. IEEE Transactions on Visualization and Computer Graphics 14(3), 513–525 (2008)

    Article  Google Scholar 

  12. Glesne, C., Peshkin, A.: Becoming qualitative researchers: An introduction. Longman White Plains, New york (1992)

    Google Scholar 

  13. Granlund, R.: Web-based micro-world simulation for emergency management training. Future Generation Computer Systems 17, 561–572 (2001)

    Article  Google Scholar 

  14. Grasset, R., Lamb, P., Billinghurst, M.: Evaluation of mixed-space collaboration. In: ISMAR ’05: Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 90–99. IEEE Computer Society, Washington, DC, USA (2005). DOI http://dx.doi.org/10.1109/ISMAR.2005.30

  15. HITLAB: http://www.hitl.washington.edu/artoolkit/ (2007)

  16. Hollnagel, E., Woods, D.D.: Cognitive systems engineering: New wine in new bottles. International Journal of Man-Machine Studies 18(6), 583–600 (1983)

    Article  Google Scholar 

  17. Hollnagel, E., Woods, D.D.: Joint cognitive systems: Foundations of cognitive systems engineering. CRC Press, Boca Raton, FL (2005)

    Chapter  Google Scholar 

  18. Hutchins, E.: Cognition in the wild. MIT Press, London, England (1995). URL http://www.netlibrary.com/summary.asp?id=1687

  19. Klein, G., Feltovich, P.J., Bradshaw, J.M., Woods, D.D.: Common ground and coordination in joint activity, vol. Organizational Simulation, Chap. 6, pp. 139–184. John Wiley & Sons, New York (2005). http://dx.doi.org/10.1002/0471739448.ch6

  20. Liao, Z., Landry, R.: An empirical study on organizational acceptance of new information systems in a commercial bank. In: HICSS (2000). URL http://computer.org/proceedings/hicss/0493/04932/04932021abs.htm

  21. Lincoln, Y.S., Guba, E.G.: Naturalistic Inquiry. Sage, Beverly Hills, CA (1985)

    Google Scholar 

  22. Livingston, M.A.: Evaluating human factors in augmented reality systems. IEEE Computer Graphics and Applications 25(6), 6–9 (2005). URL http://doi.ieeecomputersociety.org/10.1109/MCG.2005.130

  23. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Transactions on Information Systems E77-D(12) (1994)

    Google Scholar 

  24. Neisser, U.: Cognition and reality. W.H. Freeman, San Francisc, CA (1976)

    Google Scholar 

  25. Nielsen, J.: Usability Engineering. Academic Press, Boston, MA (1993)

    MATH  Google Scholar 

  26. Nilsson, S., Johansson, B.: User experience and acceptance of a mixed reality system in a naturalistic setting – A case study. In: ISMAR, pp. 247–248. IEEE (2006). URL http://dx.doi.org/10.1109/ISMAR.2006.297827

  27. Nilsson, S., Johansson, B.: Fun and usable: augmented reality instructions in a hospital setting. In: B. Thomas (ed.) Proceedings of the 2007 Australasian Computer-Human Interaction Conference, OZCHI 2007, Adelaide, Australia, November 28–30, 2007, ACM International Conference Proceeding Series, vol. 251, pp. 123–130. ACM (2007). http://doi.acm.org/10.1145/1324892.1324915

  28. Nilsson, S., Johansson, B.: Acceptance of augmented reality instructions in a real work setting. In: M. Czerwinski, A.M. Lund, D.S. Tan (eds.) Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, Florence, Italy, April 5-10, 2008, pp. 2025–2032. ACM (2008). http://doi.acm.org/10.1145/1358628.1358633

  29. Patton, M.Q.: Qualitative evaluation and research methods, (2nd edn). Sage Publications, Thousand Oaks, CA (1990)

    Google Scholar 

  30. Qvarfordt, P., Jönsson, A., Dahlbäck, N.: The role of spoken feedback in experiencing multimodal interfaces as human-like. In: Proceedings of ICMI‧03, Vancouver, Canada (2003)

    Google Scholar 

  31. Reichardt, C., Cook, T.: Beyond qualitative versus quantitative methods. Qualitative and Quantitative Methods in Evaluation Research pp. 7–32 (1979)

    Google Scholar 

  32. Schmalstieg, D.: Rapid prototyping of augmented reality applications with the studierstube framework. In: Workshop of Industrial Augmented Reality, Wienna, Austria (2005)

    Google Scholar 

  33. Shneiderman, B.: Designing the user interface (3rd edn). Addison Wesley (1998)

    Google Scholar 

  34. Suchman, L.A.: Plans and situated actions. Cambridge University Press, Cambridge (1987)

    Google Scholar 

  35. Sutherland, I.E.: A head-mounted three-dimensional display. In: AFIPS Conference Proceedings, vol. 33, pp. 757–764 (1968)

    Google Scholar 

  36. Swan II, J.E., Gabbard, J.L.: Survey of user-based experimentation in augmented reality. In: 1st International Conference on Virtual Reality, Las Vegas, NV (2005)

    Google Scholar 

  37. Träskbäck, M.: Toward a usable mixed reality authoring tool. In: VL/HCC, pp. 160–162. IEEE Computer Society (2004). URL http://doi.ieeecomputersociety.org/10.1109/VLHCC.2004.60

  38. Tushman, M.L., OReilly III, C.A.: Ambidextrous organizations: Managing evolutionary and revolutionary change. California Management Review 38(4), 8–30 (1996)

    Google Scholar 

  39. Vygotsky, L.S.: Mind in society. Harvard University Press, Cambridge (1978)

    Google Scholar 

  40. Woods, D.D., Roth, E.M.: Cognitive engineering: human problem solving with tools. Human Factors 30(4), 415–430 (1988)

    Google Scholar 

Download references

Acknowledgements

This research is funded by the Swedish Defence Materiel Administration (FMV). The MR system was developed in close cooperation with XM Reality AB. We are deeply indebted to all the participants in our studies who volunteered their time and expertise to these projects.

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, Linköping University, Linköping, Sweden

    Susanna Nilsson

  2. Department of Computer and Information Science, Linköping University, SaabSecurity, Santa Anna IT Research Institute AB, Linköping, Sweden

    Björn Johansson & Arne Jönsson

Authors
  1. Susanna Nilsson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Björn Johansson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arne Jönsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susanna Nilsson .

Editor information

Editors and Affiliations

  1. Institut de Recherches en, Université Toulouse III, route de Narbonne 118 , Toulouse CX 4, 31062, France

    Emmanuel Dubois

  2. Dept. Computing Science, University of Glasgow, Lilybank Gdns. 17, Glasgow, G12 8QQ, United Kingdom

    Philip Gray

  3. Labo. d'Informatique de Grenoble (LIG), Université Grenoble I, rue de la Bibliotheque 385, Grenoble, 38041, France

    Laurence Nigay

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag London

About this chapter

Cite this chapter

Nilsson, S., Johansson, B., Jönsson, A. (2010). A Holistic Approach to Design and Evaluation of Mixed Reality Systems. In: Dubois, E., Gray , P., Nigay, L. (eds) The Engineering of Mixed Reality Systems. Human-Computer Interaction Series. Springer, London. https://doi.org/10.1007/978-1-84882-733-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-84882-733-2_3

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84882-732-5

  • Online ISBN: 978-1-84882-733-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation