NERVE, InterPLAY, and Design-Based Research: Advancing Experiential Learning and the Design of Virtual Patient Simulation

Living reference work entry

Latest version View entry history



Systematic reviews and meta-analyses of randomized controlled studies conclude that virtual patient simulations (VPs) are consistently associated with higher learning outcomes compared to other educational methods, such as lectures, handouts, textbooks, and standardized patients (e.g., Consorti et al., Comput Educ 59(3):1001–1008, 2012; Cook and Triola Med Educ 43(4):303–311, 2009; McGaghie et al., Acad Med J Assoc Am Med Colleges 86(6):706, 2011). However, we cannot assume that students will learn by simply giving them access to the simulations. The instructional features that are integrated before, during, and after the simulations may affect student learning as much as or more so than the simulations. The strategy used to integrate the simulation into the curriculum and evaluate student performance may also have a significant effect on its use and learning. Here, we document the design, development, and testing of NERVE (a VPs created to develop medical students’ ability to examine, interview, and diagnose patients with cranial nerve disorders) in one definitive source and elaborate on what went on in each team members’ mind as the system evolved. Specifically, we examine the skills, knowledge, and dispositions called upon and the key lessons learned by team members during the last year of research and development. Concluding remarks related the individual accounts and discuss common findings to shed further insights on the team’s experience.


Virtual patient simulations Medical simulations Medical education Simulation-based training Design-based research Instructional design Instructional theory 



Research reported in chapter paper was supported by the National Institutes of Health (NIH) under award number 1R01LM010813–01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Supplementary material

Video 1

(MP4 4075 kb)

Video 2

(MP4 10302 kb)


  1. Adams, E., Rodgers, C., Harrington, R., Young, M., & Sieber, V. (2011). How we created virtual patient cases for primary care-based learning. Medical Teacher, 33(4), 273–278.CrossRefGoogle Scholar
  2. Agency for Healthcare Research and Quality (AHRQ). (2013). PAR-11-024 advances in patient safety through simulation research. Accessed January 20.
  3. Atkinson, T., & Hirumi, A. (2010). The game brain. In A. Hirumi (Ed.), Playing games in school: Using simulations and videogames for primary and secondary education (p. 63). Eugene, WA: International Society for Technology in Education.Google Scholar
  4. Bateman, J., Allen, M. E., Kidd, J., Parsons, N., & Davies, D. (2012). Virtual patients design and its effect on clinical reasoning and student experience: A protocol for a randomised factorial multi-centre study. Medical Education, 12(1), 62.Google Scholar
  5. Bateman, J., Allen, M. E., Samani, D., Kidd, J., & Davies, D. (2013). Virtual patient design: Exploring what works and why. A grounded theory study. Medical Education, 47(6), 595–606.CrossRefGoogle Scholar
  6. Berman, N., Fall, L., Smith, S., Levine, D., Maloney, C., Potts, M., … Foster-Johnson, L. (2009). Integration strategies for using virtual patients in clinical clerkships. Academic Medicine, 84(7), 943–949.CrossRefGoogle Scholar
  7. Botezatu, M., Hult, H., Tessma, M. K., & Fors, U. G. (2010). As time goes by: Stakeholder opinions on the implementation and use of a virtual patient simulation system. Medical Teacher, 32(11), e509–e516.CrossRefGoogle Scholar
  8. Cendan, J., & Lok, B. (2012). The use of virtual patients in medical school curricula. Advances in Physiology Education, 36(1), 48–53.CrossRefGoogle Scholar
  9. Churchill, D. (2007). Towards a useful classification of learning objects. Educational Technology Research and Development, 55(5), 479–497.CrossRefGoogle Scholar
  10. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13.CrossRefGoogle Scholar
  11. Colder, B. (2011). Emulation as an integrating principle for cognition. Frontiers in Computational Neuroscience, 5(54), 1–12.Google Scholar
  12. Consorti, F., Mancuso, R., Nocioni, M., & Piccolo, A. (2012). Efficacy of virtual patients in medical education: A meta-analysis of randomized studies. Computers & Education, 59(3), 1001–1008.CrossRefGoogle Scholar
  13. Cook, D., & Triola, M. (2009). Virtual patients: A critical literature review and proposed next steps. Medical Education, 43(4), 303–311.CrossRefGoogle Scholar
  14. Cook, D. A. (2014). How much evidence does it take? A cumulative meta-analysis of outcomes of simulation-based education. Medical Education, 48(8), 750–760.CrossRefGoogle Scholar
  15. Cook, D. A., Erwin, P. J., & Triola, M. M. (2010). Computerized virtual patients in health professions education: A systematic review and meta-analysis. Academic Medicine, 85, 15890–11602.Google Scholar
  16. Cooke, S. F. (2006). Plasticity in the human central nervous system. Brain, 129(Pt 7), 1659–1673.CrossRefGoogle Scholar
  17. Dewey, J. (1938). Logic: The theory of inquiry. New York: Holt, Rinehart and Winston.Google Scholar
  18. de Jong, T., & von Joolingen, W. R. (1998). Scientific discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68(3), 179–201.CrossRefGoogle Scholar
  19. Edelbring, S. (2010). A three-fold framework for relating to innovations and technology in education: Learning from, with and about technology. In A. Bromage, L. Clouder, J. Thistlethwaite, & F. Gordon (Eds.), Interprofessional E-learning and collaborative work: Practices and technologies (pp. 23–33). Hershey, PA: IGI Global.CrossRefGoogle Scholar
  20. Edelbring, S., Broström, O., Henriksson, P., Vassiliou, D., Spaak, J., Dahlgren, L. O., & Zary, N. (2012). Integrating virtual patients into courses: Follow-up seminars and perceived benefit. Medical Education, 46(4), 417–425.CrossRefGoogle Scholar
  21. Edelbring, S., Dastmalchi, M., Hult, H., Lundberg, I. E., & Dahlgren, L. O. (2011). Experiencing virtual patients in clinical learning: A phenomenological study. Advances in Health Sciences Education, 16(3), 331–345.CrossRefGoogle Scholar
  22. Fischer, M., Hege, I., Hörnlein, A., Puppe, F., Tönshoff, B., & Huwendiek, S. (2007). Virtual patients in medical education: A comparison of various strategies for curricular integration. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen, 102(10), 648–653.CrossRefGoogle Scholar
  23. Fullan, M. (1993). Change forces: Probing the depth of educational reform. London, UK: Falmer.Google Scholar
  24. Georg, C., & Zary, N. (2014). Web-based virtual patients in nursing education: Development and validation of theory-anchored design and activity models. Journal of Medical Internet Research, 16(4), e105. Scholar
  25. Gibbons, A. S., McConkie, M., Seo, K. K., & Wiley, D. A. (2009). Simulation approach to instruction. In C. M. Reigeluth & A. A. Carr-Chellman (Eds.), Instructional-design theories and models: building a common knowledge base (Vol. III, pp. 167–193). New York, NY: Routledge.Google Scholar
  26. Gormley, G. J., Mcglade, K., Thomson, C., Mcgill, M., & Sun, J. (2011). A virtual surgery in general practice: Evaluation of a novel undergraduate virtual patient learning package. Medical Teacher, 33(10), 522–527.CrossRefGoogle Scholar
  27. Haag, M., Singer, R., Bauch, M., Heid, J., Hess, F., & Leven, F. (2007). Challenges and perspectives of computer-assisted instruction in medical education. Lessons learned from seven years of experience with the CAMPUS system. Methods of Information in Medicine, 46(1), 67.CrossRefGoogle Scholar
  28. Hall, G. E., & Hord, S. M. (2001). Implementing change: Patterns, principles and potholes. Boston, MA: Allyn and Bacon.Google Scholar
  29. Harden, R. M., Grant, J., Buckley, E. G., & Hart, I. R. (1999). BEME guide no. 1: Best evidence medical education. Medical Teacher, 21(6), 553–562.CrossRefGoogle Scholar
  30. Harrison, M., Short, C., & Roberts, C. (2003). Reflecting on reflective learning: The case of geography, earth and environmental sciences. Journal of Geography in Higher Education, 27(2), 133–152.CrossRefGoogle Scholar
  31. Hirumi, A., Johnson, K., Kleinsmith, A., Reyes, R., Rivera-Gutierrez, D., Kubovec, S., … Cendan, J. (in press). Advancing virtual patient simulations and experiential learning with InterPLAY: Examining how theory informs design and design informs theory. Journal of Applied Instructional Design.Google Scholar
  32. Hirumi, A., Johnson, T., Reyes, R. J., Lok, B., Johnsen, K., Rivera-Gutierrez, D. J., et al. (2016). Advancing virtual patient simulations through design research and InterPLAY: Part II – testing and integration. Educational Technology, Research & Development.
  33. Hirumi, A., Kleinsmith, A., Johnsen, K., Kubovec, S., Eakins, M., Bogert, K., et al. (2016). Advancing virtual patient simulations through design research and InterPLAY: Part I – Design and development. Educational Technology, Research & Development, 64(4), 763–785.CrossRefGoogle Scholar
  34. Huwendiek, S., & De Leng, B. A. (2010). Virtual patient design and curricular integration evaluation toolkit. Medical Education, 44(5), 519.CrossRefGoogle Scholar
  35. Huwendiek, S., De Leng, B. A., Kononowicz, A. A., Kunzmann, R., Muijtjens, A. M., Van Der Vleuten, C. P., … Dolmans, D. H. (2015). Exploring the validity and reliability of a questionnaire for evaluating virtual patient design with a special emphasis on fostering clinical reasoning. Medical Teacher, 37(8), 775–782.CrossRefGoogle Scholar
  36. Huwendiek, S., Duncker, C., Reichert, F., De Leng, B. A., Dolmans, D., van der Vleuten, C. P., & Tönshoff, B. (2013). Learner preferences regarding integrating, sequencing and aligning virtual patients with advancing virtual patient simulations through design other activities in the undergraduate medical curriculum: A focus group study. Medical Teacher, 35(11), 920–929.CrossRefGoogle Scholar
  37. Huwendiek, S., Reichert, F., Bosse, H. M., De Leng, B. A., Van Der Vleuten, C. P., Haag, M., & Tönshoff, B. (2009). Design principles for virtual patients: A focus group study among students. Medical Education, 43(6), 580–588.CrossRefGoogle Scholar
  38. Illeris, K. (2014). Transformative learning and identity. [electronic resource]. Abingdon, Oxon: Routledge.Google Scholar
  39. Issenberg, S. B., McGaghie, W. C., Petrusa, E., Gordon, D. L., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27(1), 10–28.CrossRefGoogle Scholar
  40. Johnson, T. R., Lyons, R., Chuah, J., Kooper, R., Lok, B., & Cendan, J. C. (2013). Optimal learning in a virtual patient simulation of cranial nerve palsies: The interaction between social learning context and student aptitude. Medical Teacher, 35(1), 899–907.CrossRefGoogle Scholar
  41. Johnson, T. R., Lyons, R., Kooper, R., Johnsen, K. J., Lok, B. C., & Cendan, J. C. (2014). Virtual patient simulations and optimal social learning context: A replication of an aptitude-treatment interaction effect. Medical Teacher, 36(6), 486–494.CrossRefGoogle Scholar
  42. Keller, J. M. (1987). Development and use of the ARCS model of instructional design. Journal of instructional development, 10(3), 2–10.CrossRefGoogle Scholar
  43. Keller, J. M. (2010). Motivation design for learning and performance: The ARCS model approach. New York: Springer.Google Scholar
  44. Kleinsmith, A., Rivera-Gutierrez, D., Finney, G., Cendan, J. C., & Lok, B. (2015). Understanding empathy training with virtual patients. Computers in Human Behavior, 52, 151–158. Scholar
  45. Kononowicz, A. A., Narracott, A. J., Manini, S., Bayley, M. J., Lawford, P. V., McCormack, K., & Zary, N. (2014). A framework for different levels of integration of computational models into web-based virtual patients. Journal of Medical Internet Research, 16(1), e23.CrossRefGoogle Scholar
  46. Lindsey, L., & Berger, N. (2009). Experiential approach to instruction. In C. Reigeluth & A. Carr-Chellman (Eds.), Instructional-design theories and models: Volume 3. Building a common knowledge based (pp. 117–142). New York, NY: Routledge.Google Scholar
  47. Maran, N. J., & Glavin, R. J. (2003). Low-to high-fidelity simulation – A continuum of medical education? Medical Education, 37(1), 22–28.CrossRefGoogle Scholar
  48. McGaghie, W. C., Issenberg, S. B., Cohen, M. E. R., Barsuk, J. H., & Wayne, D. B. (2011). Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Academic Medicine: Journal of the Association of American Medical Colleges, 86(6), 706.CrossRefGoogle Scholar
  49. McKenney, S. E., & Reeves, T. (2012). Conducting educational design research: What, why and how. London, UK: Taylor & Francis Ltd..Google Scholar
  50. Mezirow, J. (1990). Fostering critical reflection in adulthood: A guide to transformative and emancipatory learning. San Francisco. CA: Jossey-Bass.Google Scholar
  51. Moore, F., & Chalk, C. (2009). The essential neurologic examination: What should medical students be taught? Neurology, 72(23), 2020–2023.CrossRefGoogle Scholar
  52. Posel, N., Mcgee, J. B., & Fleiszer, D. M. (2014). Twelve tips to support the development of clinical reasoning skills using virtual patient cases. Medical Teacher, 37(9):1–6.Google Scholar
  53. Quinn, C. (2009). Computer-based simulations: Principles of engagement. In M. Silberman (Ed.), The handbook of experiential learning (pp. 138–171). San Francisco, CA: Pfeiffer.Google Scholar
  54. Reigeluth, C. M., & Schwartz, E. (1989). An instructional theory for the design of computer-based simulations. Journal of Computer-Based Instruction, 16(1), 1–10.Google Scholar
  55. Reyes, R. J., & Hirumi, A. (2016). Analyzing the pedagogical foundations of virtual patient simulations: A review of literature. Poster presented at the annual graduate research forum at the University of Central Florida, Orlando, FL, April 5.Google Scholar
  56. Rimmele, U. (n.d.). A primer on emotions and learning. Retrieved May 16, 2017, 2017 OECD. from
  57. Rivera-Gutierrez, D., Kleinsmith, A., Johnson, T., Lyons, R., Cendan, J., & Lok, B. (2014). Towards a reflective practicum of embodied conversational agent experiences, IEEE international conference on advanced learning technologies (ICALT). Athens, Greece: IEEE.Google Scholar
  58. Rogers, E. M., & Rogers, E. M. (2003). Diffusion of innovations. New York, NY: Free Press.Google Scholar
  59. Salas, E., & Gregory, M. E. (2011). Simulation-based training: Beyond the bells and whistles! CyberTherapy Magazine, 4, 18–19.Google Scholar
  60. Salem-Schatz, S., Ordin, D., & Mittman, B. (2010). Guide to the after action review (version 1.1). Using evaluation to improve our work: A resource guide. Retrieved March 23, 2015.
  61. Salminen, H., Zary, N., Björklund, K., Toth-Pal, E., & Leanderson, C. (2014). Virtual patients in primary care: Developing a reusable model that fosters reflective practice and clinical reasoning. Journal of Medical Internet Research, 16(1), e3.
  62. Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional-design theories and models: Vol. 2, a new paradigm of instructional theory (pp. 161–181). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  63. Shank, R. C., Berman, T. R., & Macpherson, K. A. (1992). Learning by doing. In C. M. Reigeluth (Ed.), Instructional design theories and models: A new paradigm of instructional theory (pp. 161–179). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
  64. Stapleton, C., & Hirumi, A. (2011). InterPLAY instructional strategy: Learning by engaging interactive entertainment conventions. In M. Shaughnessy & S. Fulgham (Eds.), Pedagogical models: The discipline of online teaching (pp. 183–211). Hauppauge, NY: Nova Science Publishers.Google Scholar
  65. Stapleton, C., & Hirumi, A. (2014). Designing InterPLAY learning landscapes to evoke emotions, spark the imagination, and promote creative problem solving. In A. Hirumi (Ed.), Grounded designs for online and hybrid learning (pp. 159–190). Eugene, WA: International Society for Technology in Education.Google Scholar

Authors and Affiliations

  1. 1.University of Central FloridaOrlandoUSA
  2. 2.Computer and Information Sciences and Engineering DepartmentUniversity of FloridaGainesvilleUSA
  3. 3.Johns Hopkins University School of MedicineBaltimoreUSA
  4. 4.College of EngineeringUniversity of GeorgiaAthensUSA
  5. 5.Microsoft CorporationRedmondUSA
  6. 6.Indiana State UniversityTerre HauteUSA
  7. 7.Ashford UniversitySan DiegoUSA
  8. 8.Simiosys Real World LaboratoryOrlandoUSA
  9. 9.College of MedicineUniversity of Central FloridaOrlandoUSA

Personalised recommendations