Skip to main content
SpringerLink
Log in

Overview

  • Chapter
  • First Online:
Book cover Designing User Studies in Informatics

Part of the book series: Health Informatics ((HI))

  • 883 Accesses

Abstract

This chapter provides an overview of different types of evaluation studies and their advantages and disadvantages. The studies that are reviewed range from naturalistic observations to controlled experiments. They differ in the amount of intervention by researchers and the degree to which the environment is controlled. For each, examples from the literature are included as illustrations.

Since this book focuses on the controlled experiment, the main section of the chapter provides an overview of the essential elements in such controlled experiments: the research hypotheses, the different types of variables that need to be defined, the sampling units which can be people or artifacts in informatics, random assignment and, finally, statistical analyses that allow generalization of a conclusion from a sample to a population. Although the examples in this and subsequent ­chapters are taken from the medical field, the study design methods are generic. As such, the content is applicable to the evaluation of information systems in other domains.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Murphy TE (2009) Requirements form the foundation of software quality. Research report, G00165755

    Google Scholar 

  2. Gonzales CK (2010) Eliciting user requirements using appreciative inquiry. Ph.D., Claremont Graduate University

    Google Scholar 

  3. Baroudi JJ, Olson MH, Ives B (1986) An empirical study of the impact of user involvement on system usage and information satisfaction. Commun ACM 29(3):232–238

    Article  Google Scholar 

  4. Schneider GM, Martin J, Tsai WT (1992) An experimental study of fault detection in user requirements documents. ACM Trans Software Eng Methodol (TOSEM) 1(2):188–204. doi:10.1145/128894.128897

    Article  Google Scholar 

  5. Westland JC (2002) The Cost of Errors in Software Development: Evidence from Industry. The Hournal of Systems and Software, 62, 1–9

    Article  Google Scholar 

  6. Friedman CP, Wyatt JC (2000) Evaluation methods in medical informatics. Springer-Verlag, New York

    Google Scholar 

  7. Brender J (2006) Handbook of evaluation methods for health informatics (trans: Carlander L). Elsevier Inc, San Diego

    Google Scholar 

  8. Kaplan B (2001) Evaluating informatics applications — clinical decision support systems literature review. Int J Med Inform 64:15–37

    Article  PubMed  CAS  Google Scholar 

  9. Tesh EM, Holditch-Davis D (1997) HOME inventory and NCATS: relation to mother and child behaviors during naturalistic observations. Home observation for measurement of the environment. Nursing Child Assessment Teaching Scale. Res Nurs Health 20(4):295–307

    Article  PubMed  CAS  Google Scholar 

  10. Lambrechts G, Noortgate WVD, Eeman L, Maes B (2010) Staff reactions to challenging behaviour: an observation study. Res Dev Disabil 31:525–535

    Article  PubMed  Google Scholar 

  11. Campos B, Graesch AP, Repetti R, Bradbury T, Ochs E (2009) Opportunity for interaction? A naturalistic observation study of dual-earner families after work and school. J Fam Psychol 23(6):798–807

    Article  PubMed  Google Scholar 

  12. Kaplan B (2001) Evaluating informatics applications – some alternative approaches: theory, social interactionism, and call for methodological pluralism. Int J Med Inform 64:39–56

    Article  PubMed  CAS  Google Scholar 

  13. Rosson MB, Carroll JM (2002) Usability engineering: scenario-based development of human-computer interaction. Interactive technologies. Morgan Kaufman Publishers, San Francisco

    Google Scholar 

  14. Purkis ME (1999) Embracing technology: an exploration of the effects of writing nursing. Nurs Inq 6(3):147–156

    Article  PubMed  CAS  Google Scholar 

  15. Stringer ET (1999) Action research. SAGE Publications Inc, Thousand Oaks

    Google Scholar 

  16. Boursnell M, Prosser S (2010) Increasing identification of domestic violence in emergency departments: a collaborative contribution to increasing the quality of practice of emergency nurses. Contemp Nurse 35(12):7

    Google Scholar 

  17. Borg M, Karlsson B, Kim HS (2010) Double helix of research and practice - developing a practice model for crisis resolution and home treatment through participatory action research. Int J Qualitative Stud Health Well-being 5. doi: 10.3402/qhw.v5i1.4647

    Google Scholar 

  18. Butera-Prinzi F, Charles N, Heine K, Rutherford B, Lattin D (2010) Family-to-family link up program: a community-based initiative supporting families caring for someone with an acquired brain injury. NeuroRehabilitation 27:31–47

    PubMed  Google Scholar 

  19. Davison R, Vogel D (2007) Group support systems in Hong Kong: an action research project. In: Galliers RD, Markus ML, Newell S (eds) Exploring information systems research approaches: readings and reflections. Routledge, New York, pp 33–46

    Google Scholar 

  20. Delobelle P, Onya H, Langa C, Mashamba J, Depoorter AM (2010) Advances in health promotion in Africa: promoting health through hospitals. Global Health Promot 17:33–36. doi:10.1177/1757975910363929

    Article  Google Scholar 

  21. Rust J, Golombok S (2009) Modern psychometrics: the science of psychological assessment, 3rd edn. Routledge, London

    Google Scholar 

  22. Magnusson D (1966) Test theory. Addison-Wesley Publishing Company, Reading

    Google Scholar 

  23. Drenth PJD, Sijtsma K (1990) Testtheorie: Inleiding in the Theorie van de Psychologische Test en zijn Toepassingen. Bohn Stafleu Van Loghum, Houten/Antwerpen

    Google Scholar 

  24. de Ayala RJ (2009) The theory and practice of item response theory. Methodology in the social sciences. The Guilford Press, New York

    Google Scholar 

  25. Rushton P, Miller W, Lee KR, Eng J, Yip J (2011) Development and Content Validation of the Wheelchair Use Confidence Scale: A Mixed-Methods Study. Disability and Rehabilitation Assistive Technology, 6 (1), 57–66

    Article  PubMed  Google Scholar 

  26. Fischer B, Nakamura N, Ialomiteanu A, Boak A, Rehm J (2010) Assessing the prevalence of nonmedical prescription opioid use in the general Canadian population: methodological issues and questions. La Revue Canadienne De Psychiatrie 55(9):606–609

    Google Scholar 

  27. Jaussent S, Labarère J, Boyer J, François P (2004) Psychometric characteristics of questionnaires designed to assess the knowledge, perceptions and practices of health care professionals with regards to alcoholic patients [translation: article in French]. Encephale 30(5):437–446

    Article  PubMed  CAS  Google Scholar 

  28. Palmirotta R, Savonarola A, Ludovici G, Donati P, Cavaliere F, De Marchis M, Ferroni P, Guadagni F (2010) Association between Birt Hogg Dube syndrome and cancer predisposition. Anticancer Res 30(3):751–757

    PubMed  CAS  Google Scholar 

  29. Ajzen I (1991) The theory of planned behavior. Organ Behav Hum Decis Process 50:179–211

    Article  Google Scholar 

  30. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 13(3):319–339

    Article  Google Scholar 

  31. Hu PJ-H, Sheng ORL, Chau PY, Tam K-Y, Fung H (1999) Investigation physician acceptance of telemedicine technology: a survey study in Hong Kong. In: 32nd Hawaii International Conference on System Sciences, 1999

    Google Scholar 

  32. Hu PJ-H, Chau PYK, Sheng ORL (2000) Investigation of factors affecting healthcare organization’s adoption of telemedicine technology. In: 33rd Hawaii international conference on system sciences, 2000

    Google Scholar 

  33. Rawstorne P, Jayasuriya R, Caputi P (2000) Issues in predicting and explaining usage behaviors with the technology acceptance model and the theory of planned behavior when usage is mandatory. In: International conference on information systems, Brisbane, Australia, 2000, pp 35–44

    Google Scholar 

  34. Honeth L, Bexelius C, Eriksson M, Sandin S, Litton J, Rosenhall U, Nyrén O, Bagger-Sjöbäck D (2010) An Internet-based hearing test for simple audiometry in nonclinical settings: preliminary validation and proof of principle. Otol Neurotol 31(5):708–714

    Article  PubMed  Google Scholar 

  35. Ives B, Olson MH, Baroudi JJ (1983) The measurement of user information satisfaction. Commun ACM 26(10):785–793

    Article  Google Scholar 

  36. Chinman M, Young AS, Schell T, Hassell J, Mintz J (2004) Computer-assisted self-assessment in persons with severe mental illness. J Clin Psychiatry 65(10):1343–1351

    Article  PubMed  Google Scholar 

  37. Sunaert P, Bastiaens H, Nobels F, Feyen L, Verbeke G, Vermeire E, Maeseneer JD, Willems S, Sutter AD (2010) Effectiveness of the introduction of a chronic care model-based ­program for type 2 diabetes in Belgium. BMC Health Serv Res. doi:10.1186/1472-6963-10-207

    PubMed  Google Scholar 

  38. Sunaert P, Bastiaens H, Feyen L, Snauwaert B, Nobels F, Wens J, Vermeire E, Royen PV, Maeseneer JD, Sutter AD, Willems S (2009) Implementation of a program for type 2 diabetes based on the chronic care model in a hospital-centered health care system: “the Belgian experience”. BMC Health Services Research 9 (152). doi:10.1186/1472-6963-9-152

    Google Scholar 

  39. Herz EJ, Goldberg WA, Reis JS (1984) Family life education for young adolescents: a quasi-experiment. J Youth Adolesc 13(4):309–327

    Article  PubMed  CAS  Google Scholar 

  40. Glynn S, Randolph E, Garrick T, Lui A (2010) A proof of concept trial of an online psychoeducational program for relatives of both veterans and civilians living with schizophrenia. Psychiatr Rehabil J 33(4):278–287

    Article  PubMed  Google Scholar 

  41. Kröncke K (2010) Computer-based learning versus practical course in pre-clinical education: acceptance and knowledge retention. Med Teach 32(5):408–413

    Article  PubMed  Google Scholar 

  42. Kirk RE (1995) Experimental design: procedures for the behavioral sciences, 3rd edn. Brooks/Cole Publishing Company, Pacific Grove

    Google Scholar 

  43. Timm NH (2002) Applied multivariate analysis. Springer, New York

    Google Scholar 

  44. Everitt BS, Pickles A (2004) Statistical aspects of the design and analysis of clinical trials, Revisedth edn. Imperial College Press, London

    Book  Google Scholar 

  45. Ammenwertha E, Gräber S, Herrmann G, Bürkle T, König J (2003) Evaluation of health information systems—problems and challenges. Int J Med Inform 71:125–135

    Article  Google Scholar 

  46. Anderson JR, Schonfeld TL (2009) Data-sharing dilemmas: allowing pharmaceutical company access to research data. IRB: Ethics Hum Res 31(3):17–19

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Systems and Technology, Claremont Graduate University, 130 E. Ninth Street, Claremont, CA, 91711, USA

    Gondy Leroy Ph.D.

Authors
  1. Gondy Leroy Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gondy Leroy Ph.D. .

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag London Limited

About this chapter

Cite this chapter

Leroy, G. (2011). Overview. In: Designing User Studies in Informatics. Health Informatics. Springer, London. https://doi.org/10.1007/978-0-85729-622-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-0-85729-622-1_1

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-0-85729-621-4

  • Online ISBN: 978-0-85729-622-1

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation