Skip to main content
SpringerLink
Log in

Summative Software Evaluation of a Therapeutic Guideline Assistance System for Empiric Antimicrobial Therapy in ICU

  • Published:
Journal of Clinical Monitoring and Computing Aims and scope Submit manuscript

Abstract

Objective

While developing the patient data management system ICUData in close cooperation with the software company (IMESO GmbH, Hüttenberg, Germany), a therapeutic guideline assistance system for empiric antimicrobial therapy in ICU (called “Antibiotic Wizard”) could be introduced and integrated into the existing software. After its introduction into clinical routine, the first version was to be tested, checked for usability and compared to other software products with the help of the IsoMetricss inventory (based on the EN ISO 9241-10 for computer-assisted workflows).

Methods

Half a year after introducing the “Antibiotic Wizard” in the ICUs, 40 physicians from different specialties at different levels of training were surveyed in order to detect deficiencies in the use of the program. The results of these surveys were compared to surveys on the word processing software Word® for Windows® (WinWord®) from Microsoft®, the hospital information system IS-H*MED from SAP® (online and paper surveys) and the administrative program, SAP R/3 HR, also from SAP®.

Results

Reliabilities (Cronbach’s Alpha) of the subscales ranged from satisfactory (α > 0.70) to good (α > 0.80), except for “Controllability” (α = 0.663) and “Error tolerance” (α = 0.693). Medians for individual subscales ranged between 3.04 (“Error tolerance”) and 3.96 (“Suitability for learning”). The “Antibiotic Wizard” showed significantly better results compared to both IS-H*MED and SAP R/3 HR in the subscales of “Suitability for the task”, “Self-descriptiveness” and “Suitability for learning”. In contrast, “Self-descriptiveness” “Controllability” and “Error tolerance” were significantly worse compared to WinWord®.

Conclusions

In generally, the usability of the “Antibiotic Wizard” was deemed good. Some weaknesses were found in the fields of “Error tolerance” and “Controllability”. These problems will be corrected in future versions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Berg M, Langenberg C, vd B I, Kwakkernaat J Considerations for sociotechnical design: experiences with an electronic patient record in a clinical context. Int.J Med.Inform. 1998;52:243–251

    Article  PubMed  CAS  Google Scholar 

  2. Williams LS. Microchips versus stethoscopes: Calgary hospital, MDs face off over controversial computer system. CMAJ 1992;147:1534–1547

    Google Scholar 

  3. Holz auf der Heide B Welche software-ergonomischen evaluationsverfahren können was leisten? In: Rödiger KH (ed) Software-ergonomie ‚93, Von der benutzeroberfläche zur arbeitsgestaltung. Teubner, Stuttgart; 1993:157–172

    Google Scholar 

  4. Oppermann R, Reiterer H Software-ergonomische evaluation. In: Eberleh E, Oberquelle H, Oppermann R (eds) Einführung in die software-ergonomie. Walter de Gruyter, Berlin; 1994:335–371

    Google Scholar 

  5. Dzida W. Software-ergonomische Qualitätsprüfung - Utopie oder realistisches Ziel? In: Cakir A, Cakir G, eds., Was bringen die europäischen Regelwerke für Bildschirmarbeitsplätze? Berlin: 1991:61–76

  6. Eason KD. Towards experimental study of usability. Behav Inf Technol 1984;3:133–143

    Article  Google Scholar 

  7. Rauterberg M. Lässt sich die gebrauchstauglichkeit interaktiver software messen? Und wenn ja, wie? Ergonomie and Informatik. 1992;16:3–18

    Google Scholar 

  8. Michel A, Benson M, Junger A, Sciuk G, Hempelmann G, Dudeck J, Marquardt K. Design principles of a clinical information system for intensive care units (ICUData). Stud Health TechnolInf 2000;77:921–924

    CAS  Google Scholar 

  9. Gediga G, Hamborg KC, Düntsch I. The IsoMetrics usability inventory: an operationalization of ISO 9241-10 supporting summative and formative evaluation of software systems. Behav Inf Technol 1999;18:151–164

    Article  Google Scholar 

  10. Hamborg KC, Vehse B, Bludau HB. Questionnaire based usability evaluation of hospital information systems. Elect J Inf Syst Eval 2004;7:21–30

    Google Scholar 

  11. Gruber C. Arbeitseffizienz im Büro. Psychische Einflüsse auf SAP R/3–unterstützte Arbeitsprozesse. Dissertation, Psychologisches Institut., Julius-Maximilians-Universität Würzburg, 2000:59–61

  12. Vogel F, Bodmann K. Empfehlungen zur kalkulierten parenteralen Initialtherapie bakterieller erkrankungen bei erwachsenen. Chemotherapie J 2004;13:46–105.

    Google Scholar 

Download references

Acknowledgments

We would like to thank MoReData GmbH in Giessen, Germany for their help in statistical analysis.

Financial support for this study was provided in part by a grant from IMESO GmbH, Hüttenberg, Germany. The founding agreement ensured the authors’ independence in designing the study, interpreting the data, writing and publishing the report.

Author information

Authors and Affiliations

  1. Department of Anesthesiology, Intensive Care Medicine, and Pain Therapy, University Hospital Giessen and Marburg GmbH, Campus Giessen, Germany Rudolf-Buchheim-Straße 7, 35392, Giessen, Germany

    Rainer Röhrig Dr. med., Hannah Beuteführ cand. med., Bernd Hartmann PD Dr. med., Eileen Niczko cand. med., Axel Junger Prof. Dr. med. & Gunter Hempelmann Prof. Dr. med. Dr. h.c.

  2. Department of Medical Psychology, University Hospital Giessen and Marburg GmbH, Campus Giessen, Giessen, Germany

    Birgit Quinzio Dr. hum. biol.

Authors
  1. Rainer Röhrig Dr. med.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hannah Beuteführ cand. med.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bernd Hartmann PD Dr. med.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Eileen Niczko cand. med.
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Birgit Quinzio Dr. hum. biol.
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Axel Junger Prof. Dr. med.
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Gunter Hempelmann Prof. Dr. med. Dr. h.c.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bernd Hartmann PD Dr. med..

Additional information

Röhrig R, Beuteführ H, Hartmann B, Niczko E, Quinzio B, Junger A, Hempelmann G. Summative software evaluation of a therapeutic guideline assistance system for empiric antimicrobial therapy in ICU.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Röhrig, R., Beuteführ, H., Hartmann, B. et al. Summative Software Evaluation of a Therapeutic Guideline Assistance System for Empiric Antimicrobial Therapy in ICU. J Clin Monit Comput 21, 203–210 (2007). https://doi.org/10.1007/s10877-007-9073-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10877-007-9073-0

Keywords

Search

Navigation