Skip to main content
SpringerLink
Log in

The Role of Usability Engineering in the Development of an Intelligent Decision Support System

  • Conference paper
  • First Online:
Artificial Intelligence in Health (AIH 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11326))

Included in the following conference series:

Abstract

This paper presents an overview of the usability engineering process for the development of a personalised clinical decision support system for the management of type 1 diabetes. The tool uses artificial intelligence (AI) techniques to provide insulin bolus dose advice and carbohydrate recommendations that adapt to the individual. We describe the role of human factors and user-centred design in the creation of medical systems that must adhere to international standards. We focus specifically on the formative evaluation stage of this process. The preliminary analysis of data shows promising results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aamodt, A., Plaza, E.: Case-based reasoning: foundational issues, methodological variations, and system approaches. AI Commun. 7(1), 39–59 (1994)

    Google Scholar 

  2. Abrial, J.R., Abrial, J.R.: The B-book: Assigning Programs to Meanings. Cambridge University Press, Cambridge (2005)

    MATH  Google Scholar 

  3. Årsand, E., et al.: Mobile health applications to assist patients with diabetes: lessons learned and design implications. J. Diab. Sci. Technol. 6(5), 1197–1206 (2012)

    Article  Google Scholar 

  4. Bellazzi, R.: Telemedicine and diabetes management: current challenges and future research directions. J. Diab. Sci. Technol. 2(1), 98–104 (2008)

    Article  Google Scholar 

  5. Bertini, E., Catarci, T., Dix, A., Gabrielli, S., Kimani, S., Santucci, G.: Appropriating heuristic evaluation methods for mobile computing. In: Handbook of Research on User Interface Design and Evaluation for Mobile Technology, pp. 780–801. IGI Global (2008)

    Google Scholar 

  6. Brooke, J., et al.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  7. Brown, D.: Reflections on the role and design of visualizations to support patients with type 1 diabetes (2016, in preparation)

    Google Scholar 

  8. Cefai, J.J., Shapley, J.L.: Micro-valve. US Patent 8,048,041, 1 November 2011

    Google Scholar 

  9. Cellnovo: Cellnovo system (2007). https://www.cellnovo.com/en/homepage

  10. Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 13, 319–340 (1989)

    Article  Google Scholar 

  11. Tufte, E.: The Visual Display of Quantitative Information. Graphics Press, Cheshire (2001). vol. 4, no. 5, p. 6

    Google Scholar 

  12. Fast, E., Horvitz, E.: Long-term trends in the public perception of artificial intelligence. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 963–969. International Joint Conferences on Artificial Intelligence, Menlo Park, Calif (2017)

    Google Scholar 

  13. Georgsson, M., Staggers, N., Weir, C.: A modified user-oriented heuristic evaluation of a mobile health system for diabetes self-management support. Comput. Inf. Nurs. 34(2), 77 (2016)

    Article  Google Scholar 

  14. GOV.UK: Medicines and healthcare products regulatory agency. Alerts and recalls for drugs and medical devices (2016). https://www.gov.uk/drug-device-alerts

  15. Hengstler, M., Enkel, E., Duelli, S.: Applied artificial intelligence and trust–the case of autonomous vehicles and medical assistance devices. Technol. Forecast. Soc. Chang. 105, 105–120 (2016)

    Article  Google Scholar 

  16. Herrero, P., López, B., Martin, C.: PEPPER: patient empowerment through predictive personalised decision support. In: ECAI Workshop on Artificial Intelligence for Diabetes, pp. 8–9 (2016)

    Google Scholar 

  17. Herrero, P., et al.: Safety layer for an insulin delivery system. In: Abstracts from ATTD 2017 10th International Conference on Advanced Technologies & Treatments for Diabetes Paris, France, 15-18 February 2017, p. A-62 (2017). https://doi.org/10.1089/dia.2017.2525

  18. IEC: International electrotechnical commission (2018). http://www.electropedia.org/

  19. International Standard IEC: 62366-1: 2015 medical devices part 1: application of usability engineering to medical devices. International Organization for Standardization, Geneva (2015)

    Google Scholar 

  20. International Standard IEC: 62366-2: 2016 medical devices part 2: Guidance on the application of usability engineering to medical devices. International Organization for Standardization, Geneva (2016)

    Google Scholar 

  21. ISMP: Institute for safe medication practices, fluorouracil incident root cause analysis report (2007). https://www.ismp-canada.org

  22. ISO: International organization for standardization online browsing platform (2018). https://www.iso.org/obp/ui/

  23. Kirwan, B., Ainsworth, L.K.: A Guide to Task Analysis: The Task Analysis Working Group. CRC Press, Boca Raton (1992)

    Book  Google Scholar 

  24. Klonof, D.C.: The current status of bolus calculator decision-support software. J. Diab. Sci. Technol. 6(5), 990–994 (2012)

    Article  Google Scholar 

  25. Klonoff, D.C.: Telemedicine for diabetes: current and future trends. J. Diab. Sci. Technol. 10(1), 3–5 (2016)

    Article  Google Scholar 

  26. Munzner, T.: A nested process model for visualization design and validation. IEEE Trans. Vis. Comput. Graph. 6, 921–928 (2009)

    Article  Google Scholar 

  27. Nielsen, J.: Usability Engineering. Morgan Kauffmann, San Francisco (1994)

    MATH  Google Scholar 

  28. Nielsen, J.: Severity ratings for usability problems. Pap. Essays 54, 1–2 (1995)

    Google Scholar 

  29. O’Kane, A.: Individual differences and contextual factors influence the experience and practice of self-care with type 1 diabetes technologies. Ph.D. thesis, UCL (University College London) (2016)

    Google Scholar 

  30. Rice, A.D., Lartigue, J.W.: Touch-level model (TLM): evolving KLM-GOMS for touchscreen and mobile devices. In: Proceedings of the 2014 ACM Southeast Regional Conference, p. 53. ACM (2014)

    Google Scholar 

  31. Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. AI Mag. 36(4), 105–114 (2015)

    Article  Google Scholar 

  32. Thimbleby, H.: Feature ignorance of interaction programming is killing people. Interactions 15(5), 52–57 (2008)

    Article  Google Scholar 

  33. USFDA: MAUDE - manufacturer and user facility device experience (2018). http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfMAUDE/TextSearch.cfm

  34. USFDA, et al.: Applying human factors and usability engineering to optimize medical device design. FDA Center for Devices and Radiological Health: Silver Spring, MD, USA (2011)

    Google Scholar 

  35. Zhang, J., Johnson, T.R., Patel, V.L., Paige, D.L., Kubose, T.: Using usability heuristics to evaluate patient safety of medical devices. J. Biomed. Inform. 36(1–2), 23–30 (2003)

    Article  Google Scholar 

Download references

Acknowledgement

This work has received funding from the EU Horizon 2020 research and innovation programme under grant agreement No. 689810. We thank all partners of the PEPPER consortium, in particular clinical teams in UK and Spain who conducted the Phase 2 of the formative usability evaluation. Ethical approval has been obtained from the relevant authorities for all elements involving users.

Author information

Authors and Affiliations

  1. School of Engineering, Computing and Mathematics, Oxford Brookes University, Oxford, UK

    Clare Martin, Arantza Aldea, David Duce, Rachel Harrison & Bedour Alshaigy

Authors
  1. Clare Martin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arantza Aldea
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Duce
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rachel Harrison
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bedour Alshaigy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Clare Martin or Bedour Alshaigy .

Editor information

Editors and Affiliations

  1. University of Melbourne, Melbourne, VIC, Australia

    Fernando Koch

  2. IIIA-CSIC and Autonomous University of Barcelona, Barcelona, Spain

    Andrew Koster

  3. Universitat Rovira i Virgili, Tarragona, Spain

    David Riaño

  4. Università di Bologna, Cesena, Italy

    Sara Montagna

  5. HES-SO Valais Wallis, Sierre, Switzerland

    Michael Schumacher

  6. VU University Amsterdam, Amsterdam, The Netherlands

    Annette ten Teije

  7. Karolinska Institute, Stockholm, Sweden

    Christian Guttmann

  8. University of Ulm, Ulm, Germany

    Manfred Reichert

  9. State University of New York, Oswego, NY, USA

    Isabelle Bichindaritz

  10. Imperial College London, London, UK

    Pau Herrero

  11. University of Erlangen and Nuremberg, Erlangen, Germany

    Richard Lenz

  12. University of Girona, Girona, Spain

    Beatriz López

  13. Ohio University, Athens, OH, USA

    Cindy Marling

  14. Oxford Brookes University, Oxford, UK

    Clare Martin

  15. Università del Piemonte Orientale, Alessandria, Italy

    Stefania Montani

  16. School of Computing Science and Digital Media, Robert Gordon University, Aberdeen, UK

    Nirmalie Wiratunga

Appendix A

Appendix A

The following section of this report document the notable and agreed issues and recommendations against each heuristic during a debrief with the evaluators. In some cases, issues listed by the evaluator for a particular heuristic have been relocated to a more suitable heuristic following the debrief.

  • Z6 - Informative Feedback

  • Average score: 0.33

  • Standard deviation: 0.29

  • Result: No usability issues

  • Issue: Activity monitor connection is not informative.

  • Recommendation: Provide feedback and information on the connection status of the activity monitor.

  • B1 - Visibility of System Status and Losability/Findability of the Device

  • Average score: 1.83

  • Standard deviation: 0.76

  • Result: Minor usability issue

  • Issue: Network, activity monitor and CGM status are not clearly indicated.

  • Recommendation: Include clear indications of connection status for network, activity monitor and CGM on the status bar.

  • B2 - Match Between System and Real World

  • Average score: 1.22

  • Standard deviation: 0.51

  • Result: Cosmetic problem only

  • B2.1 Issue: Ambient temperature location on the Get Bolus Advice interface is not intuitive.

  • B2.1 Recommendation: Relocate ambient temperature to the other category.

  • B2.2 Issue: The location of obtaining a capillary blood glucose reading is not intuitive.

  • B2.2 Recommendation: Include the ability to obtain a capillary blood glucose reading from the main menu and calibration interfaces.

  • B2.3 Issue: The application interface is always in portrait orientation.

  • B2.3 Recommendation: Appropriate landscape interfaces should be included. For example, changing the visualizations on the dashboard when in landscape orientation.

  • B2.4 Issue: The Android back button (bar at the bottom) interfaces with the interface due to full screen mode.

  • B2.4 Recommendation: Change PEPPER to a non-full screen application for MDI users on the handset or ensure that this bar does not interfere with the PEPPER interface.

  • B3 - Consistency and Mapping

  • Average score: 0.33

  • Standard deviation: 0.58

  • Result: No usability issues

  • Issue: The target blood glucose thresholds do not match what actually happens, stating that the lowest/highest possible value is invalid.

  • Recommendation: Investigate and correct the target blood glucose thresholds to ensure the valid range displayed to the user is correct or that valid inputs are accepted.

  • B4 - Good Ergonomics and Minimalist Design

  • Average score: 0.33

  • Standard deviation: 0.29

  • Result: No usability issues

  • Issue: The application has a mixture of light and dark interfaces.

  • Recommendation: Use a consistent colour palette throughout the interfaces.

  • B5 - Ease of Input, Screen Readability and Glanceability

  • Average score: 0.53

  • Standard deviation: 0.23

  • Result: Cosmetic problem only

  • B5.1 Issue: The abbreviation IU needs clarifying.

  • B5.1 Recommendation: Change the abbreviation IU to Units (based on clinician advice).

  • B5.2 Issue: The menu can only be accessed from the dashboard interface.

  • B5.2 Recommendation: Replace the home button with the menu option on all interfaces.

  • B5.3 Issue: The CGM cannot be automatically calibrated from the capillary blood glucose meter.

  • B5.3 Recommendation: Add a calibration option which obtains the blood glucose reading from the capillary blood glucose meter and automatically calibrates the CGM.

  • B6 - Flexibility, Efficiency of Use and Personalisation

  • Average score: 1.22

  • Standard deviation: 0.51

  • Result: Cosmetic problem only

  • B6.1 Issue: No ability to customise the application.

  • B6.1 Recommendation: Add some degree of customisation. For example, quick links and custom colour palettes.

  • B6.2 Issue: The keyboard overlaps input fields on the Get Bolus Advice interface.

  • B6.2 Recommendation: Remove the next option from the keyboard when inputting on the Get Bolus Advice interface, instead include the done button to close the keyboard.

  • B6.3 Issue: Some interfaces use number pickers rather than a keyboard for data entry. For consistency and speed these should all be keyboard inputs.

  • B6.3 Recommendation: Replace number pickers with appropriate keyboards, limiting the characters/digits to only valid inputs.

  • B7 - Aesthetic, Privacy and Social Conventions

  • Average score: 1.22

  • Standard deviation: 0.51

  • Result: Cosmetic problem only

  • B7.1 Issue: There are some harsh edges on the interface.

  • B7.1 Recommendation: Use the Android material design features to soften the edges, for example z-index.

  • B7.2 Issue: The applications lock screen does not provide security in its present state.

  • B7.2 Recommendation: The lock screen is not needed as the phone has its own lock screen. On the CSII version, the tap the Xs should be replaced by a personalised PIN to prevent unauthorized use.

  • B7.3 Issue: The application does not indicate that data has been transmitted successfully.

  • B7.3 Recommendation: Include an interface of data sent to the PEPPER server application, perhaps on the Events interface.

  • B8 - Realistic Error Management

  • Average score: 1.83

  • Standard deviation: 0.76

  • Result: Minor usability issues

  • Issue: The application does not provide any undo functionality for adding boluses/meals.

  • Recommendation: Include the ability to undo the previous bolus/meal input.

  • D1 - Safe and Efficient Numerical Data Entry

  • Average score: 1.75

  • Standard deviation: 0.25

  • Result: Minor usability issues

  • D1.1 Issue: Not all inputs are validated to ensure safe data entry.

  • D1.1 Recommendation: Clinically approved thresholds should be introduced for all inputs. For example, carbohydrates on the Get Bolus Advice interface.

  • D1.2 Issue: There is no ability to obtain a blood glucose reading from the capillary blood glucose meter on the Get Bolus Advice interface in the event of CGM failure.

  • D1.2 Recommendation: Add the option to obtain a capillary blood glucose reading via Bluetooth on the Get Bolus Advice interface. Performing this task should be mandatory if the application has not received data from the CGM recently.

  • D2 - Clear and Meaningful Numerical Data Visualization

  • Average score: 0.33

  • Standard deviation: 0.33

  • Result: No usability issues

  • D2.1 Issue: Some of the text is small and difficult to read.

  • D2.1 Recommendation: Introduce a minimum text size to ensure the information is clear. Additionally display more information when clicking on the carbohydrate/bolus/activity on the dashboard with a large text size and precise underlying data.

  • D2.2 Issue: Bolus and carbohydrate values on the dashboard can overlap if input within a small time interval.

  • D2.2 Recommendation: Merge overlapping data on the dashboard visualization (carbs and bolus) and indicate that this is multiple inputs and provide the user the ability to tap to view details.

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Martin, C., Aldea, A., Duce, D., Harrison, R., Alshaigy, B. (2019). The Role of Usability Engineering in the Development of an Intelligent Decision Support System. In: Koch, F., et al. Artificial Intelligence in Health. AIH 2018. Lecture Notes in Computer Science(), vol 11326. Springer, Cham. https://doi.org/10.1007/978-3-030-12738-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-12738-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-12737-4

  • Online ISBN: 978-3-030-12738-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation