Quality of Life Research

, Volume 24, Issue 8, pp 1911–1919 | Cite as

The use of cognitive interviews to revise the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM)

  • Niklas Bobrovitz
  • Maria J. Santana
  • Theresa Kline
  • John Kortbeek
  • Henry T. Stelfox



The “Quality of Trauma Care Patient-Reported Experience Measure” is the first measure of patient experiences with overall injury care. The objective of this study was to use cognitive interviews to inform revision of the measure into a parsimonious set of items that function as intended, in preparation for multicenter testing.


Concurrent and retrospective cognitive interviews with injured patients (n = 17) and family members (n = 13) using semi-structured interview guides. Responses were analyzed using thematic analysis.


Six broad themes were identified and guided revisions: (1) participants did not have the information to answer items (n = 9); (2) items were ambiguous or were inconsistently interpreted (n = 13); (3) items did not measure the intended constructs (n = 6); (4) items included assumptions about healthcare processes (n = 4); (5) items measured non-priority aspects of injury care (n = 8); and (6) items were redundant (n = 5). Two issues resulted in key conceptual and content changes: participants’ difficulty to evaluate pre-hospital, emergency department, and intensive care unit services due to recall issues and the challenge to evaluate the effectiveness and equity of care. In total, 39 items were deleted, 28 new items developed, and the final instrument included 63 items.


Our results informed changes to item content, format, and response options. This study highlights key issues to consider when incorporating patient/family perspectives into quality measurement, most notably, that few participants can assess the quality of care in the pre-hospital and emergency department phases of care and that novel methods are needed to evaluate the effectiveness and equity of care.


Cognitive interviews Measure development Patient experience Injury care 



The project was supported by a Partnership in Health System Improvement Grant (PHE-238551) from the Canadian Institutes of Health Research and Alberta Innovates. Nik Bobrovitz was supported by a Health Quality Council of Alberta Studentship and an Alberta Innovates Health Solutions Studentship. Dr. Stelfox was supported by a New Investigator Award from the Canadian Institutes of Health Research and a Population Health Investigator Award from Alberta Innovates. Funding sources had no role in the design of this study and had no role in the conduct or reporting of this study.

Conflict of interest

We are unaware of any conflicts of interest. None of the authors have financial or professional conflicts of interest that would influence the conduct or reporting of this study.

Supplementary material

11136_2015_919_MOESM1_ESM.docx (25 kb)
Supplementary material 1 (DOCX 25 kb)
11136_2015_919_MOESM2_ESM.docx (43 kb)
Supplementary material 2 (DOCX 43 kb)
11136_2015_919_MOESM3_ESM.pdf (113 kb)
Supplementary material 3 (PDF 112 kb)
11136_2015_919_MOESM4_ESM.pdf (124 kb)
Supplementary material 4 (PDF 123 kb)
11136_2015_919_MOESM5_ESM.pdf (151 kb)
Supplementary material 5 (PDF 150 kb)
11136_2015_919_MOESM6_ESM.pdf (158 kb)
Supplementary material 6 (PDF 157 kb)


  1. 1.
    Stelfox, H. T., Bobranska-Artiuch, B., Nathens, A., & Straus, S. E. (2010). Quality indicators for evaluating trauma care: A scoping review. Archives of Surgery, 145, 286–295.PubMedCrossRefGoogle Scholar
  2. 2.
    Bobrovitz, N., Santana, M. J., Ball, C. G., Kortbeek, J., & Stelfox, H. T. (2012). The development and testing of a survey to measure patient and family experiences with injury care. J Trauma Acute Care Surg, 73, 1332–1339.PubMedCrossRefGoogle Scholar
  3. 3.
    Streiner, D. L., & Norman, G. R. (2008). Health Measurement Scales: A practical guide to their development and use (4th ed.). Oxford, UK: Oxford University Press.CrossRefGoogle Scholar
  4. 4.
    Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.Google Scholar
  5. 5.
    Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research Journal, 12, 229–238.CrossRefGoogle Scholar
  6. 6.
    Harris-Kojetin, L. D., Fowler, F. J, Jr, Brown, J. A., Schnaier, J. A., & Sweeny, S. F. (1999). The use of cognitive testing to develop and evaluate CAHPS 1.0 core survey items. Consumer assessment of health plans study. Medical Care, 37, MS10–MS21.PubMedCrossRefGoogle Scholar
  7. 7.
    Goldstein, E., & Fyock, J. (2001). Reporting of CAHPS quality information to medicare beneficiaries. Health Services Research, 36, 477–488.PubMedCentralPubMedGoogle Scholar
  8. 8.
    Levine, R. E., Fowler, F. J, Jr, & Brown, J. A. (2005). Role of cognitive testing in the development of the CAHPS Hospital Survey. Health Services Research, 40, 2037–2056.PubMedCentralPubMedCrossRefGoogle Scholar
  9. 9.
    Shaw, M. J., Beebe, T. J., Adlis, S. A., & Talley, N. J. (2001). Reliability and validity of the digestive health status instrument in samples of community, primary care, and gastroenterology patients. Alimentary Pharmacology & Therapeutics, 15, 981–987.CrossRefGoogle Scholar
  10. 10.
    Shaw, M. J., Talley, N. J., Beebe, T. J., et al. (2001). Initial validation of a diagnostic questionnaire for gastroesophageal reflux disease. The American journal of gastroenterology, 96, 52–57.PubMedCrossRefGoogle Scholar
  11. 11.
    Grant, E. N., Turner-Roan, K., Daugherty, S. R., et al. (1999). Development of a survey of asthma knowledge, attitudes, and perceptions: The Chicago Community Asthma Survey. Chicago Asthma Surveillance Initiative Project Team. Chest, 116, 178S–183S.PubMedCrossRefGoogle Scholar
  12. 12.
    Committee on Quality of Health Care in America IoM. (2001). Crossing the quality chasm: A new health system for the 21st Century. Washington, DC: National Academy Press.Google Scholar
  13. 13.
    Jha, A. K., Orav, E. J., Zheng, J., & Epstein, A. M. (2008). Patients’ perception of hospital care in the United States. The New England journal of medicine, 359, 1921–1931.PubMedCrossRefGoogle Scholar
  14. 14.
    Stelfox, H. T., Boyd, J. M., Straus, S. E., & Gagliardi, A. R. (2013). Developing a patient and family-centred approach for measuring the quality of injury care: A study protocol. BMC health services research, 13, 31.PubMedCentralPubMedCrossRefGoogle Scholar
  15. 15.
    Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousands Oaks, California: Sage.Google Scholar
  16. 16.
    National Trauma Registry. (2014) http://www.cihi.ca/CIHI-ext-portal/internet/en/document/types+of+care/specialized+services/trauma+and+injuries/ntr_metadata#. Accessed 13 Feb 2014.Google Scholar
  17. 17.
    Housen, P., Shannon, G. R., Simon, B., et al. (2008). What the resident meant to say: Use of cognitive interviewing techniques to develop questionnaires for nursing home residents. The Gerontologist, 48, 158–169.Google Scholar
  18. 18.
    Hales, S., Gagliese, L., Nissim, R., Zimmermann, C., & Rodin, G. (2012). Understanding bereaved caregiver evaluations of the quality of dying and death: An application of cognitive interviewing methodology to the quality of dying and death questionnaire. Journal of Pain and Symptom Management, 43, 195–204.Google Scholar
  19. 19.
    Champion, H. R., Copes, W. S., Sacco, W. J., et al. (1990). The major trauma outcome study: Establishing national norms for trauma care. The Journal of trauma, 30, 1356–1365.Google Scholar
  20. 20.
    Laupland, K. B., Kortbeek, J. B., Findlay, C., & Hameed, S. M. (2005). A population-based assessment of major trauma in a large Canadian region. American Journal of Surgery, 189, 571–575. discussion 6.Google Scholar
  21. 21.
    Henoch, I., Lovgren, M., Wilde-Larsson, B., & Tishelman, C. (2012). Perception of quality of care: Comparison of the views of patients’ with lung cancer and their family members. Journal of Clinical Nursing, 21, 585–594.Google Scholar
  22. 22.
    Larsson, B. W., Larsson, G., & Carlson, S. R. (2004). Advanced home care: Patients’ opinions on quality compared with those of family members. Journal of Clinical Nursing, 13, 226–233.Google Scholar
  23. 23.
    Vedel, I., Ghadi, V., Lapointe, L., Routelous, C., Aegerter, P., & Guirimand, F. (2014). Patients’, family caregivers’, and professionals’ perspectives on quality of palliative care: A qualitative study. Palliative Medicine, 28, 1128–1138.Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Niklas Bobrovitz
    • 1
    • 2
  • Maria J. Santana
    • 2
    • 3
  • Theresa Kline
    • 4
  • John Kortbeek
    • 5
  • Henry T. Stelfox
    • 2
    • 6
    • 7
  1. 1.Nuffield Department of Primary Care Health SciencesUniversity of OxfordOxfordUK
  2. 2.Department of Community Health SciencesUniversity of CalgaryCalgaryCanada
  3. 3.W21C Research and Innovation CentreUniversity of CalgaryCalgaryCanada
  4. 4.Department of PsychologyUniversity of CalgaryCalgaryCanada
  5. 5.Department of SurgeryUniversity of CalgaryCalgaryCanada
  6. 6.Department of Critical Care MedicineUniversity of CalgaryCalgaryCanada
  7. 7.Department of MedicineUniversity of CalgaryCalgaryCanada

Personalised recommendations