Quality of Life Research

, Volume 19, Issue 4, pp 483–487 | Cite as

Applying cognitive debriefing to pre-test patient-reported outcomes in older people with multiple sclerosis

  • Michelle Ploughman
  • Mark Austin
  • Mark Stefanelli
  • Marshall Godwin
Brief Communication



The aim of this study was to reduce respondent burden and decrease data errors in patient-reported outcomes (PROs) intended for a postal survey measuring health and lifestyle factors that may affect quality of life in older people with multiple sclerosis (MS).


Participants (n = 18) were recruited from a database of outpatient visits. Using the qualitative diagnostic method, cognitive debriefing, participants completed five standardized questionnaires; Frenchay Activities Index, Barthel Index, Simple Lifestyle Indicator Questionnaire, EuroQoL EQ-5D and Personal Resources Questionnaire 2000. PRO item issues and respondent behaviors such as skipping items were recorded. Data collection was an iterative process whereby difficulties experienced by 2–3 subjects were used to modify the survey for following respondents until data saturation was reached.


Most respondents had serious difficulties with at least one PRO item. Response errors fell into three main categories: (1) respondents did not read instructions and completed the item incorrectly, (2) respondents did not understand the question and required examples to clarify and (3) respondents felt that the pre-determined response options did not apply to them. PRO reformatting, minor modifications to item wording and addition of item examples improved precision and reduced respondent burden.


Our findings support the notion that methods such as cognitive debriefing help improve precision of self-reported measures in a special population such as ours.


Cognitive interviews Outcome measures Survey methods Chronic illness Disability 



Barthel Index


Frenchay Activities Index


Multiple Sclerosis


Personal Resources Questionnaire version 2000


Patient-Reported Outcomes


Quality of Life


Simple Lifestyle Indicator Questionnaire



Research support was provided by the Eastern Health Authority, the Memorial University Dr. W. Ingram Award and Faculty of Medicine Neurology Research Fund.

Supplementary material

11136_2010_9602_MOESM1_ESM.doc (42 kb)
Supplementary material 1 (DOC 43 kb)


  1. 1.
    Leidy, N. K., & Vernon, M. (2008). Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370.CrossRefPubMedGoogle Scholar
  2. 2.
    Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., et al. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO good research practices task force report. Value in Health, 12(4), 419–429.CrossRefPubMedGoogle Scholar
  3. 3.
    United States Federal Drug Administration Agency. (2006). Guideline for Industry. Patient reported outcomes measures: Use in medical product development to support labelling claims. Federal Register.Google Scholar
  4. 4.
    Mullin, P. A., Lohr, K. N., Bresnahan, B. W., & McNulty, P. (2000). Applying cognitive design principles to formatting HRQOL instruments. Quality of Life Research, 9(1), 13–27.CrossRefPubMedGoogle Scholar
  5. 5.
    Dillman, D. A., Sinclair, M. D., & Clark, J. R. (1993). Effect of questionnaire length, respondent-friendly design, and a difficult question on response rates for occupant-addressed census mail surveys. Public Opinion Quarterly, 57, 289–304.CrossRefGoogle Scholar
  6. 6.
    O’Sullivan, S. B. (2007). Multiple sclerosis. In S. B. O’Sullivan & T. J. Schmitz (Eds.), Physical rehabilitation. Assessment and treatment. Philadelphia: F.A. Davis Company.Google Scholar
  7. 7.
    Mallinson, S. (2002). Listening to respondents: A qualitative assessment of the Short-Form 36 Health Status Questionnaire. Social Science and Medicine, 54(1), 11–21.CrossRefPubMedGoogle Scholar
  8. 8.
    Hak, T., Willems, D., van der Wal, G., & Visser, F. (2004). A qualitative validation of the Minnesota living with heart failure questionnaire. Quality of Life Research, 13(2), 417–426.CrossRefPubMedGoogle Scholar
  9. 9.
    Fisk, J. D., Brown, M. G., Sketris, I. S., Metz, L. M., Murray, T. J., & Stadnyk, K. J. (2005). A comparison of health utility measures for the evaluation of multiple sclerosis treatments. Journal of Neurology, Neurosurgery and Psychiatry, 76(1), 58–63.CrossRefGoogle Scholar
  10. 10.
    Clarke, H. H., & Schober, M. F. (1992). Asking questions and influencing answers. In J. M. Tanur (Ed.), Questions about questions Inquiries into the cognitive bases of surveys. New York: Russell Sage Foundation.Google Scholar
  11. 11.
    Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238.CrossRefPubMedGoogle Scholar
  12. 12.
    McColl, E., Meadows, K., & Barofsky, I. (2003). Cognitive aspects of survey methodology and quality of life assessment. Quality of Life Research, 12(3), 217–218.CrossRefPubMedGoogle Scholar
  13. 13.
    Turnbull, J. C., Kersten, P., Habib, M., McLellan, L., Mulee, M. A., & George, S. (2000). Validation of the Frenchay Activities Index in a general population aged 16 years and older. Archives of Physical Medicine and Rehabilitation, 81(8), 1034–1038.CrossRefPubMedGoogle Scholar
  14. 14.
    Yeo, D., Faleiro, R., & Lincoln, N. B. (1995). Barthel ADL index: A comparison of administration methods. Clinical Rehabilitation, 9(1), 34–39.CrossRefGoogle Scholar
  15. 15.
    Godwin, M., Streight, S., Dyachuk, E., van den Hooven, E. C., Ploemacher, J., Seguin, R., et al. (2008). Testing the Simple Lifestyle Indicator Questionnaire: Initial psychometric study. Canadian Family Physician, 54(1), 76–77.PubMedGoogle Scholar
  16. 16.
    Brazier, J., Jones, N., & Kind, P. (1993). Testing the validity of the Euroqol and comparing it with the SF-36 health survey questionnaire. Quality of Life Research, 2(3), 169–180.CrossRefPubMedGoogle Scholar
  17. 17.
    Weinert, C., & Brandt, P. A. (1987). Measuring social support with the Personal Resource Questionnaire. Western Journal of Nursing Research, 9(4), 589–602.CrossRefPubMedGoogle Scholar
  18. 18.
    Korner-Bitensky, N., Wood-Dauphine, S., Siemiatyki, J., Shapire, S., & Becker, R. (1994). Health-related information postdischarge: Telephone versus face-to- face interviewing. Archives of Physical Medicine and Rehabilitation, 75(12), 1287–1296.PubMedGoogle Scholar
  19. 19.
    Groenvold, M., Klee, M. C., Sprangers, M. A., & Aaronson, N. K. (1997). Validation of the EORTC QLQ-C30 quality of life questionnaire through combined qualitative and quantitative assessment of patient-observer agreement. Journal of Clinical Epidemiology, 50(4), 441–450.CrossRefPubMedGoogle Scholar
  20. 20.
    Turner, R. R., Quittner, A. L., Parasuraman, B. M., Kallich, J. D., Cleeland, C. S. FDA Patient-Reported Outcomes Consensus Meeting Group. (2007). Patient-reported outcomes: Instrument development and selection issues. Value in Health, 10(SUPPL. 2), S86–S93.CrossRefPubMedGoogle Scholar
  21. 21.
    Ojanen, V., & Gogates, G. (2006). A briefing on cognitive debriefing. Good Clinical Practice Journal, 13(12), 25–29.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Michelle Ploughman
    • 1
    • 2
  • Mark Austin
    • 1
  • Mark Stefanelli
    • 3
  • Marshall Godwin
    • 2
  1. 1.Clinical Research, Rehabilitation Program, Eastern Health AuthorityL.A. Miller CentreSt. John’sCanada
  2. 2.Primary Healthcare Research Unit, Faculty of MedicineMemorial UniversitySt. John’sCanada
  3. 3.Department of Neurology, Faculty of MedicineMemorial UniversitySt. John’sCanada

Personalised recommendations