Skip to main content
SpringerLink
Log in

Conversion of standard retrospective patient-reported outcomes to momentary versions: cognitive interviewing reveals varying degrees of momentary compatibility

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

The purpose of this study was to adapt different domains of an existing retrospective questionnaire to momentary versions, to use and assess cognitive interviewing for evaluating the new versions, and to compare momentary compatibility (i.e. an item’s potential to be validly converted to a momentary version) across different scales.

Methods

Initial momentary versions of retrospective patient-reported outcomes were produced by converting present perfect tense wording to present tense wording. Cognitive interviews were conducted iteratively with 21 patients to determine which reference period they actually employed, and to identify problems with new, revised versions. A think-aloud interview protocol was supplemented with non-specific concurrent and specific retrospective probing. The momentary compatibility of each item was evaluated by calculating the proportion of interviews wherein momentary reference periods were identified; problems were categorized according to cognitive aspects of survey methodology taxonomy. The efficiency of various cognitive interviewing techniques was determined by evaluating whether applied reference periods were identified by think-aloud alone or by supplementary probes.

Results

The momentary compatibility varied from 5 to 100% across items. Cognitive interviews revealed potential problems of various severities in the majority of items. Think-aloud alone was sufficient at determining the applied reference period in one-third of the cases, and the efficiency of additional concurrent and retrospective probing was 50 and 94%, respectively.

Conclusions

Cognitive interviewing techniques proved useful for developing and evaluating momentary items. Researchers should be aware of the applied reference period and of emerging problems when evaluating adapted momentary items, since not all concepts are suitable. We recommend the proposed method in future adaptations of existing instruments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahmed, S., Berzon, R. A., Revicki, D. A., Lenderking, W. R., Moinpour, C. M., Basch, E., Reeve, B. B., & Wu, A. W., & International Society for Quality of Life, R. (2012). The use of patient-reported outcomes (PRO) within comparative effectiveness research: Implications for clinical practice and health care policy. Medical Care, 50(12), 1060–1070.

    Article  PubMed  Google Scholar 

  2. Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32.

    Article  PubMed  Google Scholar 

  3. Stone, A. A., Shiffman, S., Atienza, A. A., & Nebeling, L. (2007). The science of real-time data capture: Self-reports in health research. Oxford: Oxford University Press.

  4. Christodoulou, C., Junghaenel, D. U., DeWalt, D. A., Rothrock, N., & Stone, A. A. (2008). Cognitive interviewing in the evaluation of fatigue items: Results from the patient-reported outcomes measurement information system (PROMIS). Quality of Life Research, 17(10), 1239–1246.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Watt, T., Rasmussen, A. K., Groenvold, M., Bjorner, J. B., Watt, S. H., Bonnema, S. J., Hegedus, L., & Feldt-Rasmussen, U. (2008). Improving a newly developed patient-reported outcome for thyroid patients, using cognitive interviewing. Quality of Life Research, 17(7), 1009–1017.

    Article  PubMed  Google Scholar 

  6. Watt, T., Bjorner, J. B., Groenvold, M., Rasmussen, A. K., Bonnema, S. J., Hegedus, L., & Feldt-Rasmussen, U. (2009). Establishing construct validity for the thyroid-specific patient reported outcome measure (ThyPRO): An initial examination. Quality of Life Research, 18(4), 483–496.

    Article  PubMed  Google Scholar 

  7. Watt, T., Hegedus, L., Groenvold, M., Bjorner, J. B., Rasmussen, A. K., Bonnema, S. J., & Feldt-Rasmussen, U. (2010). Validity and reliability of the novel thyroid-specific quality of life questionnaire, ThyPRO. European Journal of Endocrinology, 162(1), 161–167.

    Article  CAS  PubMed  Google Scholar 

  8. Watt, T., Cramon, P., Hegedus, L., Bjorner, J. B., Bonnema, S. J., Rasmussen, A. K., Feldt-Rasmussen, U., & Groenvold, M. (2014). The thyroid-related quality of life measure ThyPRO has good responsiveness and ability to detect relevant treatment effects. The Journal of Clinical Endocrinology & Metabolism, 99(10), 3708–3717.

    Article  CAS  Google Scholar 

  9. Watt, T., Groenvold, M., Deng, N., Gandek, B., Feldt-Rasmussen, U., Rasmussen, A. K., Hegedus, L., Bonnema, S. J., & Bjorner, J. B. (2014). Confirmatory factor analysis of the thyroid-related quality of life questionnaire ThyPRO. Health and Quality of Life Outcomes, 12, 126.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Watt, T., Groenvold, M., Hegedus, L., Bonnema, S. J., Rasmussen, A. K., Feldt-Rasmussen, U., & Bjorner, J. B. (2014). Few items in the thyroid-related quality of life instrument ThyPRO exhibited differential item functioning. Quality of Life Research, 23(1), 327–338.

    Article  PubMed  Google Scholar 

  11. Watt, T., Barbesino, G., Bjorner, J. B., Bonnema, S. J., Bukvic, B., Drummond, R., Groenvold, M., Hegedus, L., Kantzer, V., Lasch, K. E., Marcocci, C., Mishra, A., Netea-Maier, R., Ekker, M., Paunovic, I., Quinn, T. J., Rasmussen, A. K., Russell, A., Sabaretnam, M., Smit, J., Torring, O., Zivaljevic, V., & Feldt-Rasmussen, U. (2015). Cross-cultural validity of the thyroid-specific quality-of-life patient-reported outcome measure, ThyPRO. Quality of Life Research, 24(3), 769–780.

    Article  PubMed  Google Scholar 

  12. Cramon, P., Winther, K. H., Watt, T., Bonnema, S. J., Bjorner, J. B., Ekholm, O., Groenvold, M., Hegedus, L., Feldt-Rasmussen, U., & Rasmussen, A. K. (2016). Quality-of-life impairments persist six months after treatment of graves’ hyperthyroidism and toxic nodular goiter: A prospective cohort study. Thyroid, 26(8), 1010–1018.

    Article  CAS  PubMed  Google Scholar 

  13. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks: Sage Publications.

    Book  Google Scholar 

  14. Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  15. Smith, T. J., & Hegedus, L. (2016). Graves’ disease. New England Journal of Medicine, 375(16), 1552–1565.

    Article  PubMed  Google Scholar 

  16. Mullin, P. A., Lohr, K. N., Bresnahan, B. W., & McNulty, P. (2000). Applying cognitive design principles to formatting HRQOL instruments. Quality of Life Research, 9(1), 13–27.

    Article  CAS  PubMed  Google Scholar 

  17. Watt, T. (2017). Development, validation, application and abbreviation of an international thyroid-related quality of life patient-reported outcome measure Unpublished Doctoral Dissertation, University of Copenhagen.

  18. Wilson, I. B., & Cleary, P. D. (1995). Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA, 273(1), 59–65.

    Article  CAS  PubMed  Google Scholar 

  19. Burke, L. (2012). An FDA perspective on clinical trial endpoint measurements. ACTTION-II. U. S. F. a. D. Administration (Ed.).

  20. Kastien-Hilka, T., Rosenkranz, B., Bennett, B., Sinanovic, E., & Schwenkglenks, M. (2016). How to evaluate health-related quality of life and its association with medication adherence in pulmonary tuberculosis—designing a prospective observational study in South Africa. Frontiers in Pharmacology, 7, 125.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Thielsch, C., Andor, T., & Ehring, T. (2015). Do metacognitions and intolerance of uncertainty predict worry in everyday life? An ecological momentary assessment study. Behavior Therapy, 46(4), 532–543.

    Article  PubMed  Google Scholar 

  22. Thielsch, C., Ehring, T., Nestler, S., Wolters, J., Kopei, I., Rist, F., Gerlach, A. L., & Andor, T. (2015). Metacognitions, worry and sleep in everyday life: Studying bidirectional pathways using ecological momentary assessment in GAD patients. Journal of Anxiety Disorders, 33, 53–61.

    Article  PubMed  Google Scholar 

  23. Stone, A. A., Schwartz, J. E., Neale, J. M., Shiffman, S., Marco, C. A., Hickcox, M., Paty, J., Porter, L. S., & Cruise, L. J. (1998). A comparison of coping assessed by ecological momentary assessment and retrospective recall. Journal of Personality and Social Psychology, 74(6), 1670–1680.

    Article  CAS  PubMed  Google Scholar 

  24. Redelmeier, D. A., & Kahneman, D. (1996). Patients’ memories of painful medical treatments: Real-time and retrospective evaluations of two minimally invasive procedures. Pain, 66(1), 3–8.

    Article  CAS  PubMed  Google Scholar 

  25. Ware, J. E. Jr., & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30(6), 473–483.

    Article  PubMed  Google Scholar 

  26. Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., de Haes, J. C., et al. (1993). The European organization for research and treatment of cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. Journal of the National Cancer Institute, 85(5), 365–376.

    Article  CAS  PubMed  Google Scholar 

  27. Conner, T. S., & Barrett, L. F. (2012). Trends in ambulatory self-report: The role of momentary experience in psychosomatic medicine. Psychosomatic Medicine, 74(4), 327–337.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Schwarz, N. (2007). Retrospective and concurrent self-reports: The rationale for real-time data capture. In The science of real-time data capture: Self-reports in health research, (pp. 25 (p. 396). Oxford: Oxford University Press.

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Bo Kehlet Watt (Kehlet IT) for designing and programming the smartphone app. LH is supported by an unrestricted grant from the Novo Nordisk Foundation.

Funding

This study was funded by the Danish Agency for Science, Technology and Innovation (grant 271-09-0143).

Author information

Authors and Affiliations

  1. Department of Endocrinology, Copenhagen University Hospital Rigshospitalet, Copenhagen, Denmark

    Victor Brun Boesen, Åse Krogh Rasmussen, Ulla Feldt-Rasmussen & Torquil Watt

  2. Department of Public Health, University of Copenhagen, Copenhagen, Denmark

    Stine Birk Nissen, Mogens Groenvold & Jakob Bue Bjorner

  3. Department of Palliative Medicine, Bispebjerg Hospital, Copenhagen, Denmark

    Mogens Groenvold

  4. Department of Endocrinology and Metabolism, Odense University Hospital, Odense, Denmark

    Laszlo Hegedüs & Steen Joop Bonnema

Authors
  1. Victor Brun Boesen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stine Birk Nissen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mogens Groenvold
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jakob Bue Bjorner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Laszlo Hegedüs
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Steen Joop Bonnema
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Åse Krogh Rasmussen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ulla Feldt-Rasmussen
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Torquil Watt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Torquil Watt.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

The study was performed in compliance with the Declarations of Helsinki. The study protocol was reviewed by the local Ethical Committee (reg. # H-A-2009-FSP23). According to Danish law, questionnaire studies do not require and thus cannot obtain approval by ethical committees.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boesen, V.B., Nissen, S.B., Groenvold, M. et al. Conversion of standard retrospective patient-reported outcomes to momentary versions: cognitive interviewing reveals varying degrees of momentary compatibility. Qual Life Res 27, 1065–1076 (2018). https://doi.org/10.1007/s11136-017-1762-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-017-1762-7

Keywords

Search

Navigation