Quality of Life Research

, Volume 21, Issue 7, pp 1159–1164 | Cite as

Reliability of adverse symptom event reporting by clinicians

  • Thomas M. Atkinson
  • Yuelin Li
  • Charles W. Coffey
  • Laura Sit
  • Mary Shaw
  • Dawn Lavene
  • Antonia V. Bennett
  • Mike Fruscione
  • Lauren Rogak
  • Jennifer Hay
  • Mithat Gönen
  • Deborah Schrag
  • Ethan Basch



Adverse symptom event reporting is vital as part of clinical trials and drug labeling to ensure patient safety and inform risk–benefit decision making. The purpose of this study was to assess the reliability of adverse event reporting of different clinicians for the same patient for the same visit.


A retrospective reliability analysis was completed for a sample of 393 cancer patients (42.8% men; age 26–91, M = 62.39) from lung (n = 134), prostate (n = 113), and Ob/Gyn (n = 146) clinics. These patients were each seen by two clinicians who independently rated seven Common Terminology Criteria for Adverse Events (CTCAE) symptoms. Twenty-three percent of patients were enrolled in therapeutic clinical trials.


The average time between rater evaluations was 68 min. Intraclass correlation coefficients were moderate for constipation (0.50), diarrhea (0.58), dyspnea (0.69), fatigue (0.50), nausea (0.52), neuropathy (0.71), and vomiting (0.46). These values demonstrated stability over follow-up visits. Two-point differences, which would likely affect treatment decisions, were most frequently seen among symptomatic patients for constipation (18%), vomiting (15%), and nausea (8%).


Agreement between different clinicians when reporting adverse symptom events is moderate at best. Modification of approaches to adverse symptom reporting, such as patient self-reporting, should be considered.


Drug toxicity Reproducibility of results Risk assessment Statistical data interpretation 



Common Terminology Criteria for Adverse Events


Intraclass correlation coefficient(s)


Medical dictionary of regulatory activities


Medical record number(s)


National Cancer Institute


Patient-reported outcome(s)


Patient-reported outcomes version of the Common Terminology Criteria for Adverse Events



This project was supported by a National Institutes of Health Research Training Grant (T32 CA009461-25); a National Institutes of Health Support Grant (P30-CA-008748). The findings in this manuscript were partially reported at the 31st Annual Meeting and Scientific Sessions of the Society of Behavioral Medicine, Seattle, WA, April 7–10, 2010.


  1. 1.
    NCI: National Cancer Institute. (2001). Cancer therapy evaluation program. NCI guidelines—Expedited adverse event reporting requirements for NCI investigational agents. Bethesda: National Cancer Institute.Google Scholar
  2. 2.
    Basch, E., Iasonos, A., Barz, A., et al. (2007). Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. Journal of Clinical Oncology, 25, 5374–5380.PubMedCrossRefGoogle Scholar
  3. 3.
    Basch, E. (2010). The missing voice of patients in drug-safety reporting. New England Journal of Medicine, 362, 865–869.PubMedCrossRefGoogle Scholar
  4. 4.
    Basch, E., Jia, X., Heller, G., et al. (2009). Adverse symptom event reporting by patients vs clinicians: Relationships with clinical outcomes. Journal of the National Cancer Institute, 101, 1624–1632.PubMedCrossRefGoogle Scholar
  5. 5.
    Belknap, S. M., Georgopoulos, C. H., West, D. P., et al. (2010). Quality of methods for assessing and reporting serious adverse events in clinical trials of cancer drugs. Clinical Pharmacology and Therapeutics, 88, 231–236.PubMedCrossRefGoogle Scholar
  6. 6.
    Ahmad, S. R. (2003). Adverse drug event monitoring at the Food and Drug Administration. Journal of General Internal Medicine, 18, 57–60.PubMedCrossRefGoogle Scholar
  7. 7.
    Trotti, A., Colevas, A. D., Setser, A., et al. (2007). Patient-reported outcomes and the evolution of adverse event reporting in oncology. Journal of Clinical Oncology, 25, 5121–5127.PubMedCrossRefGoogle Scholar
  8. 8.
    Trotti, A., Colevas, A. D., Setser, A., et al. (2003). CTCAE v3.0: Development of a comprehensive grading system for the adverse events of cancer treatment. Seminars in Radiation Oncology, 13, 176–181.PubMedCrossRefGoogle Scholar
  9. 9.
    Basch, E., Iasonos, A., McDonough, T., et al. (2006). Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: Results of a questionnaire-based study. The Lancet Oncology, 7, 903–909.PubMedCrossRefGoogle Scholar
  10. 10.
    Bruner, D. W., Bryan, C. J., Aaronson, N., et al. (2007). Issues and challenges with integrating patient-reported outcomes in clinical trials supported by the National Cancer Institute-sponsored clinical trials networks. Journal of Clinical Oncology, 25, 5051–5057.PubMedCrossRefGoogle Scholar
  11. 11.
    US Food and Drug Administration. (2009). Guidance for industry. Patient-reported outcome measures: Use in medical development to support labeling claims. Available from Accessed 9 Dec 2010
  12. 12.
    Uebersax, J. S. (1987). Diversity of decision-making models and the measurement of interrater agreement. Psychological Bulletin, 101, 140–146.CrossRefGoogle Scholar
  13. 13.
    Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.PubMedCrossRefGoogle Scholar
  14. 14.
    Rosner, B. (2005). Fundamentals of biostatistics. Belmont, CA: Duxbury Press.Google Scholar
  15. 15.
    McGill, R., Tukey, J. W., & Larsen, W. A. (1978). Variation of box plots. The American Statistician, 32, 12–16.Google Scholar
  16. 16.
    Zegers, M., de Bruijne, M. C., Wagner, C., et al. (2010). The inter-rater agreement of retrospective assessments of adverse events does not improve with two reviewers per patient record. Journal of Clinical Epidemiology, 63, 94–102.PubMedCrossRefGoogle Scholar
  17. 17.
    Weingart, S. N., Gandhi, T. K., Seger, A. C., et al. (2005). Patient-reported medication symptoms in primary care. Archives of Internal Medicine, 165, 234–240.PubMedCrossRefGoogle Scholar
  18. 18.
    Berry, D. L., Moinpour, C. M., Jiang, C. S., et al. (2006). Quality of life and pain in advanced stage prostate cancer: Results of a Southwest Oncology Group randomized trial comparing docetaxel and estramustine to mitoxantrone and prednisone. Journal of Clinical Oncology, 24, 2828–2835.PubMedCrossRefGoogle Scholar
  19. 19.
    Pakhomov, S., Jacobsen, S. J., Chute, C. G., et al. (2008). Agreement between patient-reported symptoms and their documentation in the medical record. The American Journal of Managed Care, 14, 530–539.PubMedGoogle Scholar
  20. 20.
    Hahn, E. A., Cella, D., Chassany, O., et al. (2007). Precision of health-related quality-of-life data compared with other clinical measures. Mayo Clinic Proceedings, 82, 1244–1254.PubMedCrossRefGoogle Scholar
  21. 21.
    Ioannidis, J. P., Evans, S. J., Gøtzsche, P. C., et al. (2004). Better reporting of harms in randomized trials: An extension of the CONSORT statement. Annals of Internal Medicine, 141, 781–788.PubMedGoogle Scholar
  22. 22.
    Institute of Medicine, National Academy of Sciences. (2006). The future of drug safety: Promoting and protecting the health of the public. Washington, DC: National Academies Press.Google Scholar
  23. 23.
    Kirkova, J., Davis, M. P., Walsh, D., et al. (2006). Cancer symptom assessment instruments: A systematic review. Journal of Clinical Oncology, 24, 1459–1473.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  • Thomas M. Atkinson
    • 1
  • Yuelin Li
    • 1
  • Charles W. Coffey
    • 2
  • Laura Sit
    • 3
  • Mary Shaw
    • 3
  • Dawn Lavene
    • 3
  • Antonia V. Bennett
    • 3
  • Mike Fruscione
    • 3
  • Lauren Rogak
    • 3
  • Jennifer Hay
    • 1
  • Mithat Gönen
    • 3
  • Deborah Schrag
    • 4
  • Ethan Basch
    • 3
  1. 1.Department of Psychiatry and Behavioral SciencesMemorial Sloan-Kettering Cancer CenterNew YorkUSA
  2. 2.University of Kansas Medical CenterKansas CityUSA
  3. 3.Department of Epidemiology and BiostatisticsMemorial Sloan-Kettering Cancer CenterNew YorkUSA
  4. 4.Department of Outcomes ResearchDana-Farber Cancer InstituteBostonUSA

Personalised recommendations