Annals of Surgical Oncology

, Volume 24, Issue 5, pp 1234–1241 | Cite as

Diagnostic Reproducibility: What Happens When the Same Pathologist Interprets the Same Breast Biopsy Specimen at Two Points in Time?

  • Sara L. Jackson
  • Paul D. Frederick
  • Margaret S. Pepe
  • Heidi D. Nelson
  • Donald L. Weaver
  • Kimberly H. Allison
  • Patricia A. Carney
  • Berta M. Geller
  • Anna N. A. Tosteson
  • Tracy Onega
  • Joann G. Elmore
Breast Oncology



Surgeons may receive a different diagnosis when a breast biopsy is interpreted by a second pathologist. The extent to which diagnostic agreement by the same pathologist varies at two time points is unknown.


Pathologists from eight U.S. states independently interpreted 60 breast specimens, one glass slide per case, on two occasions separated by ≥9 months. Reproducibility was assessed by comparing interpretations between the two time points; associations between reproducibility (intraobserver agreement rates); and characteristics of pathologists and cases were determined and also compared with interobserver agreement of baseline interpretations.


Sixty-five percent of invited, responding pathologists were eligible and consented; 49 interpreted glass slides in both study phases, resulting in 2940 interpretations. Intraobserver agreement rates between the two phases were 92% [95% confidence interval (CI) 88–95] for invasive breast cancer, 84% (95% CI 81–87) for ductal carcinoma-in-situ, 53% (95% CI 47–59) for atypia, and 84% (95% CI 81–86) for benign without atypia. When comparing all study participants’ case interpretations at baseline, interobserver agreement rates were 89% (95% CI 84–92) for invasive cancer, 79% (95% CI 76–81) for ductal carcinoma-in-situ, 43% (95% CI 41–45) for atypia, and 77% (95% CI 74–79) for benign without atypia.


Interpretive agreement between two time points by the same individual pathologist was low for atypia and was similar to observed rates of agreement for atypia between different pathologists. Physicians and patients should be aware of the diagnostic challenges associated with a breast biopsy diagnosis of atypia when considering treatment and surveillance decisions.


Breast Density Atypical Ductal Hyperplasia Agreement Rate Intraobserver Agreement Breast Pathology 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors appreciate the efforts of the pathologists who participated in this study. Supported in part by the National Cancer Institute of the National Institutes of Health (R01 CA140560, U54 CA163303, KO5 CA104699, and R01 CA172343); and by the National Cancer Institute–funded Breast Cancer Surveillance Consortium (HHSN261201100031C). The content is solely the responsibility of the authors and does not necessarily represent the views of the National Cancer Institute or the National Institutes of Health. The collection of cancer and vital status data used in this study was supported in part by several state public health departments and cancer registries throughout the United States. A full description of these sources is available online (


The authors declare no conflict of interest.


  1. 1.
    Welch HG, Black WC. Overdiagnosis in cancer. J Natl Cancer Inst. 2010;102:605–13.CrossRefPubMedGoogle Scholar
  2. 2.
    U.S. Preventive Services Task Force. Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2009;151:716–26.Google Scholar
  3. 3.
    Page DL, Schuyler PA, Dupont WD, Jensen RA, Plummer WD Jr, Simpson JF. Atypical lobular hyperplasia as a unilateral predictor of breast cancer risk: a retrospective cohort study. Lancet. 2003;361(9352):125–9.CrossRefPubMedGoogle Scholar
  4. 4.
    Hartmann LC, Degnim AC, Santen RJ, Dupont WD, Ghosh K. Atypical hyperplasia of the breast—risk assessment and management options. N Engl J Med. 2015;372:78–89.CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    Tuttle TM, Jarosek S, Habermann EB, et al. Increasing rates of contralateral prophylactic mastectomy among patients with ductal carcinoma in situ. J Clin Oncol. 2009;27:1362–7.CrossRefPubMedGoogle Scholar
  6. 6.
    Arrington AK, Jarosek SL, Virnig BA, Habermann EB, Tuttle TM. Patient and surgeon characteristics associated with increased use of contralateral prophylactic mastectomy in patients with breast cancer. Ann Surg Oncol. 2009;16:2697–704.CrossRefPubMedGoogle Scholar
  7. 7.
    Rosai J. Borderline epithelial lesions of the breast. Am J Surg. Pathol. 1991;15:209–21.CrossRefPubMedGoogle Scholar
  8. 8.
    Schnitt SJ, Connolly JL, Tavassoli FA, et al. Interobserver reproducibility in the diagnosis of ductal proliferative breast-lesions using standardized criteria. Am J Surg Pathol. 1992;16:1133–43.CrossRefPubMedGoogle Scholar
  9. 9.
    Wells WA, Carney PA, Eliassen MS, Tosteson AN, Greenberg ER. Statewide study of diagnostic agreement in breast pathology. J Natl Cancer Inst. 1998;90:142–5.CrossRefPubMedGoogle Scholar
  10. 10.
    Della Mea V, Puglisi F, Bonzanini M, et al. Fine-needle aspiration cytology of the breast: a preliminary report on telepathology through internet multimedia electronic mail. Mod Pathol. 1997;10:636–41.PubMedGoogle Scholar
  11. 11.
    Elmore JG, Longton G, Carney PA, et al. Diagnostic concordance among pathologists interpreting breast biopsy specimens. JAMA. 2015;313:1122–32.CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Elmore JG, Tosteson AN, Pepe MS, et al. Evaluation of 12 strategies for obtaining second opinions to improve interpretation of breast histopathology: simulation study. BMJ. 2016;353:i3069.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Geller BM, Nelson HD, Carney PA, et al. Second opinion in breast pathology: policy, practice and perception. J Clin Pathol. 2014;67:955–60.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Elmore JG, Harris RP. The harms and benefits of modern screening mammography. BMJ. 2014;348:g3824.CrossRefPubMedGoogle Scholar
  15. 15.
    O’Malley FP, Pinder SE, Mulligan AM. Breast pathology. Philadelphia: Elsevier/Saunders; 2011.Google Scholar
  16. 16.
    Schnitt SJ, Collins LC. Biopsy interpretation of the breast. Philadelphia: Wolters Kluwer Health/Lippincott Williams & Wilkins; 2009.Google Scholar
  17. 17.
    Jain RK, Mehta R, Dimitrov R, et al. Atypical ductal hyperplasia: interobserver and intraobserver variability. Mod Pathol. 2011;24:917–23.CrossRefPubMedGoogle Scholar
  18. 18.
    Onega T, Weaver D, Geller B, et al. Digitized whole slides for breast pathology interpretation: current practices and perceptions. J Digit Imaging. 2014;27:642–8.CrossRefPubMedPubMedCentralGoogle Scholar
  19. 19.
    National Cancer Institute. Breast Cancer Surveillance Consortium.
  20. 20.
    Allison KH, Reisch LM, Carney PA, et al. Understanding diagnostic variability in breast pathology: lessons learned from an expert consensus review panel. Histopathology. 2014;65:240–51.CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Esserman LJ, Thompson IM, Reid B, et al. Addressing overdiagnosis and overtreatment in cancer: a prescription for change. Lancet Oncol. 2014;15:e234–42.CrossRefPubMedPubMedCentralGoogle Scholar
  22. 22.
    Reyes C, Ikpatt OF, Nadji M, Cote RJ. Intra-observer reproducibility of whole slide imaging for the primary diagnosis of breast needle biopsies. J Pathol Inform. 2014;5:5.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Society of Surgical Oncology 2016

Authors and Affiliations

  • Sara L. Jackson
    • 1
  • Paul D. Frederick
    • 1
  • Margaret S. Pepe
    • 6
  • Heidi D. Nelson
    • 2
    • 3
  • Donald L. Weaver
    • 10
  • Kimberly H. Allison
    • 5
  • Patricia A. Carney
    • 4
  • Berta M. Geller
    • 7
  • Anna N. A. Tosteson
    • 8
    • 9
  • Tracy Onega
    • 8
  • Joann G. Elmore
    • 1
  1. 1.Department of MedicineUniversity of Washington School of MedicineSeattleUSA
  2. 2.Providence Cancer CenterProvidence Health and Services OregonPortlandUSA
  3. 3.Departments of Medical Informatics and Clinical Epidemiology and MedicineOregon Health & Science UniversityPortlandUSA
  4. 4.Department of Family MedicineOregon Health & Science UniversityPortlandUSA
  5. 5.Department of PathologyStanford University School of MedicineStanfordUSA
  6. 6.Program in Biostatistics and BiomathematicsFred Hutchinson Cancer Research CenterSeattleUSA
  7. 7.Department of Family MedicineUniversity of VermontBurlingtonUSA
  8. 8.Department of Community and Family MedicineThe Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Norris Cotton Cancer CenterLebanonUSA
  9. 9.Department of MedicineGeisel School of Medicine at DartmouthHanoverUSA
  10. 10.Department of Pathology and University of Vermont Cancer CenterUniversity of VermontBurlingtonUSA

Personalised recommendations