Skip to main content

Advertisement

SpringerLink
Log in

Inter-rater variability of visual interpretation and comparison with quantitative evaluation of 11C-PiB PET amyloid images of the Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI) multicenter study

  • Original Article
  • Published:
European Journal of Nuclear Medicine and Molecular Imaging Aims and scope Submit manuscript

Abstract

Purpose

The aim of this study was to assess the inter-rater variability of the visual interpretation of 11C-PiB PET images regarding the positivity/negativity of amyloid deposition that were obtained in a multicenter clinical research project, Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI). The results of visual interpretation were also compared with a semi-automatic quantitative analysis using mean cortical standardized uptake value ratio to the cerebellar cortex (mcSUVR).

Methods

A total of 162 11C-PiB PET scans, including 45 mild Alzheimer’s disease, 60 mild cognitive impairment, and 57 normal cognitive control cases that had been acquired as J-ADNI baseline scans were analyzed. Based on visual interpretation by three independent raters followed by consensus read, each case was classified into positive, equivocal, and negative deposition (ternary criteria) and further dichotomized by merging the former two (binary criteria).

Results

Complete agreement of visual interpretation by the three raters was observed for 91.3% of the cases (Cohen κ = 0.88 on average) in ternary criteria and for 92.3% (κ = 0.89) in binary criteria. Cases that were interpreted as visually positive in the consensus read showed significantly higher mcSUVR than those visually negative (2.21 ± 0.37 vs. 1.27 ± 0.09, p < 0.001), and positive or negative decision by visual interpretation was dichotomized by a cut-off value of mcSUVR = 1.5. Significant positive/negative associations were observed between mcSUVR and the number of raters who evaluated as positive (ρ = 0.87, p < 0.0001) and negative (ρ = −0.85, p < 0.0001) interpretation. Cases of disagreement among raters showed generally low mcSUVR.

Conclusions

Inter-rater agreement was almost perfect in 11C-PiB PET scans. Positive or negative decision by visual interpretation was dichotomized by a cut-off value of mcSUVR = 1.5. As some cases of disagreement among raters tended to show low mcSUVR, referring to quantitative method may facilitate correct diagnosis when evaluating images of low amyloid deposition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack Jr CR, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7:263–9.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Karran E, Mercken M, De Strooper B. The amyloid cascade hypothesis for Alzheimer’s disease: an appraisal for the development of therapeutics. Nat Rev Drug Discov. 2011;10:698–712.

    Article  CAS  PubMed  Google Scholar 

  3. Jack Jr CR, Knopman DS, Jagust WJ, Shaw LM, Aisen PS, Weiner MW, et al. Hypothetical model of dynamic biomarkers of the Alzheimer’s pathological cascade. Lancet Neurol. 2010;9:119–28. doi:10.1016/S474-4422(09)70299-6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Klunk W, Engler H, Nordberg A, Wang Y, Blomqvist G, Holt D, et al. Imaging brain amyloid in Alzheimer’s disease with Pittsburgh Compound-B. Ann Neurol. 2004;55:306–19.

    Article  CAS  PubMed  Google Scholar 

  5. Klunk WE, Koeppe RA, Price JC, Benzinger TL, Devous Sr MD, Jagust WJ, et al. The Centiloid Project: standardizing quantitative amyloid plaque estimation by PET. Alzheimers Dement. 2015;11:1–15.

    Article  PubMed  Google Scholar 

  6. Sabri O, Seibyl J, Rowe C, Barthel H. Beta-amyloid imaging with florbetaben. Clin Transl Imaging. 2015;3:13–26.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Curtis C, Gamez JE, Singh U, Sadowsky CH, Villena T, Sabbagh MN, et al. Phase 3 trial of flutemetamol labeled with radioactive fluorine 18 imaging and neuritic plaque density. JAMA Neurol. 2015;72:287–94.

    Article  PubMed  Google Scholar 

  8. Yang L, Rieves D, Ganley C. Brain amyloid imaging—FDA approval of florbetapir F18 injection. N Engl J Med. 2012;367:885–7. doi:10.1056/NEJMp1208061.

    Article  CAS  PubMed  Google Scholar 

  9. Hatashita S, Yamasaki H, Suzuki Y, Tanaka K, Wakebe D, Hayakawa H. [18F]Flutemetamol amyloid-beta PET imaging compared with [11C]PIB across the spectrum of Alzheimer’s disease. Eur J Nucl Med Mol Imaging. 2014;41:290–300.

    Article  CAS  PubMed  Google Scholar 

  10. Rabinovici G, Rosen H, Alkalay A, Kornak J, Furst A, Agarwal N, et al. Amyloid vs. FDG-PET in the differential diagnosis of AD and FTLD. Neurology. 2011;77:2034–42.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Ng S, Villemagne VL, Berlangieri S, Lee ST, Cherk M, Gong SJ, et al. Visual assessment versus quantitative assessment of 11C-PIB PET and 18F-FDG PET for detection of Alzheimer’s disease. J Nucl Med. 2007;48:547–52.

    Article  CAS  PubMed  Google Scholar 

  12. Rosario BL, Weissfeld LA, Laymon CM, Mathis CA, Klunk WE, Berginc MD, et al. Inter-rater reliability of manual and automated region-of-interest delineation for PiB PET. Neuroimage. 2011;55:933–41.

    Article  PubMed  Google Scholar 

  13. Akamatsu G, Ikari Y, Ohnishi A, Nishida H, Aita K, Sasaki M, et al. Automated PET-only quantification of amyloid deposition with adaptive template and empirically pre-defined ROI. Phys Med Biol. 2016;61:5768–80.

    Article  CAS  PubMed  Google Scholar 

  14. Suotunen T, Hirvonen J, Immonen-Raiha P, Aalto S, Lisinen I, Arponen E, et al. Visual assessment of [(11)C]PIB PET in patients with cognitive impairment. Eur J Nucl Med Mol Imaging. 2010;37:1141–7.

    Article  PubMed  Google Scholar 

  15. Tolboom N, van der Flier WM, Boverhoff J, Yaqub M, Wattjes MP, Raijmakers PG, et al. Molecular imaging in the diagnosis of Alzheimer’s disease: visual assessment of [11C]PIB and [18F]FDDNP PET images. J Neurol Neurosurg Psychiatry. 2010;81:882–4.

    Article  PubMed  Google Scholar 

  16. Seibyl J, Catafau AM, Barthel H, Ishii K, Rowe CC, Leverenz JB, et al. Impact of training method on the robustness of the visual assessment of 18F-Florbetaben PET scans: results from a phase-3 study. J Nucl Med. 2016;57:900–6.

    Article  PubMed  Google Scholar 

  17. Nayate AP, Dubroff JG, Schmitt JE, Nasrallah I, Kishore R, Mankoff D, et al. Use of standardized uptake value ratios decreases interreader variability of [18F] Florbetapir PET brain scan interpretation. AJNR Am J Neuroradiol. 2015;36:1237–44.

    Article  CAS  PubMed  Google Scholar 

  18. Yamane T, Ikari Y, Nishio T, Ishii K, Ishii K, Kato T, et al. Visual-statistical interpretation of (18)F-FDG-PET images for characteristic Alzheimer patterns in a multicenter study: inter-rater concordance and relationship to automated quantitative evaluation. AJNR Am J Neuroradiol. 2014;35:244–9.

    Article  CAS  PubMed  Google Scholar 

  19. Clark CM, Pontecorvo MJ, Beach TG, Bedell BJ, Coleman RE, Doraiswamy PM, et al. Cerebral PET with florbetapir compared with neuropathology at autopsy for detection of neuritic amyloid-β plaques: a prospective cohort study. Lancet Neurol. 2012;11:669–78.

    Article  CAS  PubMed  Google Scholar 

  20. Tolboom N, van der Flier WM, Yaqub M, Koene T, Boellaard R, Windhorst AD, et al. Differential association of [11C]PIB and [18F]FDDNP binding with cognitive impairment. Neurology. 2009;73:2079–85.

    Article  CAS  PubMed  Google Scholar 

  21. Grimmer T, Wutz C, Alexopoulos P, Drzezga A, Forster S, Forstl H, et al. Visual versus fully automated analyses of 18F-FDG and Amyloid PET for prediction of dementia due to Alzheimer disease in mild cognitive impairment. J Nucl Med. 2016;57:204–7.

    Article  CAS  PubMed  Google Scholar 

  22. Iwatsubo T. Japanese Alzheimer’s disease neuroimaging initiative: present status and future. Alzheimers Dement. 2010;6:297–9.

    Article  PubMed  Google Scholar 

  23. Ikari Y, Nishio T, Makishi Y, Miya Y, Ito K, Koeppe RA, et al. Head motion evaluation and correction for PET scans with 18F-FDG in the Japanese Alzheimer’s disease neuroimaging initiative (J-ADNI) multi-center study. Ann Nucl Med. 2012;26:535–44.

    Article  PubMed  Google Scholar 

  24. Japanese Alzheimer’s disease neuroimaging initiative (J-ADNI) (NBDC Research ID: hum0043.v1). National Bioscience Database Center website. http://humandbs.biosciencedbc.jp/en/hum0043-v1.

  25. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan E. Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology. 1984;34:939–44.

    Article  CAS  PubMed  Google Scholar 

  26. Braak H, Braak E. Frequency of stages of Alzheimer-related lesions in different age categories. Neurobiol Aging. 1997;18:351–7.

    Article  CAS  PubMed  Google Scholar 

  27. Delacourte A, David JP, Sergeant N, Buee L, Wattez A, Vermersch P, et al. The biochemical pathway of neurofibrillary degeneration in aging and Alzheimer’s disease. Neurology. 1999;52:1158–65.

    Article  CAS  PubMed  Google Scholar 

  28. Thurfjell L, Lilja J, Lundqvist R, Buckley C, Smith A, Vandenberghe R, et al. Automated quantification of 18F-flutemetamol PET activity for categorizing scans as negative or positive for brain amyloid: concordance with visual image reads. J Nucl Med. 2014;55:1623–8.

    Article  CAS  PubMed  Google Scholar 

  29. Schreiber S, Landau SM, Fero A, Schreiber F, Jagust WJ. Alzheimer’s disease neuroimaging I. Comparison of visual and quantitative Florbetapir F 18 positron emission tomography analysis in predicting mild cognitive impairment outcomes. JAMA Neurol. 2015;72:1183–90.

    Article  PubMed  Google Scholar 

  30. Chen ML. Ethnic or racial differences revisited: impact of dosage regimen and dosage form on pharmacokinetics and pharmacodynamics. Clin Pharmacokinet. 2006;45:957–64.

    Article  CAS  PubMed  Google Scholar 

  31. Senda M, Yamamoto Y, Sasaki M, Yamane T, Brooks DJ, Farrar G, et al. An exploratory efficacy study of the amyloid imaging agent [(18)F]flutemetamol in Japanese Subjects. Ann Nucl Med. 2015;29:391–9.

    Article  CAS  PubMed  Google Scholar 

  32. Senda M, Sasaki M, Yamane T, Shimizu K, Patt M, Barthel H, et al. Ethnic comparison of pharmacokinetics of (18)F-florbetaben, a PET tracer for beta-amyloid imaging, in healthy Caucasian and Japanese subjects. Eur J Nucl Med Mol Imaging. 2015;42:89–96.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank the J-ADNI Imaging Pharmaceutical Industry Scientific Advisory Board and other organizations for their support of this work.

Author information

Authors and Affiliations

  1. Department of Nuclear Medicine, Saitama Medical University Saitama International Center, 1397-1 Yamane, Hidaka, 350-1298, Japan

    Tomohiko Yamane

  2. Division of Molecular Imaging, Institute of Biomedical Research and Innovation, 2-2 Minatojima-Minamimachi, Chuo-ku, Kobe, 650-0047, Japan

    Tomohiko Yamane, Yasuhiko Ikari, Tomoyuki Nishio & Michio Senda

  3. Team for Neuroimaging Research, Tokyo Metropolitan Institute of Gerontology, 35-2 Sakaecho, Itabashi-ku, Tokyo, 173-0015, Japan

    Tomohiko Yamane, Kenji Ishii & Muneyuki Sakata

  4. http://humandbs.biosciencedbc.jp/en/hum0043-v1

    Tomohiko Yamane, Kenji Ishii, Muneyuki Sakata, Yasuhiko Ikari, Tomoyuki Nishio, Kazunari Ishii, Takashi Kato, Kengo Ito & Michio Senda

  5. Research Association for Biotechnology, 1-7-1 Misuji, Taito-ku, Tokyo, 111-0055, Japan

    Yasuhiko Ikari & Tomoyuki Nishio

  6. Department of Radiology, Kinki University Hospital, 377-2 Onohigashi, Osaka, Sayama, 589-8511, Japan

    Kazunari Ishii

  7. Department of Brain Science and Molecular Imaging, National Center for Geriatrics and Gerontology, 7-430 Morioka-machi, Obu, 474-8511, Japan

    Takashi Kato & Kengo Ito

Authors
  1. Tomohiko Yamane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kenji Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muneyuki Sakata
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yasuhiko Ikari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tomoyuki Nishio
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kazunari Ishii
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Takashi Kato
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kengo Ito
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Michio Senda
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

J-ADNI Study Group

Corresponding author

Correspondence to Tomohiko Yamane.

Ethics declarations

Funding

This work is a part of the Translational Research Promotion Project/Research Project for the Development of a Systematic Method for the Assessment of Alzheimer’s Disease, sponsored by the New Energy and Industrial Technology Development Organization of Japan. The Japanese Alzheimer’s Disease Neuroimaging Initiative is also supported by a Grant-in-Aid for Comprehensive Research on Dementia from the Japanese Ministry of Health, Labour and Welfare, as well as by the grants from J-ADNI Pharmaceutical Industry Scientific Advisory Board companies.

Conflict of interest

TN has been an employee of Eli Lilly Japan since February 2016. However, all of his effort in the present study had been done when he was a member of Institute of Biomedical Research and Innovation. K. Ito has received research grants from Nihon Medi-Physics Co., Ltd. and a speaker honorarium from Eli Lilly Japan. MS is a principal investigator of clinical trials sponsored by AVID/Eli Lilly and GE Healthcare. MS is also an advisor for AVID/Eli Lilly, GE Healthcare and Piramal Imaging for which fee was paid to the institution.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the each institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Written informed consent was obtained from every subject prior to participation and also from caregiver of the subject categorized as AD or MCI.

Additional information

Research group of the Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI) comprised investigators from 38 different facilities. The investigators contributed to the design and implementation of J-ADNI and/or provided data but did not participate in the analyses of this report.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yamane, T., Ishii, K., Sakata, M. et al. Inter-rater variability of visual interpretation and comparison with quantitative evaluation of 11C-PiB PET amyloid images of the Japanese Alzheimer’s Disease Neuroimaging Initiative (J-ADNI) multicenter study. Eur J Nucl Med Mol Imaging 44, 850–857 (2017). https://doi.org/10.1007/s00259-016-3591-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00259-016-3591-2

Keywords

Search

Navigation