Skip to main content
SpringerLink
Log in

Automated estimation of image quality for coronary computed tomographic angiography using machine learning

  • Cardiac
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

Our goal was to evaluate the efficacy of a fully automated method for assessing the image quality (IQ) of coronary computed tomography angiography (CCTA).

Methods

The machine learning method was trained using 75 CCTA studies by mapping features (noise, contrast, misregistration scores, and un-interpretability index) to an IQ score based on manual ground truth data. The automated method was validated on a set of 50 CCTA studies and subsequently tested on a new set of 172 CCTA studies against visual IQ scores on a 5-point Likert scale.

Results

The area under the curve in the validation set was 0.96. In the 172 CCTA studies, our method yielded a Cohen’s kappa statistic for the agreement between automated and visual IQ assessment of 0.67 (p < 0.01). In the group where good to excellent (n = 163), fair (n = 6), and poor visual IQ scores (n = 3) were graded, 155, 5, and 2 of the patients received an automated IQ score > 50 %, respectively.

Conclusion

Fully automated assessment of the IQ of CCTA data sets by machine learning was reproducible and provided similar results compared with visual analysis within the limits of inter-operator variability.

Key points

• The proposed method enables automated and reproducible image quality assessment.

• Machine learning and visual assessments yielded comparable estimates of image quality.

• Automated assessment potentially allows for more standardised image quality.

• Image quality assessment enables standardization of clinical trial results across different datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

AUC:

Area under the curve

CAD:

Coronary artery disease

CCTA:

Coronary computed tomographic angiography

CNR:

Contrast-to-noise ratio

FFR:

Fractional flow reserve

ICA:

Invasive coronary angiography

IQ:

Image quality

ML:

Machine learning

References

  1. Budoff MJ, Dowe D, Jollis JG et al (2008) Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter ACCURACY (Assessment by Coronary Computed Tomographic Angiography of Individuals Undergoing Invasive Coronary Angiography) trial. J Am Coll Cardiol 52(21):1724–1732

    Article  PubMed  Google Scholar 

  2. Achenbach S, Moselewski F, Ropers D et al (2004) Detection of calcified and noncalcified coronary atherosclerotic plaque by contrast-enhanced, submillimeter multidetector spiral computed tomography: a segment-based comparison with intravascular ultrasound. Circulation. 109(1):14–17

    Article  PubMed  Google Scholar 

  3. Min JK, Leipsic J, Pencina MJ et al (2012a) Diagnostic accuracy of fractional flow reserve from anatomic CT angiography. JAMA. 308(12):1237–1245

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  4. Douglas PS, Hoffmann U, Patel MR et al (2015) Outcomes of anatomical versus functional testing for coronary artery disease. N Engl J Med. 372(14):1291–1300

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. SCOT-HEART investigators (2015) CT coronary angiography in patients with suspected angina due to coronary heart disease (SCOT-HEART): an open-label, parallel-group, multicentre trial. Lancet 385(9985):2383–91.

  6. Leipsic J, Abbara S, Achenbach S et al (2014) SCCT guidelines for the interpretation and reporting of coronary CT angiography: a report of the Society of Cardiovascular Computed Tomography Guidelines Committee. J Cardiovasc Comput Tomogr. 8(5):342–358

    Article  PubMed  Google Scholar 

  7. Naeemi MDRJ, Hollcroft N, Alessio AM, Roychowdhury S (2016) Application of big data analytics for automated estimation of CT image quality. Big Data (Big Data). IEEE International Conference on. IEEE 2016:3422–3431

    Google Scholar 

  8. Fronthaler HKK, Bigun J, Fierrez J, Alonso-Fernandez F, Ortega-Garcia J, Gonzalez-Rodriguez J (2008) Fingerprint image-quality estimation and its application to multialgorithm verification. IEEE Transactions on Information Forensics and Security. 3(2):331–338

    Article  Google Scholar 

  9. Marin TKM, Hendrik PP, Wernick MN, Yang Y, Brankov JG (2011) Numerical observer for cardiac motion assessment using machine learning. In Proc. of SPIE. 7966:79660G–796601

    Article  Google Scholar 

  10. Koo BK, Erglis A, Doh JH et al (2011) Diagnosis of ischemia-causing coronary stenoses by noninvasive fractional flow reserve computed from coronary computed tomographic angiograms. Results from the prospective multicenter DISCOVER-FLOW (Diagnosis of Ischemia-Causing Stenoses Obtained Via Noninvasive Fractional Flow Reserve) study. J Am Coll Cardiol. 58(19):1989–1997

    Article  PubMed  Google Scholar 

  11. Min JK, Berman DS, Budoff MJ et al (2011) Rationale and design of the DeFACTO (Determination of Fractional Flow Reserve by Anatomic Computed Tomographic AngiOgraphy) study. J Cardiovasc Comput Tomogr. 5(5):301–309

    Article  PubMed  Google Scholar 

  12. Leipsic J, Labounty TM, Heilbron B et al (2010) Adaptive statistical iterative reconstruction: assessment of image noise and image quality in coronary CT angiography. AJR Am J Roentgenol. 195(3):649–654

    Article  PubMed  Google Scholar 

  13. Min JK, Koo BK, Erglis A et al (2012b) Effect of image quality on diagnostic accuracy of noninvasive fractional flow reserve: results from the prospective multicenter international DISCOVER-FLOW study. J Cardiovasc Comput Tomogr. 6(3):191–199

    Article  PubMed  Google Scholar 

  14. Abbara S, Blanke P, Maroules CD et al (2016) SCCT guidelines for the performance and acquisition of coronary computed tomographic angiography: A report of the society of Cardiovascular Computed Tomography Guidelines Committee: Endorsed by the North American Society for Cardiovascular Imaging (NASCI). J Cardiovasc Comput Tomogr. 10(6):435–449

    Article  PubMed  Google Scholar 

  15. Sun K, Li K, Han R et al (2015) Evaluation of high-pitch dual-source CT angiography for evaluation of coronary and carotid-cerebrovascular arteries. Eur J Radiol. 84(3):398–406

    Article  PubMed  Google Scholar 

  16. Zir LM, Miller SW, Dinsmore RE, Gilbert JP, Harthorne JW (1976) Interobserver variability in coronary angiography. Circulation. 53(4):627–632

    Article  PubMed  CAS  Google Scholar 

  17. Cury RC, Abbara S, Achenbach S et al (2016) CAD-RADS(TM) Coronary Artery Disease - Reporting and Data System. An expert consensus document of the Society of Cardiovascular Computed Tomography (SCCT), the American College of Radiology (ACR) and the North American Society for Cardiovascular Imaging (NASCI). Endorsed by the American College of Cardiology. J Cardiovasc Comput Tomogr. 10(4):269–281

    Article  PubMed  Google Scholar 

  18. Hamon M, Biondi-Zoccai GG, Malagutti P et al (2006) Diagnostic performance of multislice spiral computed tomography of coronary arteries as compared with conventional invasive coronary angiography: a meta-analysis. J Am Coll Cardiol. 48(9):1896–1910

    Article  PubMed  Google Scholar 

  19. Puchner SB, Liu T, Mayrhofer T et al (2014) High-risk plaque detected on coronary CT angiography predicts acute coronary syndromes independent of significant stenosis in acute chest pain: results from the ROMICAT-II trial. J Am Coll Cardiol. 64(7):684–692

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

The authors state that this work has not received any funding.

Author information

Author notes

  1. Rine Nakanishi and Sethuraman Sankaran these two authors contributed equally to this work.

Authors and Affiliations

  1. Los Angeles Biomedical Research Institute at Harbor UCLA Medical Center, Torrance, CA, USA

    Rine Nakanishi, Jenifer Malpeso, Kazuhiro Osawa, Indre Ceponiene, Negin Nazarat, Sina Rahmani, Kendall Kissel, Eranthi Jayawardena, Christopher Dailing & Matthew J. Budoff

  2. HeartFlow Inc., Redwood City, CA, USA

    Sethuraman Sankaran, Leo Grady, Razik Yousfi, Christopher Zarins & Charles A. Taylor

  3. Department of Medicine, Seoul National University Hospital, Seoul, South Korea

    Bon-Kwon Koo

  4. Department of Radiology, Weill Cornell Medical College and the New York Presbyterian Hospital, New York, NY, USA

    James K. Min

Authors
  1. Rine Nakanishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sethuraman Sankaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leo Grady
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jenifer Malpeso
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Razik Yousfi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kazuhiro Osawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Indre Ceponiene
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Negin Nazarat
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Sina Rahmani
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Kendall Kissel
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Eranthi Jayawardena
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Christopher Dailing
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Christopher Zarins
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Bon-Kwon Koo
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. James K. Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Charles A. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Matthew J. Budoff
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew J. Budoff.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dr. Matthew J. Budoff.

Conflict of interest

The authors of this manuscript declare relationships with the following companies: Dr. Matthew Budoff receives grant support from GE Healthcare. Dr. Sankaran, Dr. Grady, Mr. Yousfi, Dr. Zarins, and Dr. Taylor are employees of HeartFlow. Dr. Min received modest speakers’ bureau medical advisory board compensation and significant research support from GE Healthcare. All other authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary in the current study.

Informed consent

Written informed consent was obtained from all subjects (patients) in this study.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

Some study subjects have been previously reported in the JAMA and JACC.

Methodology

• This is a retrospective observational study using two previous multicentre studies.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakanishi, R., Sankaran, S., Grady, L. et al. Automated estimation of image quality for coronary computed tomographic angiography using machine learning. Eur Radiol 28, 4018–4026 (2018). https://doi.org/10.1007/s00330-018-5348-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-018-5348-8

Keyword

Search

Navigation