Journal of Digital Imaging

, Volume 31, Issue 5, pp 604–610 | Cite as

Efficiency Improvement in a Busy Radiology Practice: Determination of Musculoskeletal Magnetic Resonance Imaging Protocol Using Deep-Learning Convolutional Neural Networks

  • Young Han Lee


The purposes of this study are to evaluate the feasibility of protocol determination with a convolutional neural networks (CNN) classifier based on short-text classification and to evaluate the agreements by comparing protocols determined by CNN with those determined by musculoskeletal radiologists. Following institutional review board approval, the database of a hospital information system (HIS) was queried for lists of MRI examinations, referring department, patient age, and patient gender. These were exported to a local workstation for analyses: 5258 and 1018 consecutive musculoskeletal MRI examinations were used for the training and test datasets, respectively. The subjects for pre-processing were routine or tumor protocols and the contents were word combinations of the referring department, region, contrast media (or not), gender, and age. The CNN Embedded vector classifier was used with Word2Vec Google news vectors. The test set was tested with each classification model and results were output as routine or tumor protocols. The CNN determinations were evaluated using the receiver operating characteristic (ROC) curves. The accuracies were evaluated by a radiologist-confirmed protocol as the reference protocols. The optimal cut-off values for protocol determination between routine protocols and tumor protocols was 0.5067 with a sensitivity of 92.10%, a specificity of 95.76%, and an area under curve (AUC) of 0.977. The overall accuracy was 94.2% for the ConvNet model. All MRI protocols were correct in the pelvic bone, upper arm, wrist, and lower leg MRIs. Deep-learning-based convolutional neural networks were clinically utilized to determine musculoskeletal MRI protocols. CNN-based text learning and applications could be extended to other radiologic tasks besides image interpretations, improving the work performance of the radiologist.


Artificial neural networks Machine learning Magnetic resonance imaging protocol Image protocols 



The authors would like to thank Kwan-Yuet Ho, PhD (Developer of ShortText) for his help with the installation and application.


This work was supported by a National Research Foundation (NRF) grant funded by the Korea Government, Ministry of Science and ICT (MSIP, 2018R1A2B6009076).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.


  1. 1.
    ACR Practice parameter for performing and interpreting magnetic resonance imaging (MRI). Available at: Accessed Nov 1, 2017
  2. 2.
    Edelstein WA, Mahesh M, Carrino JA: MRI: time is dose—and money and versatility. J Am Coll Radiol 7:650–652, 2010CrossRefGoogle Scholar
  3. 3.
    Mekle R, Wu EX, Meckel S, Wetzel SG, Scheffler K: Combo acquisitions: balancing scan time reduction and image quality. Magn Reson Med 55:1093–1105, 2006CrossRefGoogle Scholar
  4. 4.
    LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 521:436–444, 2015CrossRefGoogle Scholar
  5. 5.
    Ayaru L, Ypsilantis PP, Nanapragasam A, Choi RCH, Thillanathan A, Min-Ho L, Montana G: Prediction of outcome in acute lower gastrointestinal bleeding using gradient boosting. PLoS One 10:e0132485, 2015CrossRefGoogle Scholar
  6. 6.
    Ogutu JO, Schulz-Streeck T, Piepho HP: Genomic selection using regularized linear regression models: ridge regression, lasso, elastic net and their extensions. BMC Proc 6(Suppl 2):S10, 2012CrossRefGoogle Scholar
  7. 7.
    Forsberg D, Sjoblom E, Sunshine JL: Detection and labeling of vertebrae in MR images using deep learning with clinical annotations as training data. J Digit Imaging 30:406–412, 2017CrossRefGoogle Scholar
  8. 8.
    Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH: Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations using IBM Watson’s natural language processing algorithm. J Digit Imaging., 2017CrossRefGoogle Scholar
  9. 9.
    Kim Y: Convolutional neural networks for sentence classification. ArXiv e-prints abs/1408.5882, 2014Google Scholar
  10. 10.
    Thrall JH, Li X, Li Q, Cruz C, Do S, Dreyer K, Brink J: Artificial intelligence and machine learning in radiology: opportunities, challenges, pitfalls, and criteria for success. J Am Coll Radiol 15:504–508, 2018CrossRefGoogle Scholar
  11. 11.
    Kim Y: Convolutional neural networks for sentence classification. CoRR abs/1408.5882 %U, 2014
  12. 12.
    Pereira S, Pinto A, Alves V, Silva CA: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans Med Imaging 35:1240–1251, 2016CrossRefGoogle Scholar
  13. 13.
    Setio AAA, Ciompi F, Litjens G, Gerke P, Jacobs C, van Riel SJ, Wille MMW, Naqibullah M, Sanchez CI, van Ginneken B: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging 35:1160–1169, 2016CrossRefGoogle Scholar
  14. 14.
    Sangwaiya MJ, Saini S, Blake MA, Dreyer KJ, Kalra MK: Errare humanum est: frequency of laterality errors in radiology reports. AJR Am J Roentgenol 192:W239–W244, 2009CrossRefGoogle Scholar
  15. 15.
    Luetmer MT, Hunt CH, McDonald RJ, Bartholmai BJ, Kallmes DF: Laterality errors in radiology reports generated with and without voice recognition software: frequency and clinical significance. J Am Coll Radiol 10:538–543, 2013CrossRefGoogle Scholar
  16. 16.
    Lee YH, Yang J, Suh JS: Detection and correction of laterality errors in radiology reports. J Digit Imaging 28:412–416, 2015CrossRefGoogle Scholar
  17. 17.
    Siegel E. It’s the effect size, stupid. Will Computers Replace Radiologists? Debunking the hype of AI. Available at: Accessed Apr 1, 2017
  18. 18.
    Gyftopoulos S, Kim D, Aaltonen E, Horwitz LI: Patient recall imaging in the ambulatory setting. Am J Roentgenol 206:787–791, 2016CrossRefGoogle Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2018

Authors and Affiliations

  1. 1.Department of Radiology, Research Institute of Radiological Science, YUHS-KRIBB Medical Convergence Research Institute and Center for Clinical Imaging Data ScienceYonsei University College of MedicineSeoulSouth Korea

Personalised recommendations