Skip to main content

Advertisement

SpringerLink
Log in

Automated detection and classification of shoulder arthroplasty models using deep learning

  • Scientific Article
  • Published:
Skeletal Radiology Aims and scope Submit manuscript

Abstract

Objective

To develop and evaluate the performance of deep convolutional neural networks (DCNN) to detect and identify specific total shoulder arthroplasty (TSA) models.

Materials and methods

We included 482 radiography studies obtained from publicly available image repositories with native shoulders, reverse TSA (RTSA) implants, and five different TSA models. We trained separate ResNet DCNN–based binary classifiers to (1) detect the presence of shoulder arthroplasty implants, (2) differentiate between TSA and RTSA, and (3) differentiate between the five TSA models, using five individual classifiers for each model, respectively. Datasets were divided into training, validation, and test datasets. Training and validation datasets were 20-fold augmented. Test performances were assessed with area under the receiver-operating characteristic curves (AUC-ROC) analyses. Class activation mapping was used to identify distinguishing imaging features used for DCNN classification decisions.

Results

The DCNN for the detection of the presence of shoulder arthroplasty implants achieved an AUC-ROC of 1.0, whereas the AUC-ROC for differentiation between TSA and RTSA was 0.97. Class activation map analysis demonstrated the emphasis on the characteristic arthroplasty components in decision-making. DCNNs trained to distinguish between the five TSA models achieved AUC-ROCs ranging from 0.86 for Stryker Solar to 1.0 for Zimmer Bigliani-Flatow with class activation map analysis demonstrating an emphasis on unique implant design features.

Conclusion

DCNNs can accurately identify the presence of and distinguish between TSA & RTSA, and classify five specific TSA models with high accuracy. The proof of concept of these DCNNs may set the foundation for an automated arthroplasty atlas for rapid and comprehensive model identification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Roberts CC, Ekelund AL, Renfree KJ, Liu PT, Chew FS. Radiologic assessment of reverse shoulder arthroplasty. RadioGraphics. 2007;27:223–35. https://doi.org/10.1148/rg.271065076.

    Article  PubMed  Google Scholar 

  2. McFarland EG, Sanguanjit P, Tasaki A, Keyurapan E, Fishman EK, Fayad LM. The reverse shoulder prosthesis: a review of imaging features and complications. Skelet Radiol. 2006;35:488–96. https://doi.org/10.1007/s00256-006-0109-1.

    Article  Google Scholar 

  3. Dekker TJ, Steele JR, Vinson EV, Garrigues GE. Current peri-operative imaging concepts surrounding shoulder arthroplasty. Skelet Radiol. 2019;48:1485–97. https://doi.org/10.1007/s00256-019-03183-3.

    Article  Google Scholar 

  4. Lee DH, Choi YS, Potter HG, Endo Y, Sivakumaran T, Lim TK, et al. Reverse total shoulder arthroplasty: an imaging overview. Skelet Radiol. 2020;49:19–30. https://doi.org/10.1007/s00256-019-03275-0.

    Article  CAS  Google Scholar 

  5. Lin DJ, Wong TT, Kazam JK. Shoulder arthroplasty, from indications to complications: what the radiologist needs to know. RadioGraphics. 2016;36:192–208. https://doi.org/10.1148/rg.2016150055.

    Article  PubMed  Google Scholar 

  6. Branovacki G. Ortho atlas - hip arthroplasty - U.S. femoral implants 1938–2008: Ortho Atlas Publishing, Inc.; 2008.

  7. IMPLANT ATLAS — Hip & Knee Book https://hipandkneebook.com/hip-implants. Accessed 15 Dec 2019.

  8. Wilson NA, Jehn M, York S, Davis CM. Revision total hip and knee arthroplasty implant identification: implications for use of Unique Device Identification 2012 AAHKS member survey results. J Arthroplast. 2014;29:251–5. https://doi.org/10.1016/j.arth.2013.06.027.

    Article  Google Scholar 

  9. Chea P, Mandell JC. Current applications and future directions of deep learning in musculoskeletal radiology. Skelet Radiol. 2019:1–15. https://doi.org/10.1007/s00256-019-03284-z.

  10. Yi PH, Kim TK, Wei J, Shin J, Hui FK, Sair HI, et al. Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning. Pediatr Radiol. 2019;49:1066–70. https://doi.org/10.1007/s00247-019-04408-2.

    Article  PubMed  Google Scholar 

  11. Hemke R, Buckless CG, Tsao A, Wang B, Torriani M. Deep learning for automated segmentation of pelvic muscles, fat, and bone from CT studies for body composition assessment. Skelet Radiol. 2019:1–9. https://doi.org/10.1007/s00256-019-03289-8.

  12. Lee S, Choe EK, Kang HY, Yoon JW, Kim HS. The exploration of feature extraction and machine learning for predicting bone density from simple spine X-ray images in a Korean population. Skelet Radiol. 2019:1–6. https://doi.org/10.1007/s00256-019-03342-6.

  13. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88:581–6. https://doi.org/10.1080/17453674.2017.1344459.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol. 2018;73:439–45. https://doi.org/10.1016/j.crad.2017.11.015.

    Article  CAS  PubMed  Google Scholar 

  15. Lindsey R, Daluiski A, Chopra S, Lachapelle A, Mozer M, Sicular S, et al. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci. 2018;115:11591–6. https://doi.org/10.1073/pnas.1806905115.

    Article  CAS  PubMed  Google Scholar 

  16. Radiopaedia.org, the wiki-based collaborative Radiology resource https://radiopaedia.org/. Accessed 15 Dec 2019.

  17. Google https://www.google.com/. Accessed 15 Dec 2019.

  18. Open-i. https://openi.nlm.nih.gov/. Accessed 15 Dec 2019.

  19. Common US Shoulder Prostheses faculty.washington.edu/alexbert/Shoulder/CommonUSShoulderProstheses.htm. Accessed 15 Dec 2019.

  20. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8. https://doi.org/10.1038/nature21056.

    Article  CAS  PubMed  Google Scholar 

  21. Burlina PM, Joshi NJ, Ng E, Billings SD, Rebman AW, Aucott JN. Automated detection of erythema migrans and other confounding skin lesions via deep learning. Comput Biol Med. 2019;105:151–6. https://doi.org/10.1016/j.compbiomed.2018.12.007.

    Article  PubMed  Google Scholar 

  22. Lakhani P, Gray DL, Pett CR, Nagy P, Shih G. Hello world deep learning in medical imaging. J Digit Imaging. 2018;31:283–9. https://doi.org/10.1007/s10278-018-0079-6.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Yi PH, Wei J, Kim TK, Sair HI, Hui FK, Hager GD, et al. Automated detection & classification of knee arthroplasty using deep learning. Knee. 2019. https://doi.org/10.1016/j.knee.2019.11.020.

  24. Borjali A, Chen AF, Muratoglu OK, Morid MA, Varadarajan KM. Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network. J Orthop Res. 2020. https://doi.org/10.1002/jor.24617.

  25. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge 2014.

  26. Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, et al. Deep learning: a primer for radiologists. RadioGraphics. 2017;37:2113–31. https://doi.org/10.1148/rg.2017170077.

    Article  PubMed  Google Scholar 

  27. Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E. Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology. 2019;290:590–606. https://doi.org/10.1148/radiol.2018180547.

    Article  PubMed  Google Scholar 

  28. Cheng PM, Malhi HS. Transfer learning with convolutional neural networks for classification of abdominal ultrasound images. J Digit Imaging. 2017;30:234–43. https://doi.org/10.1007/s10278-016-9929-2.

    Article  PubMed  Google Scholar 

  29. Chung SW, Han SS, Lee JW, Oh K-S, Kim NR, Yoon JP, et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop. 2018;89:468–73. https://doi.org/10.1080/17453674.2018.1453714.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N. Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skelet Radiol. 2018. https://doi.org/10.1007/s00256-018-3016-3.

  31. Lakhani P. Deep convolutional neural networks for endotracheal tube position and X-ray image classification: challenges and opportunities. J Digit Imaging. 2017;30:460–8. https://doi.org/10.1007/s10278-017-9980-7.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Goksuluk D, Korkmaz S, Zararsiz G, Karaagaoglu A,Ergu. easyROC: an interactive web-tool for ROC curve analysis using R language environment. R J 2016;8:213. https://doi.org/10.32614/RJ-2016-042.

  33. Incorrectly Labeled RTSA 1. https://ryortho.com/wp-content/uploads/2015/03/Zimmer_ReverseTotalShoulderReplacementXRay_WEB.jpg. Accessed 19 Apr 2020.

  34. Incorrectly Labeled RTSA 2. http://www.jacksonorthopaedicsurgery.com/wp-content/uploads/2015/01/Monoblock_Post.jpg. Accessed 19 Apr 2020.

  35. Incorrectly Labeled TSA 1. http://www.jacksonorthopaedicsurgery.com/wp-content/uploads/2015/01/Turon_postop.jpg. Accessed 19 Apr 2020.

  36. Ghazala CG, Candal-Couto J. Anatomic shoulder arthroplasty. Orthop Traumatol. 2018;32:171–7. https://doi.org/10.1016/j.mporth.2018.03.005.

    Article  Google Scholar 

  37. Lucas RM, Hsu JE, Gee AO, Neradilek MB, Matsen FA. Impaction autografting: bone-preserving, secure fixation of a standard humeral component. J Shoulder Elb Surg. 2016;25:1787–94. https://doi.org/10.1016/J.JSE.2016.03.008.

    Article  Google Scholar 

  38. Killian ML, Cavinatto L, Galatz LM, Thomopoulos S. Recent advances in shoulder research. Arthritis Res Ther. 2012;14:214. https://doi.org/10.1186/AR3846.

    Article  PubMed  PubMed Central  Google Scholar 

  39. False Positive 1 for Stryker Solar Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/171b.jpg. Accessed 19 Apr 2020.

  40. False Positive 2 for Stryker Solar Classifier (Actual: Biomet Biomodular) http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/36i.jpg. Accessed 19 Apr 2020.

  41. False Positive 5 for DePuy HRP Classifier (Actual: Zimmer Bigliani Flatow). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/120f.jpg. Accessed 19 Apr 2020.

  42. False Positive 4 for DePuy HRP Classifier (Actual: Stryker Solar). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/290t.jpg. Accessed 19 Apr 2020.

  43. False Positive 1 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/34.jpg. Accessed 19 Apr 2020.

  44. False Positive 2 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/165c.jpg. Accessed 19 Apr 2020.

  45. False Positive 3 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/184s.jpg. Accessed 19 Apr 2020.

  46. False Positive for DePuy Global Classifier and False Negative for DePuy HRP Classifier (Actual: DePuy HRP). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/50.jpg. Accessed 19 Apr 2020.

  47. False Negative 1 for DePuy Global Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/174c.jpg. Accessed 19 Apr 2020.

  48. False Negative for Biomet Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/3c.jpg. Accessed 19 Apr 2020.

  49. False Negative 2 for Stryker Solar Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/82610579a.jpg. Accessed 19 Apr 2020.

  50. False Negative 1 for Stryker Solar Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/82610579c.jpg. Accessed 19 Apr 2020.

  51. False Negative 2 for DePuy Global Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/39c.jpg. Accessed 19 Apr 2020.

  52. Huang KT, Silva MA, See AP, Wu KC, Gallerani T, Zaidi HA, et al. A computer vision approach to identifying the manufacturer and model of anterior cervical spinal hardware. J Neurosurg Spine. 2019:1–7. https://doi.org/10.3171/2019.6.SPINE19463.

  53. Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284:574–82. https://doi.org/10.1148/radiol.2017162326.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins University School of Medicine, Baltimore, MD, USA

    Paul H. Yi, Tae Kyung Kim & Haris I. Sair

  2. Radiology Artificial Intelligence Lab (RAIL), Malone Center for Engineering in Healthcare, Johns Hopkins University Whiting School of Engineering, Baltimore, MD, USA

    Paul H. Yi, Tae Kyung Kim, Jinchi Wei, Gregory D. Hager & Haris I. Sair

  3. Department of Orthopaedic Surgery, Boston University School of Medicine, Boston, MA, USA

    Xinning Li

  4. Department of Radiology, Division of Musculoskeletal Radiology, New York University Grossman School of Medicine, 660 1st Ave, 3rd Floor, Rm #313, New York, NY, 10016, USA

    Jan Fritz

Authors
  1. Paul H. Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tae Kyung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinchi Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinning Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gregory D. Hager
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Haris I. Sair
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jan Fritz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Fritz.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PDF 1572 kb)

ESM 2

(PDF 79 kb)

ESM 3

(PDF 171 kb)

ESM 4

(PDF 79 kb)

ESM 5

(PDF 296 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yi, P.H., Kim, T.K., Wei, J. et al. Automated detection and classification of shoulder arthroplasty models using deep learning. Skeletal Radiol 49, 1623–1632 (2020). https://doi.org/10.1007/s00256-020-03463-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00256-020-03463-3

Keywords:

Search

Navigation