Automated detection and classification of shoulder arthroplasty models using deep learning

Abstract

Objective

To develop and evaluate the performance of deep convolutional neural networks (DCNN) to detect and identify specific total shoulder arthroplasty (TSA) models.

Materials and methods

We included 482 radiography studies obtained from publicly available image repositories with native shoulders, reverse TSA (RTSA) implants, and five different TSA models. We trained separate ResNet DCNN–based binary classifiers to (1) detect the presence of shoulder arthroplasty implants, (2) differentiate between TSA and RTSA, and (3) differentiate between the five TSA models, using five individual classifiers for each model, respectively. Datasets were divided into training, validation, and test datasets. Training and validation datasets were 20-fold augmented. Test performances were assessed with area under the receiver-operating characteristic curves (AUC-ROC) analyses. Class activation mapping was used to identify distinguishing imaging features used for DCNN classification decisions.

Results

The DCNN for the detection of the presence of shoulder arthroplasty implants achieved an AUC-ROC of 1.0, whereas the AUC-ROC for differentiation between TSA and RTSA was 0.97. Class activation map analysis demonstrated the emphasis on the characteristic arthroplasty components in decision-making. DCNNs trained to distinguish between the five TSA models achieved AUC-ROCs ranging from 0.86 for Stryker Solar to 1.0 for Zimmer Bigliani-Flatow with class activation map analysis demonstrating an emphasis on unique implant design features.

Conclusion

DCNNs can accurately identify the presence of and distinguish between TSA & RTSA, and classify five specific TSA models with high accuracy. The proof of concept of these DCNNs may set the foundation for an automated arthroplasty atlas for rapid and comprehensive model identification.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. 1.

    Roberts CC, Ekelund AL, Renfree KJ, Liu PT, Chew FS. Radiologic assessment of reverse shoulder arthroplasty. RadioGraphics. 2007;27:223–35. https://doi.org/10.1148/rg.271065076.

    Article  PubMed  Google Scholar 

  2. 2.

    McFarland EG, Sanguanjit P, Tasaki A, Keyurapan E, Fishman EK, Fayad LM. The reverse shoulder prosthesis: a review of imaging features and complications. Skelet Radiol. 2006;35:488–96. https://doi.org/10.1007/s00256-006-0109-1.

    Article  Google Scholar 

  3. 3.

    Dekker TJ, Steele JR, Vinson EV, Garrigues GE. Current peri-operative imaging concepts surrounding shoulder arthroplasty. Skelet Radiol. 2019;48:1485–97. https://doi.org/10.1007/s00256-019-03183-3.

    Article  Google Scholar 

  4. 4.

    Lee DH, Choi YS, Potter HG, Endo Y, Sivakumaran T, Lim TK, et al. Reverse total shoulder arthroplasty: an imaging overview. Skelet Radiol. 2020;49:19–30. https://doi.org/10.1007/s00256-019-03275-0.

    CAS  Article  Google Scholar 

  5. 5.

    Lin DJ, Wong TT, Kazam JK. Shoulder arthroplasty, from indications to complications: what the radiologist needs to know. RadioGraphics. 2016;36:192–208. https://doi.org/10.1148/rg.2016150055.

    Article  PubMed  Google Scholar 

  6. 6.

    Branovacki G. Ortho atlas - hip arthroplasty - U.S. femoral implants 1938–2008: Ortho Atlas Publishing, Inc.; 2008.

  7. 7.

    IMPLANT ATLAS — Hip & Knee Book https://hipandkneebook.com/hip-implants. Accessed 15 Dec 2019.

  8. 8.

    Wilson NA, Jehn M, York S, Davis CM. Revision total hip and knee arthroplasty implant identification: implications for use of Unique Device Identification 2012 AAHKS member survey results. J Arthroplast. 2014;29:251–5. https://doi.org/10.1016/j.arth.2013.06.027.

    Article  Google Scholar 

  9. 9.

    Chea P, Mandell JC. Current applications and future directions of deep learning in musculoskeletal radiology. Skelet Radiol. 2019:1–15. https://doi.org/10.1007/s00256-019-03284-z.

  10. 10.

    Yi PH, Kim TK, Wei J, Shin J, Hui FK, Sair HI, et al. Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning. Pediatr Radiol. 2019;49:1066–70. https://doi.org/10.1007/s00247-019-04408-2.

    Article  PubMed  Google Scholar 

  11. 11.

    Hemke R, Buckless CG, Tsao A, Wang B, Torriani M. Deep learning for automated segmentation of pelvic muscles, fat, and bone from CT studies for body composition assessment. Skelet Radiol. 2019:1–9. https://doi.org/10.1007/s00256-019-03289-8.

  12. 12.

    Lee S, Choe EK, Kang HY, Yoon JW, Kim HS. The exploration of feature extraction and machine learning for predicting bone density from simple spine X-ray images in a Korean population. Skelet Radiol. 2019:1–6. https://doi.org/10.1007/s00256-019-03342-6.

  13. 13.

    Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88:581–6. https://doi.org/10.1080/17453674.2017.1344459.

    Article  PubMed  PubMed Central  Google Scholar 

  14. 14.

    Kim DH, MacKinnon T. Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol. 2018;73:439–45. https://doi.org/10.1016/j.crad.2017.11.015.

    CAS  Article  PubMed  Google Scholar 

  15. 15.

    Lindsey R, Daluiski A, Chopra S, Lachapelle A, Mozer M, Sicular S, et al. Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci. 2018;115:11591–6. https://doi.org/10.1073/pnas.1806905115.

    CAS  Article  PubMed  Google Scholar 

  16. 16.

    Radiopaedia.org, the wiki-based collaborative Radiology resource https://radiopaedia.org/. Accessed 15 Dec 2019.

  17. 17.

    Google https://www.google.com/. Accessed 15 Dec 2019.

  18. 18.

    Open-i. https://openi.nlm.nih.gov/. Accessed 15 Dec 2019.

  19. 19.

    Common US Shoulder Prostheses faculty.washington.edu/alexbert/Shoulder/CommonUSShoulderProstheses.htm. Accessed 15 Dec 2019.

  20. 20.

    Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115–8. https://doi.org/10.1038/nature21056.

    CAS  Article  PubMed  Google Scholar 

  21. 21.

    Burlina PM, Joshi NJ, Ng E, Billings SD, Rebman AW, Aucott JN. Automated detection of erythema migrans and other confounding skin lesions via deep learning. Comput Biol Med. 2019;105:151–6. https://doi.org/10.1016/j.compbiomed.2018.12.007.

    Article  PubMed  Google Scholar 

  22. 22.

    Lakhani P, Gray DL, Pett CR, Nagy P, Shih G. Hello world deep learning in medical imaging. J Digit Imaging. 2018;31:283–9. https://doi.org/10.1007/s10278-018-0079-6.

    Article  PubMed  PubMed Central  Google Scholar 

  23. 23.

    Yi PH, Wei J, Kim TK, Sair HI, Hui FK, Hager GD, et al. Automated detection & classification of knee arthroplasty using deep learning. Knee. 2019. https://doi.org/10.1016/j.knee.2019.11.020.

  24. 24.

    Borjali A, Chen AF, Muratoglu OK, Morid MA, Varadarajan KM. Detecting total hip replacement prosthesis design on plain radiographs using deep convolutional neural network. J Orthop Res. 2020. https://doi.org/10.1002/jor.24617.

  25. 25.

    Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge 2014.

  26. 26.

    Chartrand G, Cheng PM, Vorontsov E, Drozdzal M, Turcotte S, Pal CJ, et al. Deep learning: a primer for radiologists. RadioGraphics. 2017;37:2113–31. https://doi.org/10.1148/rg.2017170077.

    Article  PubMed  Google Scholar 

  27. 27.

    Soffer S, Ben-Cohen A, Shimon O, Amitai MM, Greenspan H, Klang E. Convolutional neural networks for radiologic images: a radiologist’s guide. Radiology. 2019;290:590–606. https://doi.org/10.1148/radiol.2018180547.

    Article  PubMed  Google Scholar 

  28. 28.

    Cheng PM, Malhi HS. Transfer learning with convolutional neural networks for classification of abdominal ultrasound images. J Digit Imaging. 2017;30:234–43. https://doi.org/10.1007/s10278-016-9929-2.

    Article  PubMed  Google Scholar 

  29. 29.

    Chung SW, Han SS, Lee JW, Oh K-S, Kim NR, Yoon JP, et al. Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop. 2018;89:468–73. https://doi.org/10.1080/17453674.2018.1453714.

    Article  PubMed  PubMed Central  Google Scholar 

  30. 30.

    Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N. Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skelet Radiol. 2018. https://doi.org/10.1007/s00256-018-3016-3.

  31. 31.

    Lakhani P. Deep convolutional neural networks for endotracheal tube position and X-ray image classification: challenges and opportunities. J Digit Imaging. 2017;30:460–8. https://doi.org/10.1007/s10278-017-9980-7.

    Article  PubMed  PubMed Central  Google Scholar 

  32. 32.

    Goksuluk D, Korkmaz S, Zararsiz G, Karaagaoglu A,Ergu. easyROC: an interactive web-tool for ROC curve analysis using R language environment. R J 2016;8:213. https://doi.org/10.32614/RJ-2016-042.

  33. 33.

    Incorrectly Labeled RTSA 1. https://ryortho.com/wp-content/uploads/2015/03/Zimmer_ReverseTotalShoulderReplacementXRay_WEB.jpg. Accessed 19 Apr 2020.

  34. 34.

    Incorrectly Labeled RTSA 2. http://www.jacksonorthopaedicsurgery.com/wp-content/uploads/2015/01/Monoblock_Post.jpg. Accessed 19 Apr 2020.

  35. 35.

    Incorrectly Labeled TSA 1. http://www.jacksonorthopaedicsurgery.com/wp-content/uploads/2015/01/Turon_postop.jpg. Accessed 19 Apr 2020.

  36. 36.

    Ghazala CG, Candal-Couto J. Anatomic shoulder arthroplasty. Orthop Traumatol. 2018;32:171–7. https://doi.org/10.1016/j.mporth.2018.03.005.

    Article  Google Scholar 

  37. 37.

    Lucas RM, Hsu JE, Gee AO, Neradilek MB, Matsen FA. Impaction autografting: bone-preserving, secure fixation of a standard humeral component. J Shoulder Elb Surg. 2016;25:1787–94. https://doi.org/10.1016/J.JSE.2016.03.008.

    Article  Google Scholar 

  38. 38.

    Killian ML, Cavinatto L, Galatz LM, Thomopoulos S. Recent advances in shoulder research. Arthritis Res Ther. 2012;14:214. https://doi.org/10.1186/AR3846.

    Article  PubMed  PubMed Central  Google Scholar 

  39. 39.

    False Positive 1 for Stryker Solar Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/171b.jpg. Accessed 19 Apr 2020.

  40. 40.

    False Positive 2 for Stryker Solar Classifier (Actual: Biomet Biomodular) http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/36i.jpg. Accessed 19 Apr 2020.

  41. 41.

    False Positive 5 for DePuy HRP Classifier (Actual: Zimmer Bigliani Flatow). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/120f.jpg. Accessed 19 Apr 2020.

  42. 42.

    False Positive 4 for DePuy HRP Classifier (Actual: Stryker Solar). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/290t.jpg. Accessed 19 Apr 2020.

  43. 43.

    False Positive 1 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/34.jpg. Accessed 19 Apr 2020.

  44. 44.

    False Positive 2 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/165c.jpg. Accessed 19 Apr 2020.

  45. 45.

    False Positive 3 for DePuy HRP Classifier (Actual: DePuy Global). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/184s.jpg. Accessed 19 Apr 2020.

  46. 46.

    False Positive for DePuy Global Classifier and False Negative for DePuy HRP Classifier (Actual: DePuy HRP). http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/50.jpg. Accessed 19 Apr 2020.

  47. 47.

    False Negative 1 for DePuy Global Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/174c.jpg. Accessed 19 Apr 2020.

  48. 48.

    False Negative for Biomet Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/3c.jpg. Accessed 19 Apr 2020.

  49. 49.

    False Negative 2 for Stryker Solar Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/82610579a.jpg. Accessed 19 Apr 2020.

  50. 50.

    False Negative 1 for Stryker Solar Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/82610579c.jpg. Accessed 19 Apr 2020.

  51. 51.

    False Negative 2 for DePuy Global Classifier. http://faculty.washington.edu/alexbert/Shoulder/Prostheses/Xrays/39c.jpg. Accessed 19 Apr 2020.

  52. 52.

    Huang KT, Silva MA, See AP, Wu KC, Gallerani T, Zaidi HA, et al. A computer vision approach to identifying the manufacturer and model of anterior cervical spinal hardware. J Neurosurg Spine. 2019:1–7. https://doi.org/10.3171/2019.6.SPINE19463.

  53. 53.

    Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284:574–82. https://doi.org/10.1148/radiol.2017162326.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jan Fritz.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(PDF 1572 kb)

ESM 2

(PDF 79 kb)

ESM 3

(PDF 171 kb)

ESM 4

(PDF 79 kb)

ESM 5

(PDF 296 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Yi, P.H., Kim, T.K., Wei, J. et al. Automated detection and classification of shoulder arthroplasty models using deep learning. Skeletal Radiol 49, 1623–1632 (2020). https://doi.org/10.1007/s00256-020-03463-3

Download citation

Keywords:

  • Shoulder arthroplasty
  • Deep learning
  • Artificial intelligence
  • Implant identification