Deep Learning Technique for Musculoskeletal Analysis

  • Naoki KamiyaEmail author
Part of the Advances in Experimental Medicine and Biology book series (AEMB, volume 1213)


Advancements in musculoskeletal analysis have been achieved by adopting deep learning technology in image recognition and analysis. Unlike musculoskeletal modeling based on computational anatomy, deep learning-based methods can obtain muscle information automatically. Through analysis of image features, both approaches can obtain muscle characteristics such as shape, volume, and area, and derive additional information by analyzing other image textures. In this chapter, we first discuss the necessity of musculoskeletal analysis and the required image processing technology. Then, the limitations of skeletal muscle recognition based on conventional handcrafted features are discussed, and developments in skeletal muscle recognition using machine learning and deep learning technology are described. Next, a technique for analyzing musculoskeletal systems using whole-body computed tomography (CT) images is shown. This study aims to achieve automatic recognition of skeletal muscles throughout the body and automatic classification of atrophic muscular disease using only image features, to demonstrate an application of whole-body musculoskeletal analysis driven by deep learning. Finally, we discuss future development of musculoskeletal analysis that effectively combines deep learning with handcrafted feature-based modeling techniques.


Skeletal muscle Musculoskeletal analysis Musculoskeletal segmentation Surface muscle Deep muscle Random forest FCN-8s 2D U-Net 3D U-Net 



The authors would like to thank all the members of the Fujita Laboratory in the Graduate School of Medicine, Gifu University, for their collaboration. We especially thank Ms. Oshima and Mr. Wakamatsu in the School of Information Science and Technology, Aichi Prefectural University for providing binary code and testing systems. This work was supported in part by a JSPS Grant-in-Aid for Scientific Research on Innovative Areas (Multidisciplinary Computational Anatomy, #26108005 and #17H05301) and for Challenging Exploratory Research (#16K15346), Japan.


  1. 1.
    Yu W, Liu W, Tan L et al (2018) Multi-object model-based multi-atlas segmentation constrained grid cut for automatic segmentation of lumbar vertebrae from CT images, intelligent orthopaedics. Adv Exp Med Biol 1093:65–71. CrossRefPubMedGoogle Scholar
  2. 2.
    Zeng G, Zheng G (2018) Deep learning-based automatic segmentation of the proximal femur from MR images, intelligent orthopaedics. Adv Exp Med Biol 1093:73–79. CrossRefPubMedGoogle Scholar
  3. 3.
    Yu W, Zheng G (2018) Atlas-based 3D intensity volume reconstruction from 2D long leg standing X-rays: application to hard and soft tissues in lower extremity, intelligent orthopaedics. Adv Exp Med Biol 1093:105–112. CrossRefPubMedGoogle Scholar
  4. 4.
    Kamiya N (2018) Muscle segmentation for orthopedic interventions, intelligent orthopaedics. Adv Exp Med Biol 1093:81–91. CrossRefPubMedGoogle Scholar
  5. 5.
    Rothstein JD (2017) Edaravone: a new drug approved for ALS. Cell 171(4):725CrossRefGoogle Scholar
  6. 6.
    Ministry of Health, Labour and Welfare, JAPAN, National Health Promotion Movement in the 21st Century (Healthy Japan 21)Google Scholar
  7. 7.
    Kobatake H, Masutani Y et al (2017) Computational anatomy based on whole body imaging: basic principles of computer-assisted diagnosis and therapy. SpringerGoogle Scholar
  8. 8.
    Hanaoka S, Kamiya N, Sato Y et al (2017) Skeletal muscle, understanding medical images based on computational anatomy models. Springer, pp 165–171Google Scholar
  9. 9.
    Fujita H, Hara T, Zhou X et al (2014) Model construction for computational anatomy: progress overview FY2009-FY2013. In: Proceedings of the Fifth International Symposium on the Project “Computational Anatomy”, pp 25–35Google Scholar
  10. 10.
    Multidisciplinary Computational Anatomy and Its Application to Highly Intelligent Diagnosis and Therapy.
  11. 11.
    Fujita H, Hara T, Zhou X et al (2019) Function integrated diagnostic assistance based on multidisciplinary computational anatomy models -Progress Overview FY2014-FY2018-. In: Proceedings of the Fifth International Symposium on the Project “Multidisciplinary Computational Anatomy”, pp 115–128Google Scholar
  12. 12.
    Tanimura K, Sato S, Fuseya Y et al (2016) Quantitative assessment of erector spinae muscles in patients with chronic obstructive pulmonary disease. Novel chest computed tomography-derived index for prognosis. Ann Am Thorac Soc 13(3):334–341CrossRefGoogle Scholar
  13. 13.
    Kamiya N, Li J, Kume M et al (2018) Fully automatic segmentation of paraspinal muscles from 3D torso CT images via multi-scale iterative random forest classifications. Int J Comput Assist Radiol Surg 13(11):1697–1706. CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Kume M, Kamiya N, Zhou X et al (2017) Automated recognition of the erector spinae muscle based on deep CNN at the level of the twelfth thoracic vertebrae in torso CT images. In: Proceedings of the 36th JAMIT annual meetingGoogle Scholar
  15. 15.
    Zhou X, Takayama R, Wang S et al (2017) Deep learning of the sectional appearances of 3D CT images for anatomical structure segmentation based on an FCN voting method. Med Phys 44(10):5221–5233. CrossRefPubMedGoogle Scholar
  16. 16.
    Kamiya N, Kume M, Zheng G et al (2019) Automated recognition of erector spinae muscles and their skeletal attachment region via deep learning in torso CT images. Comput Methods Clin Appl Musculoskelet Imaging:1–10. Google Scholar
  17. 17.
    Zhou X, Ito T, Takayama R et al (2016) Three-dimensional CT image segmentation by combining 2D fully convolutional network with 3D majority voting. In: Proceedings of the Workshop on the 2nd Deep Learning in Medical Image Analysis (DLMIA) in MICCAI 2016, LNCS 10008, pp 111–120CrossRefGoogle Scholar
  18. 18.
    Kamiya N, Ieda K, Zhou X et al (2017) Automated analysis of whole skeletal muscle for muscular atrophy detection of ALS in whole-body CT images: preliminary study. In: Proceedings of the SPIE Medical Imaging 2017, Computer-Aided Diagnosis, 10134, 1013442-1-1013442-6.
  19. 19.
    Kamiya N, Oshima A, Asano E et al (2019) Initial study on the classification of amyotrophic diseases using texture analysis and deep learning in whole-body CT images. In: Proceedings of the SPIE 11050, International Forum on Medical Imaging in Asia 2019, 110500X.
  20. 20.
    Oshima A, Kamiya N, Zhou X et al (2019) Automated segmentation of surface muscle in whole-body CT images using 2D U-Net: preliminary study. In: Proceedings of the IEEE EMBC2019, ThPOS-32.34, p 71Google Scholar
  21. 21.
    Wakamatsu Y, Kamiya N, Zhou X et al (2019) Bone segmentation in whole-body CT images using 2D U-Net. In: Proceedings of the IEEE EMBC2019, ThPOS-32.35, p 72Google Scholar
  22. 22.
    Klein A, Warszawski J, Hillengaß J et al (2019) Automatic bone segmentation in whole-body CT images. Int J Comput Assist Radiol Surg 14(1):21–29CrossRefGoogle Scholar
  23. 23.
    Kume M, Kamiya N, Zhou X et al (2019) Development of representation method of muscle running using attachment region of the spinal column erector muscle in the torso CT images. IEICE Tech Rep 118(412):39–40Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Information Science and Technology, Aichi Prefectural UniversityNagakuteJapan

Personalised recommendations