Predicting Scoliosis in DXA Scans Using Intermediate Representations

  • Amir JamaludinEmail author
  • Timor Kadir
  • Emma Clark
  • Andrew Zisserman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11397)


We describe a method to automatically predict scoliosis in Dual-energy X-ray Absorptiometry (DXA) scans. We also show that intermediate representations, which in our case are segments of body parts, help improve performance. Hence, we propose a two step process for prediction: (i) we learn to segment body parts via a segmentation Convolutional Neural Network (CNN), which we show outperforms the noisy labels it was trained on, and (ii) we predict with a classification CNN that uses as input both the raw DXA scan and also the intermediate representation, i.e. the segmented body parts. We demonstrate that this two step process can predict scoliosis with high accuracy, and can also localize the spinal curves (i.e. geometry) without additional supervision. Furthermore, we also propose a soft score of scoliosis based on the classification CNN which correlates to the severity of scoliosis.



We are extremely grateful to all the families who took part in this study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses. The UK Medical Research Council and the Wellcome Trust (Grant ref: 102215/2/13/2) and the University of Bristol provide core support for ALSPAC. This publication is the work of the authors and Amir Jamaludin will serve as guarantor for the contents of this paper. This research was specifically funded by the British Scoliosis Research Foundation, and the DXA scans were funded through the Wellcome Trust (grants 084632 and 079960).


  1. 1.
    Asher, M.A., Burton, D.C.: Adolescent idiopathic scoliosis: natural history and long term treatment effects. Scoliosis 1(1), 2 (2006)CrossRefGoogle Scholar
  2. 2.
    Burkhart, T.A., Arthurs, K.L., Andrews, D.M.: Manual segmentation of DXA scan images results in reliable upper and lower extremity soft and rigid tissue mass estimates. J. Biomech. 42(8), 1138–1142 (2009)CrossRefGoogle Scholar
  3. 3.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process 10(2), 266–277 (2001). Scholar
  4. 4.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: delving deep into convolutional nets. In: Proceedings of BMVC (2014)Google Scholar
  5. 5.
    Clark, E.M., Tobias, J.H., Fairbank, J.: The impact of small spinal curves in adolescents who have not presented to secondary care: a population-based cohort study. Spine 41(10), E611–617 (2016)CrossRefGoogle Scholar
  6. 6.
    Jamaludin, A., Kadir, T., Zisserman, A.: SpineNet: automatically pinpointing classification evidence in spinal MRIs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 166–175. Springer, Cham (2016). Scholar
  7. 7.
    Jamaludin, A., Kadir, T., Zisserman, A.: SpineNet: automated classification and evidence visualization in spinal MRIs. Med. Image Anal. 41, 63–73 (2017)CrossRefGoogle Scholar
  8. 8.
    Pehrsson, K., Bake, B., Larsson, S., Nachemson, A.: Lung function in adult idiopathic scoliosis: a 20 year follow up. Thorax 46(7), 474–478 (1991)CrossRefGoogle Scholar
  9. 9.
    Roberts, M.G., Pacheco, E.M., Mohankumar, R., Cootes, T.F., Adams, J.E.: Detection of vertebral fractures in DXA VFA images using statistical models of appearance and a semi-automatic segmentation. Osteoporos. Int. 21(12), 2037–2046 (2010)CrossRefGoogle Scholar
  10. 10.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). Scholar
  11. 11.
    Shepherd, J.A., Ng, B.K., Fan, B., Schwartz, A.V., Cawthon, P., Cummings, S.R., Kritchevsky, S., Nevitt, M., Santanasto, A., Cootes, T.F.: Modeling the shape and composition of the human body using dual energy X-ray absorptiometry images. PLoS ONE 12(4), e0175857 (2017)CrossRefGoogle Scholar
  12. 12.
    Taylor, H.J., et al.: Identifying scoliosis in population-based cohorts: development and validation of a novel method based on total-body dual-energy x-ray absorptiometric scans. Calcif. Tissue Int. 92(6), 539–547 (2013)CrossRefGoogle Scholar
  13. 13.
    Vedaldi, A., Lenc, K.: MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of ACMM (2015)Google Scholar
  14. 14.
    Wu, J., Wang, Y., Xue, T., Sun, X., Freeman, W.T., Tenenbaum, J.B.: MarrNet: 3D shape reconstruction via 2.5D sketches. In: Advances in Neural Information Processing Systems (2017)Google Scholar
  15. 15.
    Wu, J., et al.: Single image 3D interpreter network. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 365–382. Springer, Cham (2016). Scholar
  16. 16.
    Zhang, J., Bargal, S.A., Lin, Z., Brandt, J., Shen, X., Sclaroff, S.: Top-down neural attention by excitation backprop. Int. J. Comput. Vis. 126, 1084–1102 (2017). Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Amir Jamaludin
    • 1
    Email author
  • Timor Kadir
    • 2
  • Emma Clark
    • 3
  • Andrew Zisserman
    • 1
  1. 1.VGG, Department of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.OptellumOxfordUK
  3. 3.Musculoskeletal Research Unit, School of Clinical SciencesUniversity of BristolBristolUK

Personalised recommendations