Advertisement

Multi-task Localization and Segmentation for X-Ray Guided Planning in Knee Surgery

  • Florian KordonEmail author
  • Peter Fischer
  • Maxim Privalov
  • Benedict Swartman
  • Marc Schnetzke
  • Jochen Franke
  • Ruxandra Lasowski
  • Andreas Maier
  • Holger Kunze
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11769)

Abstract

X-ray based measurement and guidance are commonly used tools in orthopaedic surgery to facilitate a minimally invasive workflow. Typically, a surgical planning is first performed using knowledge of bone morphology and anatomical landmarks. Information about bone location then serves as a prior for registration during overlay of the planning on intra-operative X-ray images. Performing these steps manually however is prone to intra-rater/inter-rater variability and increases task complexity for the surgeon. To remedy these issues, we propose an automatic framework for planning and subsequent overlay. We evaluate it on the example of femoral drill site planning for medial patellofemoral ligament reconstruction surgery. A deep multi-task stacked hourglass network is trained on 149 conventional lateral X-ray images to jointly localize two femoral landmarks, to predict a region of interest for the posterior femoral cortex tangent line, and to perform semantic segmentation of the femur, patella, tibia, and fibula with adaptive task complexity weighting. On 38 clinical test images the framework achieves a median localization error of 1.50 mm for the femoral drill site and mean IOU scores of 0.99, 0.97, 0.98, and 0.96 for the femur, patella, tibia, and fibula respectively. The demonstrated approach consistently performs surgical planning at expert-level precision without the need for manual correction.

Keywords

Landmark localization Multi-label bone segmentation MPFL X-ray guidance Orthopaedics Surgical planning 

References

  1. 1.
    Bier, B., et al.: X-ray-transform invariant anatomical landmark detection for pelvic trauma surgery. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) Medical Image Computing and Computer Assisted Intervention - MICCAI 2018. LNCS, vol. 11073, pp. 55–63. Springer, Cham (2018)CrossRefGoogle Scholar
  2. 2.
    Breininger, K., et al.: Multiple device segmentation for fluoroscopic imaging using multi-task learning. In: Stoyanov, D., et al. (eds.) LABELS/CVII/STENT -2018. LNCS, vol. 11043, pp. 19–27. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01364-6_3CrossRefGoogle Scholar
  3. 3.
    Chen, Z., Badrinarayanan, V., Lee, C., Rabinovich, A.: Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: Proceedings of International Conference Machine Learning, pp. 793–802 (2018)Google Scholar
  4. 4.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_38CrossRefGoogle Scholar
  5. 5.
    Joskowicz, L., Hazan, E.J.: Computer aided orthopaedic surgery: Incremental shift or paradigm change? Med. Image Anal. 33, 84–90 (2016)CrossRefGoogle Scholar
  6. 6.
    Kordon, F., Lasowski, R., Swartman, B., Franke, J., Fischer, P., Kunze, H.: Improved X-ray bone segmentation by normalization and augmentation strategies. In: Handels, H., Deserno, T., Maier, A., Maier-Hein, K., Palm, C., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2019. I, pp. 104–109. Springer, Wiesbaden (2019).  https://doi.org/10.1007/978-3-658-25326-4_24CrossRefGoogle Scholar
  7. 7.
    Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision - European Conference on Computer Vision 2016, pp. 483–499. Springer, Cham (2016)CrossRefGoogle Scholar
  8. 8.
    Schöttle, P.B., Schmeling, A., Rosenstiel, N., Weiler, A.: Radiographic landmarks for femoral tunnel placement in medial patellofemoral ligament reconstruction. Am. J. Sports Med. 35(5), 801–804 (2007)CrossRefGoogle Scholar
  9. 9.
    Székely, G., Nolte, L.P.: Image guidance in orthopaedics and traumatology: a historical perspective. Med. Image Anal. 33, 79–83 (2016)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Florian Kordon
    • 1
    • 2
    • 3
    Email author
  • Peter Fischer
    • 1
    • 2
  • Maxim Privalov
    • 4
  • Benedict Swartman
    • 4
  • Marc Schnetzke
    • 4
  • Jochen Franke
    • 4
  • Ruxandra Lasowski
    • 3
  • Andreas Maier
    • 1
  • Holger Kunze
    • 2
  1. 1.Pattern Recognition Lab, Department of Computer ScienceFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Advanced TherapiesSiemens Healthcare GmbHForchheimGermany
  3. 3.Faculty of Digital MediaHochschule FurtwangenFurtwangenGermany
  4. 4.Department for Trauma and Orthopaedic SurgeryBG Trauma Center LudwigshafenLudwigshafenGermany

Personalised recommendations