Unsupervised pathology detection in medical images using conditional variational autoencoders

  • Hristina UzunovaEmail author
  • Sandra Schultz
  • Heinz Handels
  • Jan Ehrhardt
Original Article



Pathology detection in medical image data is an important but a rather complicated task. In particular, the big variability of the pathologies is a challenge to automatic detection methods and even to machine learning methods. Supervised algorithms would usually learn the appearance of a single pathological structure based on a large annotated dataset. As such data is not usually available, especially in large amounts, in this work we pursue a different unsupervised approach.


Our method is based on learning the entire variability of healthy data and detect pathologies by their differences to the learned norm. For this purpose, we use conditional variational autoencoders which learn the reconstruction and encoding distribution of healthy images and also have the ability to integrate certain prior knowledge about the data (condition).


Our experiments on different 2D and 3D datasets show that the approach is suitable for the detection of pathologies and deliver reasonable Dice coefficients and AUCs. Also this method can estimate missing correspondences in pathological images and thus can be used as a pre-step to a registration method. Our experiments show improving registration results on pathological data when using this approach.


Overall the presented approach is suitable for a rough pathology detection in medical images and can be successfully used as a preprocessing step to other image processing methods.


Conditional variational autoencoder Unsupervised pathology detection Image registration 



This work is supported by the German Research Foundation (DFG: HA 2355/7-2).

Compliance with ethical standards

Conflict of interest

Hristina Uzunova, Sandra Schultz, Heinz Handels and Jan Ehrhardt declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

For this type of study, formal consent is not required.


  1. 1.
    Bao J, Chen D, Wen F, Li H, Hua G (2017) CVAE-GAN: fine-grained image generation through asymmetric training. In: IEEE international conference on computer vision, ICCV 2017, Venice, Italy, October 22–29, 2017, pp 2764–2773Google Scholar
  2. 2.
    Ehrhardt J, Schmidt-Richberg A, Werner R, Handels H (2015) Variational registration—a flexible open-source ITK toolbox for nonrigid image registration. In: Bildverarbeitung Für Die Medizin 2015, Informatik aktuell, Lübeck, pp 209–214Google Scholar
  3. 3.
    Faktor A, Irani M (2012) Clustering by Composition—unsupervised discovery of image categories. In: Fitzgibbon A, Lazebnik S, Perona P, Sato Y, Schmid C (eds) Computer vision—ECCV 2012. Lecture notes in computer science, vol 7578. Springer, Berlin, Heidelber,Google Scholar
  4. 4.
    Kamnitsas K, Ferrante E, Parisot S, Ledig C, Nori AV, Criminisi A, Rueckert D, Glocker B (2016) Deepmedic forbrain tumor segmentation. In: BrainLes 2016, with the challenges on BRATS, ISLES and mTOP, MICCAI, pp 138–149Google Scholar
  5. 5.
    Kingma DP, Welling M (2013) Auto-encoding variational bayes. In: Proceedings of the 2nd international conference on learning representations—ICLRGoogle Scholar
  6. 6.
    Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang MC, Christensen GE, Collins DL, Gee J, Hellier P, Song JH, Jenkinson M, Lepage C, Rueckert D, Thompson P, Vercauteren T, Woods RP, Mann JJ, Parsey RV (2009) Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuroimage 46(3):786–802CrossRefGoogle Scholar
  7. 7.
    Krüger J, Ehrhardt J, Handels H (2015) Probabilistic appearance models for segmentation and classification. In: International conference on computer vision—ICCV, pp 1698–1706Google Scholar
  8. 8.
    Larsen ABL, Sønderby SK, Larochelle H, Winther O (2016) Autoencoding beyond pixels using a learned similarity metric. In: Proceedings of the 33rd international conference on international conference on machine learning, vol 48, pp 1558–1566Google Scholar
  9. 9.
    Litjens GJS, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRefGoogle Scholar
  10. 10.
    Lotan O, Irani M (2016) Needle-match: Reliable patch matching under high uncertainty. In: IEEE conference on computer vision and pattern recognition—CVPRGoogle Scholar
  11. 11.
    Maier O, Menze BH, von der Gablentz J, Häni L, Heinrich MP, Liebrand M, Winzeck S, Basit A, Bentley P, Chen L, Christiaens D, Dutil F, Egger K, Feng C, Glocker B, Götz M, Haeck T, Halme HL, Havaei M, Iftekharuddin KM, Jodoin PM, Kamnitsas K, Kellner E, Korvenoja A, Larochelle H, Ledig C, Lee JH, Maes F, Mahmood Q, Maier-Hein KH, McKinley R, Muschelli J, Pal C, Pei L, Rangarajan JR, Reza SM, Robben D, Rueckert D, Salli E, Suetens P, Wang CW, Wilms M, Kirschke JS, Krämer UM, Münte TF, Schramm P, Wiest R, Handels H, Reyes M (2017) Isles 2015—a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Med Image Anal 35:250–269CrossRefGoogle Scholar
  12. 12.
    McKinley R, Häni L, Wiest R, Reyes M (2016) Segmenting the ischemic penumbra: a decision forest approach with automatic threshold finding. Springer, Berlin, pp 275–283Google Scholar
  13. 13.
    Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner ER, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp C, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SMS, Ryan MT, Sarikaya D, Schwartz LH, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Székely G, Taylor TJ, Thomas OM, Tustison NJ, Ünal GB, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Leemput KV (2015) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging 34(10):1993–2024CrossRefGoogle Scholar
  14. 14.
    Pawlowski N, Lee MCH, Rajchl M, McDonagh S, Ferrante E, Kamnitsas K, Cooke S, Stevenson SK, Khetani AM, Newman T, Zeiler FA, Digby RJ, Coles JP, Rueckert D, Menon DK, Newcombe VFJ, Glocker B (2018) Unsupervised lesion detection in brain CT using bayesian convolutional autoencoders. In: Medical imaging with deep learning—MIDLGoogle Scholar
  15. 15.
    Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Information processing in medical imaging—IPMI, pp 146–157Google Scholar
  16. 16.
    Shattuck DW, Mirza M, Adisetiyo V, Hojatkashani C, Salamon G, Narr KL, Poldrack RA, Bilder RM, Toga AW (2008) Construction of a 3d probabilistic atlas of human cortical structures. NeuroImage 39(3):1064–1080CrossRefGoogle Scholar
  17. 17.
    Shechtman E, Irani M (2007) Matching local self-similarities across images and videos. In: IEEE conference on computer vision and pattern recognition—CVPRGoogle Scholar
  18. 18.
    Sohn K, Lee H, Yan X (2015) Learning structured output representation using deep conditional generative models. Adv Neural Inf Process Syst 28:3483–3491Google Scholar
  19. 19.
    Uzunova H, Handels H, Ehrhardt J (2018) Unsupervised pathology detection in medical images using learning-based methods. Bildverarbeitung für die Medizin 2018. Springer, Berlin, pp 61–66CrossRefGoogle Scholar
  20. 20.
    Werner R, Schmidt-Richberg A, Handels H, Ehrhardt J (2014) Estimation of lung motion fields in 4D CT data by variational non-linear intensity-based registration: a comparison and evaluation study. Phys Med Biol 59(15):4247–4260CrossRefGoogle Scholar
  21. 21.
    Xue W, Islam A, Bhaduri M, Li S (2017) Direct multitype cardiac indices estimation via joint representation and regression learning. IEEE Trans Med Imaging 36(10):2057–2067CrossRefGoogle Scholar
  22. 22.
    Yan X, Yang J, Sohn K, Lee H (2016) Attribute2image: conditional image generation from visual attributes. In: European conference on computer vision—ECCV, vol 9908. Springer, pp 776–791Google Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  1. 1.Institute of Medical InformaticsUniversity of LübeckLübeckGermany

Personalised recommendations