Towards Automated Multiscale Imaging and Analysis in TEM: Glomerulus Detection by Fusion of CNN and LBP Maps

  • Elisabeth WetzerEmail author
  • Joakim Lindblad
  • Ida-Maria Sintorn
  • Kjell Hultenby
  • Nataša Sladoje
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11134)


Glomerulal structures in kidney tissue have to be analysed at a nanometer scale for several medical diagnoses. They are therefore commonly imaged using Transmission Electron Microscopy. The high resolution produces large amounts of data and requires long acquisition time, which makes automated imaging and glomerulus detection a desired option. This paper presents a deep learning approach for Glomerulus detection, using two architectures, VGG16 (with batch normalization) and ResNet50. To enhance the performance over training based only on intensity images, multiple approaches to fuse the input with texture information encoded in local binary patterns of different scales have been evaluated. The results show a consistent improvement in Glomerulus detection when fusing texture-based trained networks with intensity-based ones at a late classification stage.


Glomerulus detection Transmission Electron Microscopy Convolutional Neural Networks Local binary patterns Digital pathology 



This work is supported by VINNOVA, MedTech4Health grants 2016-02329 and 2017-02447, the Ministry of Education, Science, and Techn. Development of the Rep. of Serbia (proj. ON174008 and III44006), and the Centre for Interdisciplinary Mathematics, Uppsala University.


  1. 1.
    Anwer, R.M., Khan, F.S., van de Weijer, J., Molinier, M., Laaksonen, J.: Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification. ISPRS J. Photogramm. Remote. Sens. 138, 74–85 (2018)CrossRefGoogle Scholar
  2. 2.
    Borg, I., Groenen, P.J.F.: Modern Multidimensional Scaling - Theory and Applications. Springer, New York (2005)zbMATHGoogle Scholar
  3. 3.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)Google Scholar
  4. 4.
    Fernández, A., Ghita, O., González, E., Bianconi, F., Whelan, P.F.: Evaluation of robustness against rotation of LBP, CCR and ILBP features in granite texture classification. Mach. Vis. Appl. 22(6), 913–926 (2011)CrossRefGoogle Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)Google Scholar
  6. 6.
    Juefei-Xu, F., Boddeti, V.N., Savvides, M.: Local binary convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 19–28. IEEE (2017)Google Scholar
  7. 7.
    Kruskal, J.B., Wish, M.: Multidimensional Scaling. Sage University Papers Series. Quantitative Applications in the Social Sciences, vol. 11, pp. 234–778 (1978)CrossRefGoogle Scholar
  8. 8.
    Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of ACM International Conference on Multimodal Interaction, pp. 503–510. ACM (2015)Google Scholar
  9. 9.
    Li, L., Feng, X., Xia, Z., Jiang, X., Hadid, A.: Face spoofing detection with local binary pattern network. J. Vis. Commun. Image Represent. 54, 182–192 (2018)CrossRefGoogle Scholar
  10. 10.
    Liu, L., Fieguth, P., Guo, Y., Wang, X., Pietikainen, M.: Local binary features for texture classification: taxonomy and experimental study. Patt. Rec. 62, 135–160 (2017)CrossRefGoogle Scholar
  11. 11.
    Lu, J., Liong, V.E., Zhou, X., Zhou, J.: Learning compact binary face descriptor for face recognition. IEEE Trans. PAMI 37(10), 2041–2056 (2015)CrossRefGoogle Scholar
  12. 12.
    Majtner, T., Yildirim-Yayilgan, S., Hardeberg, J.Y.: Combining deep learning and hand-crafted features for skin lesion classification. In: International Conference on Image Processing Theory Tools and Applications (IPTA), pp. 1–6. IEEE (2016)Google Scholar
  13. 13.
    Nahid, A.A., Kong, Y.: Histopathological breast-image classification using local and frequency domains by convolutional neural network. Information 9(1), 19 (2018)CrossRefGoogle Scholar
  14. 14.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. PAMI 24(7), 971–987 (2002)CrossRefGoogle Scholar
  15. 15.
    Pietikainen, M., Hadid, A., Zhao, G., Ahonen, T.: Computer Vision Using Local Binary Patterns. Springer, London (2011)CrossRefGoogle Scholar
  16. 16.
    Rezaeilouyeh, H., Mollahosseini, A., Mahoor, M.H.: Microscopic medical image classification framework via deep learning and shearlet transform. J. Med. Imaging 3(4), 044501 (2016)CrossRefGoogle Scholar
  17. 17.
    Sadanandan, S.K., Ranefall, P., Wählby, C.: Feature augmented deep neural networks for segmentation of cells. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 231–243. Springer, Cham (2016). Scholar
  18. 18.
    Simon, O., Yacoub, R., Jain, S., Tomaszewski, J.E., Sarder, P.: Multi-radial LBP features as a tool for rapid glomerular detection and assessment in whole slide histopathology images. Sci. Rep. 8(1), 2032 (2018)CrossRefGoogle Scholar
  19. 19.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Uppsala UniversityUppsalaSweden
  2. 2.Karolinska InstituteSolnaSweden
  3. 3.Vironova ABStockholmSweden
  4. 4.Mathematical Institute of Serbian Academy of Sciences and ArtsBelgradeSerbia

Personalised recommendations