Understanding Historical Cityscapes from Aerial Imagery Through Machine Learning

  • Evangelos MaltezosEmail author
  • Eftychios Protopapadakis
  • Nikolaos Doulamis
  • Anastasios Doulamis
  • Charalabos Ioannidis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11196)


Understanding cityscapes using remote sensing data has been an active research field for more than two decades. Meanwhile, machine learning provides generalization capabilities compared to hierarchical and rule-based methods. This paper evaluates several machine learning algorithms in order to fuse shadow detection and shadow compensation methods for building detection using high resolution aerial imagery. Three complex and real-life urban study areas were used as test datasets with various: (i) kinds of buildings structures of special architecture, (ii) pixel resolutions and, (iii) types of data. Objective evaluation metrics have been used for assessing the compared algorithms such recall, precision and F1-score as well as rates of completeness, correctness and quality. For both approaches, i.e., shadow detection and building detection, the computational complexity of each machine learning algorithm was examined. The results indicate that deep learning schemes, such a Convolutional Neural Network (CNN), provides the best classification performance in terms of shadow detection and building detection.


Shadow detection Shadow compensation Building detection Point cloud Machine learning 



This research is supported by the European Funded Project of H2020, Terpsichore, under agreement no. 691218.


  1. 1.
    Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR, Seattle, WA, USA (2016)Google Scholar
  2. 2.
    Maltezos, Ε., Doulamis, Ν., Doulamis, A., Ioannidis, C.: Deep convolutional neural networks for building extraction from orthoimages and dense image matching point clouds. J. Appl. Remote Sens. 11(4), 042620-1–042620-22 (2017)CrossRefGoogle Scholar
  3. 3.
    Rottensteiner, F., Sohn, G., Gerke, M., Wegner, J.D., Breitkopf, U., Jung, J.: Results of the ISPRS benchmark on urban object detection and 3D building reconstruction. ISPRS J. Photogramm. Remote Sens. 93, 256–271 (2014)CrossRefGoogle Scholar
  4. 4.
    Doulamis, A., et al.: 5D modelling: an efficient approach for creating spatiotemporal predictive 3D maps of large-scale cultural resources. In: ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences (PRSSIS), II5/W3, pp. 61–68 (2015)CrossRefGoogle Scholar
  5. 5.
    Lorenzi, L., Melgani, F., Mercier, G.: A complete processing chain for shadow detection and reconstruction in VHR images. IEEE TGARS 50(9), 3440–3452 (2012)Google Scholar
  6. 6.
    Song, H., Huang, B., Zhang, K.: Shadow detection and reconstruction in high-resolution satellite images via morphological filtering and example-based learning. IEEE TGARS 52(5), 2545–2554 (2013)Google Scholar
  7. 7.
    Guislain, M., Digne, J., Chaine, R., Kudelski, D., Lefebvre-Albaret, P.: Detecting and correcting shadows in urban point clouds and image collections. In: 3DV, pp. 1–9 (2016)Google Scholar
  8. 8.
    Hosseinzadeh, S., Shakeri, M., Hong, Z.: Fast shadow detection from a single image using a patched convolutional neural network. In: CVPR, Seattle, Honolulu, Hawaii (2017)Google Scholar
  9. 9.
    Luo, L., et al.: Automated extraction of the archaeological tops of qanat shafts from VHR imagery in Google Earth. Remote Sensing 6, 11956–11976 (2014)CrossRefGoogle Scholar
  10. 10.
    Doulamis, A., et al.: 4D reconstruction of the past. In: SPIE Proceedings, vol. 8795, pp. 1–11 (2013)Google Scholar
  11. 11.
    Kadhim, N., Mourshed, M., Bray, M.: Automatic extraction of urban structures based on shadow information from satellite imagery. In: 14th Conference of International Building Performance Simulation, pp. 2607–2614 (2015)Google Scholar
  12. 12.
    Cerra, D., Plank, S., Lysandrou, V., Tian, J.: Cultural heritage sites in danger-towards automatic damage detection from space. Remote Sens. 8(781), 1–15 (2016)Google Scholar
  13. 13.
    Zhou, K., Gorte, B.: Shadow detection from VHR aerial images in urban area by using 3D city models and a decision fusion approach. In: ISPRS Annals of the PRSSIS, XLII-2/W7, pp. 579–586 (2017)CrossRefGoogle Scholar
  14. 14.
    Kadhim, N., Mourshed, M.: A shadow-overlapping algorithm for estimating building heights from VHR satellite images. IEEE GRSL 15(1), 8–12 (2018)Google Scholar
  15. 15.
    Llamas, J., Lerones, P.M., Medina, R., Zalama, E., Gómez-García-Bermejo, J.: Classification of architectural heritage images using deep learning techniques. Appl. Sci. 7(992), 1–25 (2017)Google Scholar
  16. 16.
    Yasser, A.M., Clawson, K., Bowerman, C., Lévêque, M.: Saving cultural heritage with digital make-believe: machine learning and digital techniques to the rescue. In: Proceedings of British HCI Conference, pp. 1–5 (2017)Google Scholar
  17. 17.
    Bassier, M., Vergauwen, M., Van Genechten, B.: Automated classification of heritage buildings for as-built BIM using machine learning techniques. In: ISPRS Annals of the PRSSIS, IV-2/W2, pp. 25–30 (2017)CrossRefGoogle Scholar
  18. 18.
    Uhl, J., Leyk, S., Chiang, Y., Duan, W., Knoblock, C.: Exploring the potential of deep learning for settlement symbol extraction from historical map documents. In: UCGIS (2018)Google Scholar
  19. 19.
    Cheng, G., Han, J.: A survey on object detection in optical remote sensing images. ISPRS J. Photogramm. Remote Sens. 117, 11–28 (2016)CrossRefGoogle Scholar
  20. 20.
    Bhatia, N., Vandana: Survey of nearest neighbor techniques. Int. J. Comput. Sci. Inf. Secur. 8(2), 302–305 (2010)Google Scholar
  21. 21.
    Rokach, L., Schclar, A., Itach, E.: Ensemble methods for multi-label classification. Expert Syst. Appl. 41(16), 7507–7523 (2014)CrossRefGoogle Scholar
  22. 22.
    Farid, D.M., Zhang, L., Rahman, C.M., Hossain, M.A., Strachan, R.: Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks. Expert Syst. Appl. 41(4), 1937–1946 (2014)CrossRefGoogle Scholar
  23. 23.
    Zhuang, F., et al.: Mining distinction and commonality across multiple domains using generative model for text classification. IEEE TKDE 24(11), 2025–2039 (2012)Google Scholar
  24. 24.
    Belgiu, M., Drăguţ, L.: Random forest in remote sensing: a review of applications and future directions. ISPRS J. Photogramm. Remote Sens. 114, 24–31 (2016)CrossRefGoogle Scholar
  25. 25.
    Abe, S.: Support Vector Machines for Pattern Classification. Advances in Computer Vision and Pattern Recognition. Springer, London (2010). Scholar
  26. 26.
    Protopapadakis, E., et al.: A genetically optimized neural classifier applied to numerical pile integrity tests considering concrete piles. Comput. Struct. 162, 68–79 (2016)CrossRefGoogle Scholar
  27. 27.
    Remondino, F., Spera, M.G., Nocerino, E., Menna, F., Nex., F., Barsanti, S.G.: Dense image matching: comparisons and analyses. In: Proceedings of Digital Heritage International Congress, pp. 47–54 (2013)Google Scholar
  28. 28.
    Rovithis, Ε., et al.: LiDAR-aided urban-scale assessment of soil-structure interaction effects: the case of Kalochori residential area (N. Greece). Bull. Earthq. Eng. 15(11), 4821–4850 (2017)CrossRefGoogle Scholar
  29. 29.
    Singh, K.K., Pal, K., Nigam, M.J.: Shadow detection and removal from remote sensing images using NDI and morphological operators. Int. J. Comput. Appl. 42(10), 37–40 (2012)Google Scholar
  30. 30.
    Rottensteiner, F., Sohn, G., Gerke, M., Wegner, J.D.: ISPRS test project on urban classification and 3D building reconstruction, ISPRS—Commission III—Photogrammetric Computer Vision and Image Analysis Working Group III/4–3D Scene Analysis (2013)Google Scholar
  31. 31.
    Cramer, M.: The DGPF test on digital aerial camera evaluation – overview and test design. Photogrammetrie – Fernerkundung – Geoinformation 2, 73–82 (2010). Scholar
  32. 32.
    Rutzinger, M., Rottensteiner, F., Pfeifer, N.: A comparison of evaluation techniques for building extraction from airborne laser scanning. IEEE J-STARS 2(1), 11–20 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Evangelos Maltezos
    • 1
    Email author
  • Eftychios Protopapadakis
    • 1
  • Nikolaos Doulamis
    • 1
  • Anastasios Doulamis
    • 1
  • Charalabos Ioannidis
    • 1
  1. 1.Laboratory of Photogrammetry, School of Rural and Surveying EngineeringNational Technical University of AthensAthensGreece

Personalised recommendations