UGESCO - A Hybrid Platform for Geo-Temporal Enrichment of Digital Photo Collections Based on Computational and Crowdsourced Metadata Generation

  • Steven VerstocktEmail author
  • Samnang Nop
  • Florian Vandecasteele
  • Tim Baert
  • Nico Van de Weghe
  • Hans Paulussen
  • Ettore Rizza
  • Mathieu Roeges
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11196)


The majority of digital photo collections at museums, archives and libraries are facing (meta) data problems that impact their interpretation, exploration and exploitation. In most cases, links between collection items are only supported at the highest level, which limits the item’s searchability and makes it difficult to generate scientific added value out of it or to use the collections in new end-user focused applications. The geo-temporal metadata enrichment tools that are proposed in this paper tackle these issues by extending and linking the existing collection items and by facilitating their spatio-temporal mapping for interactive querying. To further optimize the quality of the temporal and spatial annotations that are retrieved by our automatic enrichment tools, we also propose some crowdsourced microtasks to validate and improve the generated metadata. This crowdsourced input on its turn can be used to further optimize (and retrain) the automatic enrichments. Finally, in order to facilitate the querying of the data, new geo-temporal mapping services are investigated. These services facilitate cross-collection studies in time and space and ease the scientific interpretation of the collection items in a broader sense.


Digital heritage collections Geo-temporal mapping Metadata enrichment Microtask crowdsourcing Named entity recognition Rephotography 


  1. 1.
    Qiang, Y., Valcke, M., De Maeyer, P., Van de Weghe, N.: Representing time intervals in a two-dimensional space: an empirical study. J. Vis. Lang. Comput. 25(4), 466–480 (2014)CrossRefGoogle Scholar
  2. 2.
    Cham, T.J., Ciptadi, A., Tan, W.C., Pham, M.T., Chia, L.T.: Estimating camera pose from a single urban ground-view omnidirectional image and a 2D building outline map. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, US, pp. 366–373 (2010)Google Scholar
  3. 3.
    Arth, C., Reitmayr, G., Schmalstieg, D.: Full 6DOF pose estimation from geo-located images. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7726, pp. 705–717. Springer, Heidelberg (2013). Scholar
  4. 4.
    Gronat, P., Havlena, M., Sivic, J., Pajdla, T.: Building streetview datasets for place recognition and city reconstruction. Technical report, CTU-CMP-2011-16, pp. 1–13 (2011)Google Scholar
  5. 5.
    Verstockt, S., Gerke, M., Kerle, N.: Geolocalization of crowdsourced images for 3D modeling of city points of interest. Geosci. Remote Sens. Lett. 12(8), 1670–1674 (2015)CrossRefGoogle Scholar
  6. 6.
    Kelm, P., Schmiedeke, S., Cluver, K., Sikora, T.: Automatic geo-referencing of Flickr videos. In: eBook and USB Produced by Sigma Orionis, p. 32 (2011)Google Scholar
  7. 7.
    Zhou, B., Lapedriza, A., Khosla, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2018)CrossRefGoogle Scholar
  8. 8.
    Wang, L., Guo, S., Huang, W., Qiao, Y.: Places205-VGGNet models for scene recognition. arXiv:1508.01667 (2015)
  9. 9.
    Bengio, S., Dean, J., Erhan, D., Rabinovich, A., Shlens, J., Singer, Y.: Using web co-occurrence statistics for improving image categorization. arXiv:1312.5697v2 (2013)
  10. 10.
    Rao, D., McNamee, P., Dredze, M.: Entity linking: finding extracted entities in a knowledge base. In: Poibeau, T., Saggion, H., Piskorski, J., Yangarber, R. (eds.) Multi-source, Multilingual Information Extraction and Summarization, pp. 93–115. Springer, Heidelberg (2013). Scholar
  11. 11.
    Makantasis, K., Doulamis, A., Doulamis, N., Ioannides, M.: In the wild image retrieval and clustering for 3D cultural heritage landmarks reconstruction. Multimed. Tools Appl. 75(7), 3593–3629 (2016)CrossRefGoogle Scholar
  12. 12.
    Chen, N., Prasanna, V.K.: Semantic image clustering using object relation network. In: Hu, S.-M., Martin, R.R. (eds.) CVM 2012. LNCS, vol. 7633, pp. 59–66. Springer, Heidelberg (2012). Scholar
  13. 13.
    Lu, Y., et al.: GeoUGV: user-generated mobile video dataset with fine granularity spatial metadata. In: Proceedings of the 7th International Conference on Multimedia Systems, pp. 43–46, New York, USA (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Steven Verstockt
    • 1
    Email author
  • Samnang Nop
    • 1
  • Florian Vandecasteele
    • 1
  • Tim Baert
    • 2
  • Nico Van de Weghe
    • 2
  • Hans Paulussen
    • 3
  • Ettore Rizza
    • 4
  • Mathieu Roeges
    • 5
  1. 1.IDLabGhent University-ImecGhentBelgium
  2. 2.CartoGIS, Department of GeographyGhent UniversityGhentBelgium
  3. 3.KU Leuven and ImecKortrijkBelgium
  4. 4.Information and Communication Science DepartmentUniversité libre de BruxellesBrusselsBelgium
  5. 5.Direction opérationnelle CegeSoma/Archives de l’Etat DO4BrusselsBelgium

Personalised recommendations